When you meet the set of scale that Reddit looks with over 300 million recurrent active users making 5 million comments and a huge 40 million searches every day across a larger than a million communities, it’s a daunting job to find a search tool to manage that kind of volume.
The test with Reddit extends past indexing these massive numbers. They also hold to deal with the wide type of content with text, gifs, images, and video by the composition. While part of the goal was to develop common search functionality and produce more relevant results, maybe even more critically, they needed a tool to help users surface the problems that interest them without producing to explicitly state it in the search box, Nick Caldwell, VP of design and Engineering at Reddit explained.
“I think that users who come to any site, and Reddit in distinct, prefer an activity where they don’t have to do old-fashioned keyword entry, but want a constant stream of interesting content,” he said.
Reddit’s search engine had really been particularly bad and Caldwell made upgrading it a priority. “One of the points I wanted to do when I started at Reddit, was I required to fix search. People have been griping about it for five years,” he said.
Part of the problem up to that point wasn’t a lack of passion to improve the search experience. Everyone knew the issue but getting the time to update it was another matter. When Caldwell came on board, Reddit had a tiny team of 40 engineers, whose first job was managing a site of this size and scope up and running.
Caldwell said that the organization went with the Lucidworks Fusion platform because it had the right blend of technology and the ability to expand his engineering team while serving search to constantly evolve on Reddit. Getting a tool was only part of the answer though. Reddit also wanted to hire a group of engineers with what Caldwell called “world class research and relevance engineering expertise.” To that end, he has set up a 30-person engineering research team devoted to maximizing the potential of the new search platform.
Take your time to comment on this article.
