Netflix Embraces the Wisdom of the Crowd
Netflix has announced a major new initiative to draw on the "wisdom of crowds," as James Surowiecki puts it, to help answer some of its marketing questions.
According to its site, "The Netflix Prize seeks to substantially improve the accuracy of predictions about how much someone is going to love a movie based on their movie preferences. Improve it enough and you win a prize. The company is giving away $1 million, and has planned a $50,000 Progress Prize each year in addition to a grand prize for the research team that meets the criteria of the competition.
The company says that they will "provide you with a lot of anonymous rating data, and a prediction accuracy bar that is 10% better than what Cinematch can do on the same training data set...If you develop a system that we judge most beats that bar on the qualifying test we provide, you get serious money and bragging rights" but only if you share the method with Netflix and the world and explain why it works.
The company says, "We suspect the 10% improvement is pretty tough, but we also think there is a good chance it can be achieved. It may take months; it might take years...So if you know (or want to learn) something about machine learning and recommendations systems, give it a shot. We could really make it worth your while."
The blogosphere is already alive with commentary. Harry Chen, who thinks aloud, gives a fairly astute demonstration as to what other user attributes must be considered in Netflix's data to truly understand and be able to make better recommendations. Among these astute observations is a key one: "people share Netflix accounts." Chen writes of him and his wife, "It's inappropriate to consider our combined ratings as the ratings of a single person. Just because I like action movies and my wife likes comedies, one can't conclude with full confidence that we as a single Netflix account user like both action movies and comedies."
His suggestion reminds me of the problems Henry Jenkins said he had with Amazon's recommendations after he, his wife, and his son had used the site regularly, and after Henry had used the site to locate several books during various research projects that were outside of his realm of personal interest.
But researchers are cropping up and grouping together across the country to prepare for the event, including John Resig, who writes on his blog, "I don't think I could possibly be any more giddy about something, than how I am concerning The Netflix Prize." The amount of intellectual capital that the company may become privy to during this contest demonstrates the power of a collective intelligence, as Henry Jenkins writes about. And, with people saying things like, "First, I have to generate my test bed and get to work this is so cool. I don'tk now what it is with me and large, nicely formatted, datasets, but I don't think there's anything that can get me more excited," they've certainly hit a research nerve with a section of Internet users.
The discussion really got going after Monday's New York Times piece by Katie Hafner about the contest. According to the story, "Computer scientists say that after years of steady progress in this field, there has been a slowdown--which is what Netflix executives say prompted them to offer the problem to a wide audience for solution."
And now other industries are calling for a similar approach to these problems. Consider these comments from LibrarianInBlack: "Can you imagine what would happen if III or other ILS vendors conducted a similar contest? Make our relevancy ranking work better, please. Reduce the clunk and clutter in our code, please. Add RSS to our services, please. Make our products more usable and clearer, please."
Could Netflix cause a change in the way companies think about researching complex questions? Or could this be forgotten in a couple of months? We shall see...