Friday, October 06, 2006

Netflix Challenge

You might have seen Netflix’ announcement about challenging all of us data driven Marketers: Who can come up with a better consumer recommendation algorithm for Netflix’ consumer base by analyzing the publicly available data set of over 100 Million consumer recommendation? The winner will receive $1 Million. Doesn’t this sound like real fun? Check out details at www.netflixprize.com.

Besides my true excitement for this analytical and intellectual challenge, I believe that Netflix is pursuing not just an interesting self serving marketing trick but positions itself as a pioneer of solving intricate marketing problems. Putting consumer data into the public domain (while protecting consumer’s privacy) to request solutions from all over the world without constraints will not just solve an individual Netflix specific problem but it will enrich the overall wisdom of our marketing community. Especially since Netflix promised to publish the winner’s approach and methodology.

A couple of months ago I outlined the vision of companies who would publish the results of their marketing programs. The intent would be to enlist interested individuals to improve the relevance and effectiveness of their programs. Netflix’ approach starts earlier before any marketing program is even designed. It starts at defining the data driven marketing problem with clear expectation of what a successful solution needs to accomplish. Four key elements make this approach for me so attractive:
  • Setting up the marketing problem in a well described manner
  • Publishing Customer Data into the Public Domain
  • Defining Success criteria with a clear expectation of what a solution needs to achieve
  • Publishing the most successful solution

How does one or a team solve this or any similar analytical marketing problem? I believe its solution does not solely reside within the most sophisticated data mining or mathematical tools approach (Netflix believes that this is a pure machine learning challenge, I dare to disagree) but by looking at the described challenge in an unconventional manner. The winning methodology needs to combine analytical intelligence, tools, and methodologies with rephrasing the challenge. How? My bet is on an unique combination of analytical smartness with several deep qualitative insights into the mind of the DVD renting consumer, this is true Consumer Intelligence. And it’s definitely worth $1Million. But please keep in mind, it’s less about the money than about changing of how we solve data and insight based marketing problems.

6 Comments:

Anonymous Anonymous said...

Michael:

We just signed up for this (being gluttons for punishments, or thrillseekers in data mining -- you make the call :-). I've posted some interesting initial findings on my blog: http://sandeep-giri.blogspot.com/2006/10/how-to-win-1-million-from-netflix.html

I agree with your assessment that trying to solve this with ratings data alone might not be the best way to go. There seem to be so many other interesting dimensions that should influence somone's movie rating: movie characteristics like the cast, director, etc., review from critics, local media review, geo/demographic information about the Netflix member, among others. None of these are being considered in the current algorithm. I can understand Netflix's hesitancy to interface with 3rd party resources, but perhaps they should make all the datapoints within Netflix's movie database available for this contest -- and second, encourage contestants to add their own qualitative datapoints. If the goal is to approach this as a pure improvement of a data mining problem -- then increasing the depth of data should help.

- Sandeep

11:49 PM  
Anonymous Anonymous said...

i'm rooting for you guys at Loyalty Matrix!

I like Michael's perspective that this is more than just a machine learning challenge. Is there some way to generate meaningful data from consumers themselves outside of the provided dataset? For example, could you recruit a large enough panel to answer detailed questions about movie preferences that can be somehow tied to the provided dataset to get better results? You could syndicate a portion of the reward (or at least the bragging rights) for the users who contribute the most meaningfully. How can we get a swarm of users to take this beyond a pure analytics/modeling exercise?

Glad to add some crazy ideas to your blog Michael. Thanks for highlighting this!

11:18 PM  
Anonymous Anonymous said...

Here's the update on the contest, with a group already beating the control during week 1.

http://slashdot.org/articles/06/10/09/1344235.shtml
http://www.cse.ucsc.edu/~yiz/

3:54 PM  
Anonymous seslisohbet said...

Thank you for sharing a nice article.
Congratulations on your posts, not go from your site successful.

5:25 AM  
Blogger Th Brijesh Pratap Singh said...

When you love what you do then work doesn’t seem like work!If creativity is what you are all about and if animation isyour first love, then rush to ZICA today! And let yourpassion lead you!Mercedes has set the pace in taking initiatives. In August, it signed up as the automotive partner for all motor sporting events at the Budhh International Circuit in Greater Noida for two years. The Mercedes SLS AMG-with top speed of 317 km/hr-will be the 'safety car' for all racing events at the circuit. It will also provide cars to race officials.

Hyundai Alternator

5:55 AM  
Anonymous Anonymous said...

I will be your frequent visitor, that's for sure. pain relief Read a useful article about tramadol tramadol

4:19 AM  

Post a Comment

<< Home