Netflix (NFLX) Boosts Prize Economics

By at least one metric – the number of people who have mentioned it to me – my brief post about Netflix (NASDAQ:NFLX) appears to be my most popular one so far.

The post linked to a remarkable slide deck about the corporate culture that Netflix has embraced in its quest for excellence. Most memorable line: “adequate performance gets a generous severance package.” If you haven’t seen it, I encourage you to click on over. It’s worth your time.

Yesterday’s award of the first Netflix prize highlights another strength of Netflix’s culture: it clearly does not suffer from “not-invented-here” syndrome. Indeed, quite the reverse. A few years ago, Netflix realized that it had reached its limit in trying to improve the accuracy of its movie recommendation system. Even though users may rate dozens (or more) movies, it turns out to be difficult to predict what other movies they will like.

So Netflix decided to outsource this problem in an ingenious way: it offered a $1 million prize to any person or team that could improve the recommendation algorithm by at least 10%. Stated that way, the problem seems deceptively easy. But it took nearly three years before the winner – a team led by AT&T Research engineers – took home the prize.

As recounted in Netflix’s press release, this marathon ended in a race to the wire:

“We had a bona fide race right to the very end,” said [CEO Reed] Hastings. “Teams that had previously battled it out independently joined forces to surpass the 10 percent barrier. New submissions arrived fast and furious in the closing hours and the competition had more twists and turns than ‘The Crying Game,’ ‘The Usual Suspects’ and all the ‘Bourne’ movies wrapped into one.”

Netflix said “BellKor’s Pragmatic Chaos” edged out a team called “The Ensemble,” another collaboration of former competitors, with the winning submission coming just 24 minutes before the conclusion of the nearly three-year-long contest. The competition was so close and the submissions so sophisticated that it took a team of external and internal judges several weeks to validate the winner after the contest closed on July 26.

Happily, the resulting algorithm won’t be exclusive to Netflix:

The contest’s rules require the winning team to publish its methods so that businesses in many fields can benefit from the work done. The winning submission and the previously hidden ratings used to score the contest will be published at the University of California Irvine Machine Learning Repository. The team licensed its work to Netflix and is free to license it to other companies.

On the first day of my microeconomics class, I told my students that economics is all about incentives. As an example, I used the famous prize for a way to measure longitude, which inspired the invention of the chronometer (i.e., a clock of sufficient precision to measure longitude). Next time around, I will mention the Netflix prize as well.

P.S. Not one to rest on its successes, Netflix has already announced plans for a second Netflix prize. This one aims to find a better way to recommend movies to people based on demographic data (e.g., where they live) rather than movie ratings.

About Donald Marron 294 Articles

Donald Marron is an economist in the Washington, DC area. He currently speaks, writes, and consults about economic, budget, and financial issues.

From 2002 to early 2009, he served in various senior positions in the White House and Congress including: * Member of the President’s Council of Economic Advisers (CEA) * Acting Director of the Congressional Budget Office (CBO) * Executive Director of Congress’s Joint Economic Committee (JEC)

Before his government service, Donald had a varied career as a professor, consultant, and entrepreneur. In the mid-1990s, he taught economics and finance at the University of Chicago Graduate School of Business. He then spent about a year-and-a-half managing large antitrust cases (e.g., Pepsi vs. Coke) at Charles River Associates in Washington, DC. After that, he took the plunge into the world of new ventures, serving as Chief Financial Officer of a health care software start-up in Austin, TX. After that fascinating experience, he started his career in public service.

Donald received his Ph.D. in Economics from the Massachusetts Institute of Technology and his B.A. in Mathematics a couple miles down the road at Harvard.

Visit: Donald Marron

Be the first to comment

Leave a Reply

Your email address will not be published.