Two hours less than 30 days ago, the team of BellKor’s Pragmatic Chaos submitted the first entry to the Netflix Prize to exhibit a 10% improvement in performance over Netflix’s movie-recommendation algorithm. That started the final clock for the competition — whoever’s ahead at 2:42 Eastern time today wins the $1 million prize.
One of the really interesting lessons of the competition is that blendings of many algorithms seem to work better than any single algorithm, even when there’s no principled reason to do the blend. It’s sort of a “wisdom of crowds of computer programs” effect. As you can imagine, once BellKor’s Pragmatic Chaos (itself a blend of algorithms from two teams that have been leading through most of the competition) crossed the 10% threshold, just about everybody else realized their best — probably only — chance was to work together.
As of yesterday afternoon, a team called The Ensemble, made up of — well, I can’t really tell how many previously separate competitors, but a lot — has achieved a 10.09% improvement. BellKor’s Pragmatic Chaos is at 10.08%. A hundredth of a percentage point might determine who gets the million. Wow.
Is BPC sitting on a slightly better algorithm they’re planning to submit at the buzzer for the win? Check the leaderboard later this afternoon to find out.
Update: Double wow. No announcement yet, but it looks like BPC and The Ensemble both made submissions in the last hour of the contest; BGC made it to 10.09% but Ensemble, four minutes before closing time, bumped up to 10.10%.
Re-update: I didn’t read the rules carefully enough. It looks like there’s another dataset (‘test dataset”), distinct from the one generating the public scores (“quiz dataset”) and the final winner will be the program that does best on the test dataset. So the shifts in the lead, exciting as they are, aren’t necessarily relevant to the contest; we’ve got two algorithms which are essentially identically good and it ought to be a coin-flip which one does better in the final judgment.