Debrief – failed prediction

In a post I wrote ten days ago, I made an attempt to predict the future.

“I went to the website of a certain sportsbetting company and put $100 on Daft Punk to win this year’s Hottest 100 with the song Get Lucky,” I revealed.

I can now report I lost that $100 backing my own judgment. The song in question came 3rd in the Hottest 100. I had never even heard the winning song before.

Hopefully I have traded that $100 for some useful perspective and humility.

That is going to be an important theme this year as I play my part in the Good Judgment Project. I’ve been assigned to a team and just finished my training.

The training emphasised the importance of putting time and effort into exploring both “inside” and “outside” views of a prediction.

In the case of the 2013 Hottest 100, I over-emphasised the “inside” view. I was so sure Get Lucky was the best track I had heard all year, that I didn’t go and run any real analysis of what sort of tracks won the Hottest 100.

If I had done that “outside” analysis, I would have found a dearth of French disco tracks and a preponderance of  acoustic / folk tracks, such as Mumford and Sons’ Little Lion Man, which topped the poll in 2009.

I may also have noticed a lot of Australian tracks, like Gotye’s Somebody That I Used to Know (#1, 2011) And of course, the killer combo, acoustic/folk tracks by local artists like Angus and Julia Stone’s Big Jet Plane (#1, 2010).

With that sort of analysis to hand, Vance Joy’s win with Riptide is not so surprising.

It ticks all the boxes… except actually having an emotional reaction to it. It doesn’t tick that box.

So, anyway, my forecasting record is reduced to ashes. But, like the legend of the phoenix, all ends with beginnings. One is not going to make progress without making a few mistakes. The important thing is to shine a light on them and try to improve.

I can predict the future

I can predict the future, with the following caveat:

Not all the time.

I’m rather chuffed by a couple of predictions I’ve made recently, and after I tell you about them I’m going to describe the surprising results of some recent research that has me abuzz.

The first thing I predicted was the fall in Apple’s share price when it was heading to around $700 and people were excitedly predicting a price of $1000. No, I did not put my money where my mouth was.

The second thing I successfully predicted was the beginning of the end of US monetary easing, in December. No, I did not put my money where my mouth was.

Would this have happened to me if I really could predict the future?

So, is it possible to predict the future better than randomly?

Research suggests the answer is yes. A Research project sponsored by IARPA (the U.S. Intelligence Advanced Research Projects Activity) over the last four years has assembled huge panels of forecasters. It is called the Good Judgment Project. They get asked hundreds of questions on world events, such as what might happen in Syria, will  a certain country exit the euro, etc.

The Economist describes the big finding of the project:

“The big surprise has been the support for the unabashedly elitist “super-forecaster” hypothesis. The top 2% of forecasters in Year 1 showed that there is more than luck at play. If it were just luck, the “supers” would regress to the mean: yesterday’s champs would be today’s chumps. But they actually got better. When we randomly assigned “supers” into elite teams, they blew the lid off IARPA’s performance goals. They beat the unweighted average (wisdom-of-overall-crowd) by 65%; beat the best algorithms of four competitor institutions by 35-60%; and beat two prediction markets by 20-35%.”

Aggregated forecasts of the Good Judgment Project are submitted to the IARPA forecasting tournament, which they won last year.

I have applied to join the panel of forecasters in 2014, and have sat a battery of online tests, including some very difficult questions! Such as:

True or False, Cuba helped to organize negotiations between the government of Colombia and FARC (Spanish acronym), the Revolutionary Armed Forces of Colombia


the meaning of DESUETUDE

and complete the pattern games:

trauma  tuna          flight  fit          wife  __ a __ __          glossy  gravity

They did political spectrum testing, put me through an ultimatum game, did lots of spatial, mathematical and verbal IQ stuff, and checked basic general knowledge.

I’m excited to get started. The project provides feedback directly to participants on their forecasting prowess as events transpire or fail to transpire. It should serve as a check on overconfidence.

In the meantime, I have one last forecast that I have publicly made. I went to the website of a certain sportsbetting company and put $100 on Daft Punk to win this year’s Hottest 100 with the song Get Lucky.

I mention this here only in the interests of extreme transparency. While to me it is obvious the song is the best of last year, there may be a certain youth and antipodean bias at triple J. Promotion for the Hottest 100 features the teenage Kiwi heavily:


Will that bias the vote? Will I lose my $100? Will that instil new levels of humility in me? That I can’t predict.