RSS

More years of data for Accuracy of Win Predictions

30 Dec

My first reaction after I’ve finished post about Accuracy of Predictions for Wins in 2010/11 was this: “that’s nice but what about consistency of authors?”. Are writers and bloggers consistent over time or is it basically a semi-random outcome every year? Is it an educated guess or just a guess by educated people?

To answer this question I simply needed more data and because I didn’t want to wait a couple more years I went the other way – into the past. I collected some predictions for NBA standings before 2009-10 season and before 2008-09 season [btw, it's not a closed list so if you want to add any predictions let me know in the comments]. Then I again compared them to actual results by Root Mean Square Deviation [read post about 2010/11 for explanation]. If you want to play with numbers or check any single entry just download NBA Predictions Accuracy 2008-2009-2010.pdf but if you are interested only in the results here they are…

Prediction for 2009/10 Date Data from RMSE
ESPN’s forecast 09/02/2009 sports.espn.go.com/nba/news/story?page=OffseasonPredictions[...] 8,78
Consensus of all TrueHoop Network bloggers 10/26/2009 http://www.eightpointsnineseconds.com/2009/10/truehoop-network[...] 8,83
Las Vegas Hilton 10/15/2009 http://www.docsports.com/2009/nba-season-win-totals-529.html 9,20
Bradford Doolittle and Kevin Pelton 10/26/2009 http://basketballprospectus.com/article.php?articleid=747 9,43
Basketball Prospectus 10/19/2009 http://thepaintedarea.blogspot.com/2009/10/2009-10-nba-win[...] 9,70
Storyteller ???? http://storytellerscontracts.wordpress.com/2010/10/05/nba-pred[...] 9,77
John Hollinger 10/19/2009 http://thepaintedarea.blogspot.com/2009/10/2009-10-nba-win[...] 10,00
Neil Paine – Win Shares 10/27/2009 http://www.basketball-reference.com/blog/?p=3727 10,07
Wins in previous season - http://www.dougstats.com/08-09.html 10,18
deedsy404 10/27/2009 http://www.cbssports.com/mcc/blogs/entry/6173035/17934231 10,35
Bill Simmons 10/23/2009 http://sports.espn.go.com/espn/page2/story?page=simmons[...] 10,35
Early Win Shares 09/21/2009 http://www.basketball-reference.com/blog/?p=3408 10,47
Kelly Dwyer 10/19/2009 http://thepaintedarea.blogspot.com/2009/10/2009-10-nba-win[...] 10,69
Jon Peterson ???? http://answers.yahoo.com/question[...] 11,20
r1zzo23 10/29/2009 http://biggreenmachine.wordpress.com/2009/10/29/nba-[...] 11,96
Prediction for 2008/09 Downloaded from RMSE
Ed Weiland http://hoopsanalyst.com/blog/?p=215 8,741
MaxKDate http://www.cbssports.com/mcc/blogs/entry/[...] 8,771
Vegas http://wagesofwins.com/2008/10/28/projecting[...] 8,891
Bill Simmons http://wagesofwins.com/2008/10/28/projecting[...] 8,896
John Hollinger http://apbr.org/metrics/viewtopic.php?p=200#p200 9,349
Accuscore http://wagesofwins.com/2008/10/28/projecting[...] 9,391
Davis21Wylie http://wagesofwins.com/2008/10/28/projecting[...] 9,548
Mountain http://apbr.org/metrics/viewtopic.php?p=200#p200 9,571
Neil Paine http://apbr.org/metrics/viewtopic.php?p=200#p200 9,584
3yAvg WP http://wagesofwins.com/2008/10/28/projecting[...] 9,987
Accuscore 2 http://wagesofwins.com/2008/10/28/projecting[...] 9,993
wAvg http://wagesofwins.com/2008/10/28/projecting[...] 10,029
Mike G http://apbr.org/metrics/viewtopic.php?p=200#p200 10,357
Pelton WP http://wagesofwins.com/2008/10/28/projecting[...] 10,516
Kevin Pelton http://apbr.org/metrics/viewtopic.php?p=200#p200 10,784
Pelton http://wagesofwins.com/2008/10/28/projecting[...] 11,203
Wins in previous season http://www.dougstats.com/07-08.html 11,413
WoW http://wagesofwins.com/2008/10/28/projecting[...] 11,623
BR WP http://wagesofwins.com/2008/10/28/projecting[...] 13,955

Those lists are unfortunately shorter than I hoped for because:
– sadly APBR forum was hacked so their batch of past predictions vanished forever,
– Wages of Wins Network either didn’t exist or didn’t post any win predictions for 2009/10 or
I can’t quickly find them [they merged and relocated],
– surprisingly few authors even published their win predictions 3 years ago or google just can’t find them easily.

I didn’t get a big sample but it’s still bigger than just one year so
I think I can make a couple of additional points…

- last season had a higher level of accuracy than two previous ones. Was it a classic fluke year? ;-)

- Vegas odds-makers clearly know what are they doing.
So I’m just wondering… as a blogger without any good system shouldn’t you just copy their predictions with some small changes regarding those teams where you feel veeery strongly they are wrong?

- wisdom of the ESPN crowd had two really good years in a row – coincidence or this system works?
Again, I wish there was more data but they started it two years ago.

- John Hollinger had two top seasons and one average so either he is really good at this game or very lucky.

- there’s an interesting path for Bill Simmons. He had a spectacular results for 2008-09, really bad ones for 2009-10 and he didn’t even finish his predictions for 2010-11! So did he… just quit this game?

- WARP, Wins Produced and Win Shares did a surprisingly poor job in the last 3 seasons so even though it doesn’t tell us anything about those ratings I have a theory: you shouldn’t base your predictions on numbers from previous year and just plug them into a system because there are too many unpredictable variables… you really should get a little wild and add some changes not based on numbers. True or false?

- overall, it DOES seem to be at least somewhat consistent… although again I wish it was a bigger sample.
We will get there in a couple of years ;-)

 
4 Comments

Posted by on December 30, 2011 in Expanding Horizons

 

Tags: , , , , , ,

4 responses to “More years of data for Accuracy of Win Predictions

  1. Daniel M (@DSMok1)

    December 30, 2011 at 18:20

    I have some (1100 pages) of the old forum for APBR on my hard drive, and will post it on APBRmetrics when I have time.

    This is from 07-08 on APBRmetrics
    Predicted Wins Difference (Absolute values)
    Pyth+ tm JohnH DW21 MikeG Mtn JohnH DW21 MikeG Mtn
    72.3 Bos 51 50.2 54.0 51 21.3 22.1 18.3 21.3
    65.0 Det 48 48.7 47.6 53 17.0 16.3 17.4 12.0
    46.5 Was 33 39.9 39.5 37 13.5 6.6 7.0 9.5
    34.4 Phi 21 29.7 25.7 28 13.4 4.7 8.7 6.4
    37.9 Ind 28 32.0 30.1 28 9.9 5.9 7.8 9.9

    50.7 Orl 49 45.8 41.0 40 1.7 4.9 9.7 10.7
    38.0 Atl 42 33.4 31.9 35 4.0 4.6 6.1 3.0
    42.9 Tor 40 40.0 45.5 42 2.9 2.9 2.6 .9
    27.6 Mil 35 27.4 29.7 32 7.4 .2 2.1 4.4
    34.6 NJ 39 36.9 40.1 42 4.4 2.3 5.5 7.4

    27.1 Cha 34 29.7 35.3 30 6.9 2.6 8.2 2.9
    23.3 Mia 43 37.6 35.9 40 19.7 14.3 12.6 16.7
    20.1 NY 38 32.7 37.3 42 17.9 12.6 17.2 21.9
    35.9 Cle 54 57.1 53.5 49 18.1 21.2 17.6 13.1
    32.1 Chi 55 57.7 53.2 54 22.9 25.6 21.1 21.9

    39.2 east 40.7 39.9 40.0 40.2 12.1 9.8 10.8 10.8

    Pyth+ tm JohnH DW21 MikeG Mtn JohnH DW21 MikeG Mtn
    46.4 Por 26 21.8 18.6 26 20.4 24.6 27.8 20.4
    54.9 LAL 43 40.2 38.2 40 11.9 14.7 16.7 14.9
    54.0 NO 39 40.3 41.4 43 15.0 13.7 12.6 11.0
    45.8 GS 36 34.4 39.6 39 9.8 11.4 6.2 6.8
    49.7 Den 53 44.4 42.6 53 3.3 5.3 7.1 3.3

    56.5 Phx 54 56.9 58.6 57 2.5 .4 2.1 .5
    31.7 Sac 27 36.0 32.0 35 4.7 4.3 .3 3.3
    47.1 Uta 50 45.3 54.1 50 2.9 1.8 7.0 2.9
    26.3 LAC 20 33.7 33.6 28 6.3 7.4 7.3 1.7
    57.4 SAS 59 65.2 58.8 59 1.6 7.8 1.4 1.6

    54.2 Dal 58 62.2 63.5 58 3.8 8.0 9.3 3.8
    28.3 Mem 34 35.2 37.5 34 5.7 6.9 9.2 5.7
    23.2 Sea 35 34.0 27.9 29 11.8 10.8 4.7 5.8
    14.6 Min 22 26.2 26.9 23 7.4 11.6 12.3 8.4
    42.8 Hou 61 55.2 56.3 53 18.2 12.4 13.5 10.2

    42.2 west 41.1 42.1 42.0 41.8 8.3 9.4 9.2 6.7

    10.2 9.6 10.0 8.7

    I think that’s about all I’ve got, though.

     
    • wiLQ

      December 30, 2011 at 18:32

      That’s a great batch of new data, thanks! I’ll add it if I can.
      By any chance do you have something similar for 2008/09 and/or 2009/10?

       
    • wiLQ

      January 1, 2012 at 17:03

      I’ve calculated RMSE for your numbers and results look pretty bad (JohnH=11.64, DW21=11.29, MikeG=11.14, Mtn 10.51) but without any context it’s hard to tell whenever it was just a difficult year for predictions or it was a learning process or what.

       
  2. ek

    December 31, 2011 at 20:41

    This is very good stuff, wiLQ.

     

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s