PREDICTION ERRORS

This page includes charts relating to the analysis of prediction errors from any of the MoS statistical models. These errors might relate to predictions of:

  • Team Scores or Scoring Shot production
  • Total Scores or Scoring Shot production
  • Game Margins
  • Projected Wins

GAME MARGIN ERRORS

All Time

Topic: The heteroskedasticity or otherwise of game margins has been a common topic here on MoS (see, for example, this post from 2014 or this post from 2013). Generally, the evidence for heteroskedasticity has been found to be fairly weak, but such analyses depend heavily on:

  • the model used to provide the expected margins
  • the variables analysed as being potentially associated with the residuals of that model (ie the differences between the actual and expected margins

In this latest analysis, the MoSHBODS Team Rating System has been used to provide the expected margin data and expected scores for the home and away teams in each contest, these latter variables the ones postulated as being potentially associated with margin prediction errors.

What is also new about this analysis is the technique employed to define and colour each point in the chart. Every point is based on a "neighbourhood", and there are as many neighbourhoods as there are games in the data. A neighbourhood comprises the 50 points nearest to the original point, with distance based on the pre-game expected Home and Away team scores.

For each neighbourhood we calculate the weighted mean of the expected home and the expected away team scores, and a weighted mean of the absolute margin error in those games. The weights we use are the inverse of the distances from the original point that defined the neighbourhood, these distances also measured in terms of expected home team and away team scores. We plot each point at the weighted mean expected scores, and colour it based on the weighted mean absolute margin error.

This technique helps to reveal the underlying spatial patterns in the data by colouring points based on a range of similar games rather than on a single game.

Points of Interest: 

It's apparent that higher expected total scores - those towards the upper right - are associated with larger average absolute errors, and lower expected total scores with smaller absolute errors. 

That said, there are neighbourhoods of relatively high-scoring (in prospect) games where the average absolute errors are average to low. 

What seems to drive heteroskedasticity is a particularly high expected score for either of the teams almost regardless of the expected score for the other team. For the Away team, an expected score of about 110 or more is generally enough to lift expected absolute errors, while for the Home team that figure seems to be about 120.

By Era

Topic: The previous analysis is complicated by the fact that average scores and margins have varied over time. In particular, relative to the very early VFL history, scores and margins have both tended to increase. 

To control for this, we repeat the analysis on an era-by-era basis, the results of which appear at left.

Points of Interest: 

In this chart heteroskedasticity is weak or non-existent in the first two eras, and fairly mild in the third era.

For the 1960-1979 era we see much stronger evidence for heteroskedasticity, with particular large average absolute margin errors for games with very high expected Home team scores and moderately high Away team scores.

The 1980-1999 era was one in which moderate to high Away team scores were especially associated with larger absolute margin errors, while the latest era has seen the largest errors associated more with games where both teams were expected to be scoring at above-average levels.

In interpreting these charts it's important to remember that, as highlighted earlier, any finding of heteroskedasticity is particular to the model used to define expected margins and is only with respect to the variables analysed. It might well be, for example, that the MoSHBODS model has some feature or flaw that causes it to produce margin estimates that are heteroskedastic either generally or for particular eras - we can't assert that V/AFL game margins are, inherently, heteroskedastic based on any such analysis.

On the other hand, the historical accuracy of MoSHBODS' predictions relative to the TAB Bookmaker shows that, at least for the modern era, its opinions are similar to those of a motivated expert and so cannot be completely dismissed.