Improving MoSSBODS' Team and Total Predictions

/

It's rare - I think unprecedented - for me to make changes to any of the Fund algorithms during the course of a season, but this year I'm making an exception because the change:

  • effects only one of the Funds (the Overs/Unders Fund), and that's the smallest of them
  • so obviously improves the algorithm's forecasting performance in a few, specific areas

So, what am I referring to?

PREDICTING TEAM SCORES AND GAME TOTALS

In the blog post in which MoSSBODS was introduced, I talk about the various components of the Rating System, which comprises:

  • Offensive and Defensive Ratings for every team
  • Venue Performance values for every team at every Venue
  • A Travel Penalty
  • An Overall Expected number of Scoring Shots, which is set for each season equal to the average number scored in all games from the previous season

Currently, in estimating a particular team's expected Scoring Shot (SS) production against a specified opponent at a specified Venue, I use this formula:

Expected SS = Overall Expected SS + (Own Offensive Rating - Opponent Defensive Rating) + Own Venue Performance value + Own Travel Penalty

For the home team, the Own Venue Performance value is typically positive and the Travel Penalty zero since they're usually playing in their home State. For the away team, the Own Venue Performance is mostly zero, sometimes negative and only occasionally positive, and the Travel Penalty either 0, if the game is in its home State, or -3 Scoring Shots otherwise.

In this formulation then, the Travel Penalty serves only to depress the expectation of away team SS production.

In the new formulation we instead assume that all four of the Venue Performance and Travel Penalty components have a role in determining both teams' Expected SS production.

Specifically, we now use: 

Expected SS = Overall Expected SS + (Own Offensive Rating - Opponent Defensive Rating) + ((Own Venue Performance value + Own Travel Penalty) - (Opponent Venue Performance value + Opponent Travel Penalty))

This change tends to:

  1. Increase the Expected SS for home teams
  2. Increase the Expected SS for away teams
  3. Increase the Expected SS for the two teams combined

In other words, it means we tend to predict higher scores for both teams and in aggregate.

For example, in the 126 games already completed in the current season, the predicted home team scores are higher under the new formulation in 112 games, the predicted away team scores are also higher under the new formulation in 112 games, and the predicted total score is higher under the new formulation in 111 games.

If we analyse MoSSBODS performance so far this season we find that it has, indeed, tended to underestimate home team, away team and total scores, so this adjustment seems to be, at least, in the right direction.

HISTORICAL PERFORMANCE

We can apply this new formulation to the entirety of V/AFL history and compare the mean absolute error (MAE) performance using this and the previous formulation.

Doing that yields the following:

  • Superior MAEs for home team scores forecasts in about 65% of seasons and, across all seasons, equal to about 0.17 points per game (19.54 vs 19.71)
  • Superior MAEs for away team scores forecasts in about 77% of seasons and, across all seasons, equal to about 0.21 points per game (18.56 vs 18.77)
  • Superior MAEs for total score forecasts in about 80% of seasons and, across all seasons, equal to about 0.71 points per game (27.05 vs 27.76)

Lowering MAEs by that much over such a huge span of history, is fairly significant and indicative that the change is a "good thing".

Looking only at games played since 2000 sees even larger performance improvements in home team and total score forecasting, and a smaller, though still positive, improvement in away team score forecasting.

For 2016 alone, the statistics are:

  • Home team score MAEs - New Formulation: 19.6 / Old Formulation 20.6
  • Away team score MAEs - New Formulation: 19.7 / Old Formulation 19.6
  • Total score MAEs - New Formulation: 25.7 / Old Formulation 27.6

For me, that's extraordinarily compelling evidence for the change, which is why I'm implementing it immediately.

It might be worth exploring whether the now 50:50 allocation of the Venue Performance and Travel Penalty components to the home and away teams is optimal, but that's a task for the off-season, not least because this allocation, mathematically, has no effect at all on MoSSBODS' margin forecasts. With the 50:50 allocation, the increase or decrease in both the home and away team's score forecasts, relative to the old formulation, are always of an identical size.

Since MoSSBODS has done relatively well on margin forecasting in this and in previous seasons, that also seems to be a desirable outcome of the switch to the new formulation.