Modelling Team Scores as Weibull Distributions : Part II

/

In a previous post I discussed the possibility of modelling AFL team scores as Weibull distributions, finding that there was no compelling empirical or other reason to discount the idea and promising to conduct further analyses to more directly assess the Weibull distribution's suitability for the task.

What motivated this entire line of enquiry, you might recall, was to assess the applicability of the Pythagorean Expectation Equation for estimating team victory probabilities. That Equation would have theoretical justifications for this purpose, a paper that I linked to in the previous post revealed, if we found that:

  1. Team scores are distributed as independent Weibull distributions
  2. The Weibull distributions for competing teams share a common parameter, k 

THE DATA

For this analysis I've used the data for the entirety of seasons 2006 to 2013, with the target variable a team or the two teams' final scores, and the following (potential) regressors:

  • Home team status (a 0/1 variable reflecting whether or not the team whose score we're modelling is playing at home or away)
  • Interstate status (a +1/0/-1 variable reflecting the interstate nature of the clash from the point of view of the team whose score we're modelling)
  • Own MARS Rating (the MARS Rating of the team whose score we're modelling)
  • Own Venue Experience (the number of games played during the past 12 months at the same venue as the current game involving the team whose score we're modelling)
  • Own Price (the TAB Bookmaker pre-game price of the team whose score we're modelling)
  • Own Points Scored Last X (the average score of the team whose score we're modelling during its last X games. Note that we allow this averaging process to span seasons and to include Finals). I included values of X from 2 to 8 both here and for the variables below.
  • Own Points Conceded Last X (as for Own Points Scored but based on the points conceded in those games)
  • Own Form Last X (the change in MARS Rating across the last X games for the team whose score we're modelling)
  • Opponent versions of each of the Own variables, defined analogously but for the team that is the opponent of the team whose score we're modelling

THE MODELS

Using the Weibull distribution as the basis (and the survreg function in the survival package in R as the tool), five regression models were fit:

  1. Treating as the target variable the scores of all teams in all games (ie for n games we wind up with 2n rows to fit) 
  2. Treating as the target variable the scores of all home teams in all games 
  3. Treating as the target variable the scores of all away teams in all games 
  4. Treating as the target variable the scores of all pre-game favourite teams in all games 
  5. Treating as the target variable the scores of all pre-game underdog teams in all games

For models 4 and 5, where the two teams in a game were equal favourites, the home team was treated as the favourite.

Initially, all of the variables listed above were included in each model (except those that are inappropriate given the subset of games fitted - for example we exclude the Home Team variable in models 2 and 3 as it would be collinear with the intercept). Using various techniques for excluding variables - which is a hand-wavy way of implying that it was neither entirely scientific nor, I suspect, repeatable - I arrived at a subset for each model that:

  • was relatively small
  • explained only slightly less of the total variability than could be explained using all variables
  • included (mostly) variables with statistically significant coefficients

As a means of contexting the quality of the results achieved using the Weibull distribution I also constructed the same five models using the Normal distribution and a similar process for selecting variables.

The results were as as shown in the table below.

MODELLING ALL TEAMS SCORING

Focussing firstly on the results for the Weibull regressions we see, for example, that nine variables have made the final model for the All Teams model: the two MARS Rating variables, the two bookmaker prices, the Interstate Status variable, and the Own and Opponent Points Scored and Points Conceded variables for the last 8 games.

The coefficients provide the multipliers we should use to convert the value of the relevant variable into a contribution to the Shape parameter of the Weibull (ie the lambda parameter as described in the previous blog). Their signs are interesting in that they tell us that a team's score is likely to be higher, after controlling for the other variables in the model, if:

  • it is Rated higher
  • its opponent is Rated lower
  • it is enjoying the benefit of Interstate Status
  • it's scored or conceded more points in its most recent 8 games
  • its opponent has scored or conceded more points in its most recent 8 games
  • its bookmaker price is lower (reflecting an increase in relative strength)
  • its opponent's bookmaker price is higher (reflecting a decrease in relative strength)

These are not especially surprising results except for the fact that a team's defensive abilities and its opponent's offensive abilities have positive effects on a team's likely scoring in a game. This is despite the fact that the straight correlation between a team's score in the current game and the points it has conceded, on average, in the most recent 8 games is -0.23, and the straight correlation between a team's score in the current game and the points its opponent has scored, on average, in the most recent 8 games is -0.19.

The positive coefficients for these two variables should be interpreted as revealing that, once we control for the influence of the other variables in the model, teams that have recently conceded more points and teams playing opponents that have recently scored more points, will tend to score more highly in the current game. Note however that the correlation between a team's average points scored and points conceded over the most recent 8 games is -0.46, so a team with above-average points scored is likely to have below-average points conceded, and vice versa.

The value of 4.09 for the Exponent variable is the fitted estimate of the parameter k. This value is broadly consistent with the numbers we reported in the previous blog. The Log Likelihood value is presented as a means of comparing the results for the Weibull regression with that for the Normal regression of the same model. A higher (ie less negative) value signifies a better-fitting model.

Lastly, the pseudo R-squared value is the squared correlation between the predicted and actual team scores.

Comparing the All Teams model results for the Normal formula with that for the Weibull formulation we find that:

  • the signs on all of the coefficients are the same
  • the R-squared for the Normal formulation is slightly higher than the pseudo R-Squared for the Weibull distribution
  • the Log Likelihood for the Normal formulation is slightly higher than the Log Likelihood for the Weibull.

In summary, the Normal formulation is very mildly superior to the Weibull formulation.

(I note in passing that both of the models shown here - and those that I'll present in the remainder of this post - are far superior to the models I presented in this earlier blog from 2012 in which I used only TAB bookmaker prices as the only regressors.)

MODELLING HOME TEAM AND AWAY TEAM SCORES SEPARATELY

Next, we fit separate models to the scores of home teams and away teams on the assumption that different variables might be important in explaining their scores and the same variables might be important to different extents.

For the Weibull, in explaining home team scores we find that Interstate Status, Own and Opponent scoring are no longer statistically significant so we exclude them. The final model has a pseudo R-squared of about 0.23.

The Weibull model for the away team scores requires the inclusion of Own and Opponent scoring but also suggests that the Interstate Status variable is unnecessary. It has a pseudo R-squared on almost 0.28, which means that the Weibull is able to explain a larger proportion of the variability in away team scores than it can explain of the variability in home team scores.

Of particular interest is the fact that the estimated Exponents for the two models, while somewhat similar, are different. Thus one of the conditions for the applicability of the Pythagorean Expectations Equation is not quite met.

Comparing these two models with the equivalent models built using the Normal distribution we find that:

  • a slightly different variable set is required for the two models
  • the R-squareds for the Normal models are both marginally higher than the pseudo R-squareds for the equivalent Weibull
  • the log likelihood values for the fitted Normals are marginally superior to those for the fitted Weibulls

Again then we find that the Normal distributional assumption is very mildly superior to the Weibull.

One attractive feature about the Normal formulation is the straightforward manner in which the model coefficients can be interpreted. So, for example, looking at the home team model we can claim that, ceteris paribus, for every 10 Point increase in the home team MARS Rating, we expect a 1.4 point increase in their score for a particular game while for every 10 Point increase in the away team MARS Rating, we expect a 2.3 point decrease in the home team score.

Also, an away team playing Interstate is likely to score about 4 points less than the same team playing the same opponent at a non-Interstate venue (note that the Interstate variable will be -1 for an away team playing out-of-state).

At the bottom of the results for the Normal formulations I've recorded a fitted correlation between the home and the away teams' scoring. I calculated this by fitting a generalised linear model via the vglm function in the VGAM package and came up with a figure of about -0.12, which is consistent with the figure I derived in that earlier post. Unsurprisingly, when one team scores more than it was expected to, its opponent tends to score less than it was expected to.

MODELLING FAVOURITES AND UNDERDOGS SEPARATELY

Competing teams can be cleaved in twain using methods other than the obvious home/away dichotomy. We might, for example, use the pre-game TAB bookmaker data to anoint a favourite and an underdog in each game, and then fit separate models to these two groups.

The results of doing this also appear in the earlier table where we find that, for the Weibull models:

  • the Opponent Venue Experience makes its only appearance in a final model, popping up in the model for Underdog scores.
  • the fitted Exponents for the two groups of teams, while less different in this formulation than in the home/away formulation, still differ by about 5%
  • the pseudo R-squareds are much lower (though, of course, we can't directly compare them to the R-squareds for the home/away models since the variability is different)

Comparing the results for the Normal formulation with those for the Weibull again suggests that the Normal formulation is mildly superior.

SUMMARY AND CONCLUSION

In summary then we've found that:

  • For all five of the models constructed, a Normal formulation is slightly superior to a Weibull formulation.
  • Splitting the teams into groups of home and away, or groups of favourites and underdogs and then fitting Weibulls to each group produces fitted Exponents that are not equal but that differ by only about 5%.

For modelling purposes then it seems that the better choice is the Normal formulation, but, for a quick estimation of a team's victory probability a reasonable approximation can be reached by using the Pythagorean Expectation Equation with k = 4 so that we have the equation at right.

As a rule of thumb then, for a given team:

  • if the expected ratio of points conceded to points scored is 1.25, the estimated victory probability is 30%
  • if the expected ratio of points conceded to points scored is 1.1, the estimated victory probability is 40%
  • if the expected ratio of points conceded to points scored is 1, the estimated victory probability is 50%
  • if the expected ratio of points conceded to points scored is 0.9, the estimated victory probability is 60%
  • if the expected ratio of points conceded to points scored is 0.75, the estimated victory probability is 75%