More Ways to Derive Probability and Margin Predictions From Head-to-Head Prices

/

A couple of weeks ago, in this earlier blog, I described a general framework for deriving probability predictions from a bookmaker's head-to-head prices and then, if required, generating margin predictions from those probability predictions.

For the first step, that of inferring probability estimates from the head-to-head prices, I listed the three methods used most often here on MatterOfStats: the Overround-Equalising, Risk Equalising and Log Probability Score Optimising (LPSO) methods. I also included, however, a catchall "other formulations" category to acknowledge the potential for positing other equations by which to derive team probabilities from bookmaker head-to-head prices.

The only logical requirements of any such alternative formulation are that it:

  • is bounded by zero and one since it's meant to be providing probability estimates
  • decreases monotonically with the home team's head-to-head price - assuming that it is written in terms of home and away team prices and is designed to generate home team probability estimates - since shorter-priced home teams should be rated as being more likely to win

In this blog I'll be working with some alternative formulations that meet both of these requirements and using data for the period 2006 to 2013 to find, empirically, the optimal values of any parameters these formulations might include.

(While reviewing the alternative formulations I've included remember that a team's odds of victory are equal to its price minus one. Those formulations employing terms such as Home Price - 1, for example, are therefore using the Home team's odds rather than its price.)

THE RESULTS

The table below provides the results for 10 different formulations, the first three our usual Overround-Equalising, Risk-Equalising and Log Probability Score Optimising variants (though this latter in reoptimised form), and the last seven completely new but also plausible formulations. All 10 formulations are laid out in the left hand column of the table.

Most of the new formulations have two parameters, optimal values for which I've determined by either minimising the Brier Score for the period 2006 to 2013 (the first three columns) or maximising the Log Probability Score (the next three columns).

(And, yes, I acknowledge that I've fallen back into using only the Brier and Log Probability Scores having just posted about them being a mere subset of a much larger Beta Family of Proper Scoring rules.)

The optimal probability scores across all variants are shown in bold face and, whether we optimise the Brier or the Log Probability Score, are associated with the variant numbered (4), which I'll call the Logistic Odds Difference formulation. It produces a 0.1875 average Brier score per game if α is set to 0.3312 and β is set to 0.6106, and a 0.1955 average log probabililty score if, instead, α is set to 0.3840 and β to 0.5705.

Both of these probability scores are superior to those produced by the Overround-Equalising, Risk-Equalising and Log Probability Score Optimising (LPSO) approaches. (Note that the LPSO shown here has a different value of α to that in the variant used for MatterOfStats wagering and tipping. In the MatterOfStats version α is 1.0281% whereas the LPSO variants here, which have been optimised over a different set of games to the MatterOfStats version of LPSO, have α values of 1.81% (Brier Score optimised) and 1.41% (LPS optimised))

In this sense, the Logistic Odds Difference formulation with the optimised parameter values yields superior probability forecasts to the standard MatterOfStats fare, although the difference in probability score terms is fairly small.

Producing optimal margin predictions, however, can be achieved with one of the three standard MatterOfStats approaches.

Using variant (3), which is the LPSO approach with α set to 1.61%, and converting the resulting probability assessments to margin predictions using the logit formulation and a multiplier of 19.594 yields a Margin Predictor with an MAE across the 8 seasons of 28.92 points per game.

That's an outstanding MAE, notwithstanding that it is an optimised, in-sample figure.

Adopting the same, LPSO approach to deriving probability estimates but then converting these probabilities into margins using the inverse Normal CDF instead, and choosing optimal values for α and σ yields a Margin Predictor that is almost but not quite as good. It's MAE is 28.93 points per game.

For optimising margin predictions using the inverse Normal CDF to map probabilities to margins, both the mean and standard deviation of the Normal can be optimised. Initially, I included both parameters in the optimisation process but the optimised means were all so close to zero that I wound up dropping this parameter. (You know, Occam's Razor and all that ...)

THE IMPLICATIONS

As always, it's important to measure the "effect size" of any change - in short, does the distinction in formulation make a difference?

Firstly, let's look at the difference in the probability estimates produced by the Overround-Equalising, LPS-Optimised and Logisitic Odds Difference approaches considered here, the latter two being optimised in terms of Log Probability Scoring across the 8 seasons.

The differences amongst the three approaches, in percentage point terms and across the entire range of home team prices, are small, never rising much above about 2% points in absolute terms. Perhaps the most notable distinction of the LOD approach is that it assigns slightly higher probabilities than the other two approaches to home teams priced at $3 and above, so much so in fact that it implies that wagering on home teams has positive expectation (ie is profitable) whenever they are priced at $3.75 or above.

A quick analysis reveals that level-stake wagering on all home teams priced at $3.75 or above would have been profitable in 6 of the last 8 seasons, and would have been rendered unprofitable across all 8 taken as a whole only because of a disastrous performance in 2011. Lowering the price threshold to $3.70 yields profitability across the entire period (though still in only 6 of 8 seasons).

Lastly, let's compare the margin predictions from the Overround-Equalising, Risk-Equalising and LPS Optimised approaches, where now the optimisation serves to minimise the Mean Absolute Error of the resulting predictions.

Here the differences are, subjectively at least and based on a sample of one, a little more substantive and we find that the LPSO variant forecasts a better outcome for the home team - a larger victory or a smaller loss - for all home team head-to-head prices of about $1.65 or more. In points terms, the difference gets larger as the home teams price rises, reaching a maximum at the highest price shown of $11 where the LPSO variant predicts about a 6 point smaller loss for the home team.

Such differences as there are have relatively large implications for the efficacy of the three variants in the line betting market across the period. Adopting the margin predictions of the LPSO variant and wagering accordingly would have yielded a 53.9% success rate, enough for a 2% return at $1.90 prices. Adopting instead the Overround-Equalising suggestions would have led to a 52.7% success return, sufficient only to approximately breakeven at $1.90 prices, while following the Risk-Equalising outputs would have produced only a 51% success rate and a 3% loss at $1.90 prices.

Of course the question of whether these results might be expected post-sample remains moot.

THE CONCLUSION

If our goal is to produce probability forecasts with the best average Brier or Log Probability Scores then we can do better than the Risk-Equalising, Overround-Equalising or LPSO variants by adopting the LOD formulation. Though I've calculated the optimal parameters for that formulation here using data for the past 8 seasons combined, were I to put this recommendation into practice I'd probably reoptimise the formulation over a shorter time span - perhaps the last 2 or 3 years only.

Alternatively, if margin prediction is what we're after, then an LPSO approach proves best, using a logit formulation to convert probability estimates to margin predictions. Here too, were I to adopt this recommendation I'd seek to reoptimise the formulation using only the last few years of data.