2015 - Team Ratings After Round 4

/

Hawthorn and Sydney both shed Rating Points (RPs) on the ChiPS and MARS Rating Systems this past weekend but remain in 1st and 2nd positions, respectively, on both Systems.

Fremantle remains third in both Systems, though its victory and consequent RP accumulation allowed it to close the gap a little on 2nd-placed Sydney. It now sits about 5 RPs behind Sydney on ChiPS Ratings, and 2 RPs behind them on MARS Ratings.

Further down the table the Systems diverge a little in their opinions - more this week than has usually been the case. Six teams are now Ranked three places or more differently by the two Systems:

  • Richmond is Ranked 5th by ChiPS and 8th by MARS
  • Essendon is Ranked 6th by ChiPS and 9th by MARS
  • Port Adelaide is Ranked 7th by ChiPS and 4th by MARS
  • Adelaide is Ranked 10th by ChiPS and 6th by MARS
  • GWS is Ranked 11th ChiPS and 14th by MARS
  • Geelong is Ranked 14th by ChiPS and 11th by MARS

Two other teams, Collingwood and the Kangaroos, are ranked two places differently by ChiPS compared to MARS.

All of these differences in Rankings, however, are for teams squabbling for positions where Ratings are most compressed - from Ratings of about 990 to 1,010 - so the Ranking differences don't reflect significant Rating differences. As such, the correlation between ChiPS and MARS Ratings remains high. It currently stands at +0.96 and has been high all season as you might gather by inspecting the chart below of team-by-team Ratings as at the end of each round so far this season.

COLLEY, MASSEY AND ODM RATINGS

For the past few years MatterOfStats has included, alongside MARS (and now ChiPS), Ratings created using other, popular Team Rating Systems, namely:

  • A version of the Colley System (the MoS version of which you can read about here)
  • A version of the Massey System (which you can also read about here)
  • The Offense-Defense Model (ODM) Rating System (which you can read about here)

The main features that distinguish these Rating Systems are whether they :

  • consider only the result of a game (Colley), the margin (Massey), or the individual team scores in the game (ODM)
  • separately assess teams' offensive and defensive capabilities (ODM) or rate only their overall ability (Colley and Massey).

All three Systems take into account the estimated quality of the team against which a particular result or score was achieved.

They produce numerical Ratings for every team - in ODM's case, a pair of Ratings - but I report only their team Rankings here on MoS and compare them to ChiPS' and MARS', as shown below.

For the most part, the five Systems rank all 18 teams similarly, the rank correlation coefficients between all possible pairs of Rating Systems ranging from +0.64 to +0.98. The lowest of those correlations is between the Rankings of the ChiPS and the Colley Systems, the highest between Massey and ODM.

At the individual team level, the biggest differences in team Rankings by just one or two Systems compared to the rest are for:

  • Adelaide, ranked 10th by ChiPS, but 3rd to 6th by the other Systems
  • Richmond, ranked 5th by ChiPS, but 8th to 13th by the others
  • Collingwood, ranked 5th by Colley (which considers only wins and losses, not margins, and only in the current season), but 8th to 10th by the others
  • Hawthorn, ranked 9th by Colley, but 1st to 3rd by the others
  • West Coast, ranked 10th by Colley, but 4th to 7th by the others
  • Geelong, ranked 11th by MARS (which takes longest to adjust Ratings in a new season), but 14th or 15th by the others
  • GWS, ranked 14th by MARS, but 6th to 11th by the others
  • The Western Bulldogs, ranked 12th by ChiPS and MARS, but 4th or 5th by the others

Qualitatively, these differences are similar to those we saw at the same time last season and will tend to reduce as the season progresses. That said, the Colley System will always tend to follow the competition ladder more closely than the other Systems because it reacts only to wins and losses, not their magnitude.

Offense-Defense mODEL (ODM) Rankings

One of the attractive features of the ODM System is that it provides separate offensive and defensive Ratings and Rankings of each team. The chart below shows ODM's Rankings for each team's offensive and defensive abilities, and for these two components combined.

Teams Ranked significantly (ie at least five places) higher for their defensive capabilities than their offensive capabilities are:

  • Essendon, Ranked 3rd on defence, 14th on offence (and 9th overall)
  • Melbourne, Ranked 10th on defence, 15th on offence (and 13th overall)

Teams Ranked significantly higher for their offensive capabilities than their defensive capabilities are:

  • Carlton, Ranked 10th on offence, 16th on defence (and 14th overall)
  • Gold Coast, Ranked 13th on offence, 18th on defence (and 17th overall)
  • The Kangaroos, Ranked 9th on offence, 14th on defence (and 12th overall)
  • West Coast, Ranked 4th on offence, 12th on defence (and 7th overall)

Currently, the rank correlation between the teams' defensive and overall Rankings is +0.89, and between their offensive and overall Rankings is +0.88, which suggests that offensive and defensive capabilities have so far this season been about equally important in producing results.

The last comparison we'll make of the Systems is how accurately they've been able to predict the winners of games so far this season. To do this we'll make the simplifying assumption that the team Rated higher by a System prior to a game is the team that it will select to win (ie we ignore any affects of Home Ground or Interstate Advantage). Also, for the first round we'll credit each System with a score of 4.5 from 9 since Massey, Colley and ODM have no current-season results on which to base their Ratings at that point.

In the chart below all results are shown relative to MARS' performance (note that here, as in the two previous charts, I've "jittered" the results to limit the extent to which lines overlap one another when passing through what would otherwise be the same value). The fact that no line is currently above the zero value on the Score vs MARS axis reflects the fact that MARS has been the most accurate at predicting winners so far this season.

Predictions based on ChiPS Ratings have been next-most accurate after MARS, having produced just one fewer correct result, while predictions made using Massey, ODM or just the Defensive Rankings from ODM have proved next-most accurate after that, some four tips further behind.

Predictions made using only the Offensive Rankings of ODM have been right on six occasions fewer than MARS, and those using Colley have been right on seven occasions fewer.