On Wednesday, I described a set of scout ratings I received from an NHL team, and how closely forward ratings correlated to recent point totals.
An interesting wrinkle is to look at the places where the correlation between point total and rating wasn't as tight. In so doing, I hoped to get a sense for what type of player is perceived as being significantly better than his point total, to get a window into what the team values beyond point totals.
To assess this, I looked at the average points scored per 60 minutes of 5-on-5 play by all forwards with a given rating, and then looked at how far above or below that mark each player was. So what kind of player grades out much higher than his point total?
It's not a player who drives possession. If I sorted players based on how far above (or below) their expectation they were, the 20 guys at the extreme "rated higher than their point total" end had an average Corsi of 46.3%, while the 20 guys at the extreme "rated lower than their points" averaged a Corsi of 52.5%. If anything, a high Corsi seems to correlate with a player being rated less than his point total would suggest.
I tried some other things. Looking at fights, ice time, hits, team record...nothing really seemed to explain the discrepancies between point total and rating.
The place where I found the biggest impact was age. Younger players consistently scored more points than their rating, while older players scored fewer:
Nearly two-thirds (66.5%) of players age 18-26 scored more points than their rating would suggest, while barely one-third (33.7%) of players age 29 and up did. There's a clear tendency to value veterans more than their points would suggest. What's not clear is why.
Do those veteran players play tougher minutes, meaning that equivalent production would actually be a superior accomplishment? Do they play better defense, meaning that equivalent offensive production would warrant a superior overall rating? Are the scouts capturing some inherent perceived value of the veterans' experience? Do the scouts underestimate the decline of players who are past their peak? Or is there just a bias towards favoring players who have proven themselves out over a longer career?
Probably it's a little bit of all of the above. There's almost certainly some effect of role -- the scouts recognize, after all, who's expected to score and who's on the ice to play defense. However, the apparent impact of age on rating was significantly stronger than the impact of usage measures such as quality of competition or zone starts.
So while we can't say for sure without getting direct answer from the scouts who prepared the ratings, it would seem that usage alone doesn't account for all of the observed effect, that, whether implicit or explicit, there is a tendency for the scouts to be more kind towards the older players.