NHL stats: No matter the method, don’t make snap judgments when evaluating players

Weeks before the finalists for the Hart Trophy were announced, you could already guess the top three finishers; all you had to do was look at the leaderboard for points. There, Sidney Crosby, Ryan Getzlaf, and Claude Giroux filled the three slots, just like they did for MVP voting.

This correlation between points and the MVP award isn't new. In the last nine years, only three skaters have been Hart Trophy finalists without finishing in the top three of points accumulated. And two of those three came in a lockout-shortened season, when point totals were more tightly bunched than usual:

Of course, player evaluation isn't really as simple as looking at points and calling it a day. In the modern analytical era, we have a host of ways to measure a player's contributions, from how much he helps his team outshoot the opponent to how many penalties he draws. Even without those new metrics, point totals themselves are limited. They ignore factors outside the player's control like how much ice time he gets, how well his teammates perform, the quality of competition he faces, and many other factors (including simple random chance).

Yet it's not just MVP voters whose opinions mirror the points leaderboard; scouts do the same thing. A few months ago, someone in an NHL front office gave me his team's mid-season current-performance ratings for every NHL regular (and permission to publish some aggregated analysis of the ratings). After I did some work with the ratings, an interesting trend appeared: The professional scouts' ratings largely fall in line with point totals. That's not entirely shocking -- goals and assists are important after all -- but I was surprised by just how tight the link was.

More Outnumbered

• A look at dump and chase• Replacement goaltending • Zone entry matchup: Giroux vs McDonagh • Measuring defense
• The Corsi arc of a career

The team's rating system had eight levels, and the average point total at each level is shown in the table below. The correlation between the forwards' scout ratings and the number of points accumulated by mid-season was 0.78. That's an extremely high number, in light of how random fluctuations of usage, health, performance, and even lucky bounces can affect point totals.

It isn't really surprising that the scouts place a strong emphasis on the skills that lead to points. Every summer in free agency, players who are the darlings of the stat community get passed over. Metrics such as shot differential favor players like Mikhail Grabovski and Clarke MacArthur, but teams seek out -- and likely overpay for -- players with finishing talent like Mike Ribeiro and Nathan Horton.

What's surprising is that their evaluation of current performance actually isn't terribly predictive of future outcomes; especially at the high end, the ratings seem to have been influenced significantly by short-term fluctuations that may not have much predictive value. The team, of course, recognizes that short-term performance isn't everything; the scouts also prepare separate ratings that reflect a player's perceived potential.

Nevertheless, there is a lesson for all of us in this. I suspect that the scouts are better equipped than the rest of us to see through some of the near-term fluctuations, to recognize who is doing the right thing but happens not to have been rewarded recently. And if their expert evaluations of recent play aren't great at projecting even the near-term future, what hope do the rest of us have at reading anything into short-term streaks?

Every player has good days and bad days. In the long run, the number of good days depends on how good a player is, but whether he happens to perform at his best on any given day -- or even in any given month -- is largely random. For anything remotely forward-looking, even if you're just trying to project the immediate future, you're well-advised to consider more than just the recent past.