As many of you know, projections for the Maple Leafs have been a source for a lot of disagreement this year.
There are a few unarguable facts: their shot differential has been among the worst in the league, but the wins have continued to come because they have shot for a much higher percentage than their opponents. The source of the debate is how we should expect these trends to play out in the future.
I wrote a piece two weeks ago noting that their shooting percentage is very high, but not unprecedented in recent history. I observed that each of the previous teams saw their shooting percentage tumble back to the pack, and argued that the Leafs' poor shot differential doesn't leave them much margin for error.
In the comments, someone asked an interesting question: how confident am I that they aren't really as good at shooting as they've seemed? It's easy to say "it's really unlikely that they'll keep shooting this well," but can we quantify "really unlikely"?
They've now played 82 games since the start of last season. That seems like a good time to work through this in more detail.
When we're answering a question like this, we can't just look at the team in isolation; we have to look at what we know about the league as a whole.
I explained why that is (and how to do it) in an article I wrote in October. Reading that article will help you follow the analysis in this piece, but the Cliffs Notes version is as follows:
- Every player or team has hot stretches and cold stretches.
- If you don't account for the known distribution of talent around the league, you conclude that it's equally likely that they ran cold over this stretch as that they ran hot.
- Looking at how talent is distributed, you reframe that question as "is it more likely that they're good and ran hot or are they the greatest shooters ever and actually ran cold over this stretch?" This leads to a different answer.
The Leafs have 184 goals on 1,803 even-strength shots since the start of last year, a shooting percentage of 10.2 percent.
To estimate the odds that the Leafs are really just an average team that shoots 8.1 percent at even strength, we multiply together two probabilities: the odds that a team with 8.1 percent talent would score on 184 of 1,803 shots by random chance, and the frequency of teams having 8.1 percent talent.
We do the same for 8.2 percent, or 8.3 percent, or 8.4 percent and so on. From the results of all of these tests, we can calculate the best guess for their actual talent. But perhaps more importantly, we can also calculate a confidence interval, a measure of how likely our guess is to be right.
We never really know how much of a role luck played in their performance, but the larger our sample size is, the more confident we can be in our estimate of their talent.
Let's start with a look at the Leafs' shooting percentage over this time period. This plot is their cumulative even-strength shooting percentage, so at any given point you can read off their performance since the start of last year.
They've been well above average over this period. Coincidentally, their cumulative even-strength shooting percentage peaked right at the 41-game midpoint of our sample, at 11.1 percent.
For the next step of our analysis, we'll need an estimate of the distribution of shooting talent across the league. For that, we'll borrow from JLikens, who used simulations to estimate the underlying distribution of talent for an analysis very similar to what we're going to do here. He came away with this result:
Between this estimate of the talent distribution and the Leafs' actual results, we have everything we need to follow the method described above to estimate their shooting talent.
I used that method to analyze the Leafs' performance after one game, after two games and so forth up through the 82-game total. So now we can add a line to our plot that reflects our best guess at how good they really are:
Since they've been shooting better than average, our best guess is that their talent is above average. However, having 10 percent talent is still so rare that we still have to believe good luck has played a big role in their results to date.
It might seem crazy to some that we'd peg them as an 8.8 percent talent when binomial probability tells us there's only a 0.4 percent chance of an 8.8 percent team scoring on 184 of 1,803 shots. But JLikens tells us that there's only a 0.0009 percent chance of us finding a team with the talent to shoot 10.2 percent at even strength in the long run.
We already know something really unlikely has happened, and the odds are much better that the unlikely thing was a hot streak than a team assembling the kind of talent we shouldn't even expect to see once per millennium.
Back to the question of confidence in our estimate -- do we know enough about the Leafs yet to be confident that they're really an above-average shooting team? Let's add a 95 percent confidence interval to our plot:
The pinkish curves here represent the uncertainty around our estimate of their talent. The larger the sample size gets, the more confident we are in our estimate. But even after 82 games, the spread on our estimate is still fairly large.
Based on just the last 82 games, our best guess is that in the long run, the Leafs will shoot 8.8 percent at even strength. There's a 95 percent chance that their talent is somewhere between 8.0 percent and 9.6 percent. It's a pretty good guess at this point that they're genuinely above average, but we can't even guarantee that at this point -- though we can be pretty certain they aren't really as good at shooting as they've looked.
So to answer the commenter's question, based on the Leafs' shooting over the last 82 games, there's about a 0.02 percent chance that they're genuinely a 10.2 percent shooting team at even strength. It's actually probably a little better than that -- the league changes over time, which adds uncertainty to our estimates -- but it's almost certainly less than 1 percent and very likely less than 0.1 percent.
There's room for debate on exactly how far above average the Leafs need to shoot to make the playoffs. But if we think the Leafs need to keep up their 10 percent even-strength shooting, they face some pretty long odds.