The BNP Paribas Open at Indian Wells and the Sony Open at Miami concluded this month, and Advanced Baseline has a curious conclusion from the results of those tournaments: David Ferrer is No. 3 and Andy Murray is No. 4 according to their base ranks. (Rafael Nadal at No. 5 is more due to his prolonged absence than anything else, which seems like a reasonable caveat.) This seems to defy conventional wisdom across the board: Not only has Murray been consistently considered a better player for the past couple of years, but he also had better runs at Indian Wells and Miami, where he won the whole tournament after a 3-set final victory against … David Ferrer.
I promise this isn’t a case of a computer ranking system going against popular opinion and common sense for the sake of seeming different; I’m actually worried that having Murray below Ferrer will hurt AB’s credibility, since it’s pretty easy for people to glance at the top of the list and dismiss the entire ranking system based on such an egregious error at the top. Do I believe Ferrer is a better player than Murray right now? Not necessarily (and I think “better player” is a loaded term in tennis especially, more on that later), but after spending a little time in the details of the match results, I can at least understand how and why AB comes to that conclusion with regards to their base ranks. And the process of going through those details makes for an interesting case study about all the considerations a performance-based ranking system of tennis players takes into account, and why tennis is such a particularly interesting sport to analyze.
Below are Ferrer and Murray’s base rankings for three periods: pre-Indian Wells, between Indian Wells and Miami, and after Miami, along with their results from the two tournaments:
|Indian Wells||Andy Murray||Evgeny Donskoy||5-7, 6-2, 6-2|
|Indian Wells||Andy Murray||Yen-Hsun Lu||6-3, 6-2|
|Indian Wells||Andy Murray||Carlos Berlocq||7-6, 6-4|
|Indian Wells||Juan Martin del Potro||Andy Murray||6-7, 6-3, 6-1|
|Indian Wells||Kevin Anderson||David Ferrer||3-6, 6-4, 6-3|
|Miami||Andy Murray||Bernard Tomic||6-3, 6-1|
|Miami||Andy Murray||Grigor Dimitrov||7-6, 6-3|
|Miami||Andy Murray||Andreas Seppi||6-2, 6-4|
|Miami||Andy Murray||Marin Cilic||6-4, 6-3|
|Miami||Andy Murray||Richard Gasquet||6-7, 6-1, 6-2|
|Miami||Andy Murray||David Ferrer||2-6, 6-4, 7-6|
|Miami||David Ferrer||Fabio Fognini||6-1, 7-5|
|Miami||David Ferrer||Kei Nishikori||6-4, 6-2|
|Miami||David Ferrer||Jurgen Melzer||4-6, 6-3, 6-0|
|Miami||David Ferrer||Tommy Haas||4-6, 6-2, 6-3|
As shown by their before-and-after base rankings, it’s not so much that Ferrer went up as it is Murray went down after Indian Wells. Still, Murray had a better tournament run there, so why did he lose ground according to AB? Let’s look a little closer at Murray’s results. In his first match, a 3-set win over sub-100 Evgeny Donskoy, he played poorly even according to the eyeball test: top-5 players arguably shouldn’t need 3 sets to beat a sub-100 player. The driving force behind AB is reconciling each player’s margin of victory in each match with the strength of their opponent, so if an elite player like Murray squeaks by a mid-level player, that result can and should be evaluated as a worse performance than we’d expect from Murray.
This is analogous to when a basketball team like Gonzaga has a close win against a 16-seed like Southern: Even if they win, the fact that it’s a close game against a much worse team says far more than whether or not they actually won the game, and is grounds for downgrading Gonzaga’s rating on a performance basis. This is in contrast to how the ATP/WTA systems evaluate matches: in their eyes, a win is a win is a win and awards the same number of points regardless of margin of victory and/or quality of opponent. This leaves out a lot of valuable information for evaluating a player’s performance level, even if it means winning a match by low margins can count against you.
Murray’s next match was a routine 6-3, 6-2 win over Yen Hsun-Lu, about the level of play we would expect from a top-5 player. His last match, however, was a close 7-6, 6-4 win over Carlos Berlocq. While Berlocq is a better player than Donskoy and Murray didn’t need 3 sets, there’s another consideration that ends up hurting Murray here: surface factors. Murray is a solid hard-court player; Berlocq is a clay specialist: this is affirmed by both conventional wisdom and the surface factor numbers listed alongside each player’s base rank. A solid hard-court player going against a clay specialist at Indian Wells, a hard-court stadium, has something close to home-court advantage, so all else being equal, we would expect Murray to roll here. AB incorporates surface considerations by adjusting the actual score of the match according to the surface preferences of each player, similar to how we would subtract three points from the home team in a football or basketball game to produce an “effective” score. Again, even though Murray won the match, his scoring margin wasn’t as good as we would expect it to be after taking surface into consideration, so his rating gets dragged down again.
Murray’s and Ferrer’s losses were fairly similar: three-set matches against Juan Martin del Potro and Kevin Anderson, both top-20 players (according to AB) and solid hard-court players. The difference is that Murray’s loss happened late in the tournament, and Ferrer’s was a first-round upset. If you roughly translate the match results into golf terms, you get an interesting tally: Murray has bogey, par, bogey, bogey, and Ferrer has a single bogey, which leaves it at Murray +3 and Ferrer +1. This is why AB is grading Murray more harshly than Ferrer by their Indian Wells results: even though Murray had more wins, two of those three wins counted against him performance-wise, so the winning matches were a net negative for Murray. And since Ferrer went out early, he didn’t have the opportunity to play any more matches, so he couldn’t have any more “bad” wins count against him like Murray did.
This is one of the many quirks of analyzing tennis: Nearly all the tournaments are single-elimination format, so you can have instances where an early not-all-that-bad loss has a chance of being beneficial according to a performance-based ranking system. It can stop you from accumulating detrimental losses later on in the tournament. Now, an early-round loss also prevents you from picking up future “good” wins as well (and your upside from those potential good wins will absolutely outweigh any potential bad wins and/or losses), so you’re not automatically incentivized to lose according to AB. But if you do play poorly in those later matches, that’s additional information that should absolutely be incorporated one way or another.
Okay, so even if it’s understandable that AB counts all of Indian Wells as a net negative for Murray, is a single bad tournament really enough to push him below Ferrer? The analysis period for AB is two years, so it seems like it’s overreacting to a single tournament. I would agree that there’s probably some overreaction going on, but I think it’s because of another quirk of tennis that’s extremely difficult to control for. There are some other important match results to consider that are not shown in the table above: the ones between the Australian Open and Indian Wells.
Ferrer won Buenos Aires, lost in the finals to Nadal at Acapulco and had convincing margins in his wins; Murray skipped all tournaments between the Australian Open and Indian Wells in favor of training. This is an example of one of the biggest challenges in analyzing the performance record of tennis players: You will have drastically different amounts of information for each player. Some of this is due to the single-elimination tournament format, where a loss prevents you from playing any additional matches (and therefore reduces the amount of future data available for each losing player). Some of it is also the irregular schedule of each tennis player: if they’re hurt, or they just don’t feel like playing a particular tournament that isn’t mandatory, they don’t have to show up. This is in stark contrast to most other sports, where teams have a pre-set schedule for the season, ensuring consistent and reliable streams of data for each team. There’s not much you can do to get around this. The best you can hope for is to figure out how to weight all your variables and considerations that churn out the best long-term results, and have an understanding that the variability in data will make your results more volatile than other sports. Murray’s post-Indian Wells plunge is an example of this kind of volatility; without any other results to mitigate a poor Indian Wells run, his base ranking was more vulnerable to a short-term downswing than someone with a buffer of good results like Ferrer.
Okay, maybe you’re still on board that Indian Wells is an understandable speed bump for Murray, but surely Miami was enough to bring order back to the universe, right? It ended with Murray beating Ferrer, so if that isn’t enough to put Murray back on top of Ferrer, how good can AB really be? The pre-finals matches for both players were largely similar: solid wins against lesser players and understandable 3-set victories against top 10 players, so I think it’s fair to consider them a wash without going into too much more detail. As far as the final match: yes, Murray’s base rank did improve as a result, and it’s reasonable to wonder if it “shouldn’t” have gone up enough to surpass Ferrer. If you think it should have gone up further than it did, here are some questions to consider, though:
- Ferrer actually won more games than Murray in their match- how does that influence how much of a bump you think Murray should get?
- Murray is a better hard-court player than Ferrer, so Murray began the match with a natural advantage; how does that factor into how much of a bump Murray deserves?
- Ferrer was literally inches away from winning the match on a championship point he challenged that was in by a sliver. How robust is your opinion relative to those inches- if that shot went out, do you think Ferrer would’ve deserved an equally large bump in his favor?
If Murray had beaten Ferrer in straight sets by a convincing margin, I would agree with Murray moving ahead of Ferrer (and just to satisfy my curiosity, I re-ran AB with an alternate result of a 6-3, 6-2 Murray win, and that would’ve been enough to push him ahead). But the fact is, he didn’t. And when you evaluate those results according to the rules that make up your ranking system, sometimes you’ll get transient results like these.
So that’s the comprehensive explanation of why AB has Ferrer’s base rank higher than Murray right now. All of this technical discussion misses what I think is the more important point, though: a nice-and-tidy rank order of players is a pretty meaningless endeavor in tennis. The ATP/WTA rankings (and AB taken at face value, for that matter) do a great job of presenting this illusory idea of an “overall” ranking for each player: an idea that breaks down almost instantly once you account for surface conditions.
Before Murray and Ferrer’s match, I had Murray at a 67 percent favorite after taking surface factors into account; I have it at about the same after. I also think Ferrer is over 80 percent to win that same match on clay, and less than 25 percent on grass. How do you encapsulate that in something like an ATP/WTA ranking? I get that it’s nice to have a simple list that acts as a shortcut around all that nuance and just gives you the cliffs notes version by saying who generally wins more than others, but I think relying on that shortcut does a disservice by ignoring the part of the sport that’s so interesting to analyze.
That’s why I’m not worried about which players appear above or below other players on the AB front page. I know that taking the more complicated, surface-adjusted view produces a big-picture understanding that’s much more accurate in the long term, even if it raises a couple eyebrows at first glance. If there’s only one takeaway from all of this discussion that’s worth mentioning at the end, it’s probably this: There is no such thing as an overall ranking in tennis.