DONETSK, UKRAINE - JUNE 27: Cristiano Ronaldo of Portugal looks dejected after losing a penalty shoot out during the UEFA EURO 2012 semi final match between Portugal and Spain at Donbass Arena on June 27, 2012 in Donetsk, Ukraine. (Photo by Martin Rose/Getty Images)
Having Cristiano Ronaldo waiting for the tenth and final kick didn't work out for Portugal against Spain, but how can we tell whether or not it was the right call to make?
Cesc Fabregas breathed deep, took a long run-up, and slammed his penalty kick in off Rui Patricio's right-hand post. Spain are through to the Euro 2012 final, and Portugal are out. Nine kicks were taken in the shootout; none were taken by Cristiano Ronaldo.
The Real Madrid megastar, widely regarded as the best player in the tournament and Portugal's main hope of advancing, has been raked over the coals for not having participated in the shootout. He was, of course, set to kick in the fifth spot for his country, but by that time Spain had already won thanks to Bruno Alves hitting the crossbar with kick number eight.
While there was plenty of talk of 'selfishness' and 'playing the hero', the optimum way of setting up a lineup is not immediately clear from the nature of a penalty shootout itself. It's obvious that, in general, you want your best kickers to be taking the most important penalties. It's far from obvious what those are.
It's easy to make subjective arguments for the various slots, but without any solid evidence either way, we're still left in the dark. Was putting Ronaldo in at number five the wrong decision? Or was it a good call that just didn't work out?
One way to measure the relative importance of penalties is to derive what's called win probability for each state of a shootout. This is actually fairly easy to do for penalties using what's called a binomial distribution - given the average conversion rate (73.7% for all shootouts in World Cup and European Championship history) it's straightforward to build up a map of how likely a team is to win from every possible shootout combination.
It's then possible to see how important an individual kick is. Let's start with kick number one. Broadly speaking, the consensus is that the team that kicks first wins sixty percent of the time; this was a finding that doesn't turn up in international tournament data, presumably for reasons I'll get into later - this data showed nothing to indicate that penalties are anything but a fifty-fifty proposition.
Given that, what's the chances of a team winning if they score the first kick? How about if they miss? In the first case, it's about 57 percent, and in the latter it's 29 percent. The possible percentage swing, then, is roughly 28 percent of a win. This is an easy example, since there's only one scoring state possible at kick one, and that's 0-0. The rest of them get more complicated, so we'll refrain from doing some time-consuming maths here.
How does this tell us the importance of any given penalty? Each penalty has two outcomes, saved and not saved. The spread in win probability between those two outcomes is a basic measure of the pressure of a specific kick. If you know the spread in win probability for all possible penalties in a shootout, you can then compare them to spread of the average penalty kick (which turns out to be the 28 percent from kick one). Dividing the delta win range from an individual kick into the average will give you what we'll call leverage index, a measure you'll be familiar with if you're a statistically-inclined baseball fan.
What did the win probability and leverage index information look like for Wednesday's shootout? Well, it looked a little bit like this:
Kick number ten, of course, never actually happened, and thus had a leverage index of zero. Cristiano Ronaldo had no chance to have an impact on Portugal's odds of winning the shootout. But the above graphs are just from one game, and doing a retroactive analysis of one shootout to find out if a guy who didn't take a kick shouldn't have been placed tenth overall is a little bit silly, especially as we have so much history to play with.
What happens if we look at the leverage of every single kick taken in World Cup and European Championship play? We can find out the frequency of each kick being taken (there are no guarantees after kick six) as well as leverage, giving us an overall importance level of any given spot in the lineup:
Don't mind eleven through eighteen; they're just in there for completeness and as a demonstration that you really do want to put your five best penalty kick takers in the front five. Anyway, as you can see, there's some definite variation in the average leverage of kicks, but things really fall off at number ten. In other words, that's where you want to disguise your least confident or competent taker out of the chosen five*.
*As a brief aside, in the shootouts sampled, which should be the most stressful any player can reasonably face, the average conversion rate had essentially zero correlation with the stress of a situation, a distinctly unintuitive result. This, presumably, is why the first kicker is not favoured in the sample.
That's presumably (although I don't have quite enough temerity to claim to peer into his mind and read his thoughts, as so many are currently doing) not how Ronaldo saw it, since if the tenth kicker does get to take a penalty, it'll be the most important of the match. Here's that chart for penalty kicks that are actually taken:
Ronaldo was signed up to take what would have been the most important penalty of the match, and Paulo Bento essentially gambled that he'd be there to take it (assuming you think he's the best kicker of the five - personally I have no idea, although Bento himself probably should). That's not cowardice, per se, but I don't think it was a wise move. Given the above, we can start deriving a basic set of rules by which teams should be approaching penalty shootouts:
If you don't know which side kicks first, you want to average out the leverage for each kick pairing. You'd then set out your kickers from best to worst as: #4, #2, #3, #1, #5. If your side kicks first, the way to do it is #5, #2, #4, #3, #1; second is #4, #2, #3, #1, #5, the same as if you didn't know at all.
This wouldn't be a huge effect - you're probably only talking about swinging the odds a few percentage points one way or another. But all of the above indicates that teams should seriously look at fielding two possible lineups depending on how the coin flip goes. I wouldn't be overly worried about kicks one through four for each team, but that fifth pairing is worrisome. If Ronaldo really is Portugal's best spot-kick taker, Bento made an avoidable - and not insignificant - error in having him kick tenth.