We've talked about defense around here a few times before. We've talked about measuring individual player defense, we've talked about measuring team defense, and we've talked about some of the questions and limitations. To be sure, evaluating defense continues to be baseball's most hotly-contested issue in many fan circles, as few are very satisfied with the metrics we have at our disposal.
A problem is that the metrics themselves are based on an iffy foundation. They depend on things like batted ball location and batted ball speed, and these end up being subjectively recorded on the fly. That subjectivity introduces great error into the results. Throw in the fact that current defensive metrics don't account for player positioning and you have all the ingredients for a raging debate.
In large part for those reasons, statistical wizard and published author tangotiger set up what he calls the Fan Scouting Report. In the Fan Scouting Report, fans grade defensive ability based on their own visual observation. Here is a sample ballot. Fans are asked to grade each player's reaction, acceleration, velocity, hands, release, throwing strength, and throwing accuracy by choosing a number 1-through-5 from a pulldown menu in each category.
This sounds kind of weird at first. Why should we believe what fans have to say about player defense? The key isn't in the individual ballot. It's in the collection of ballots. The project is founded on the principle of the wisdom of crowds. According to this principle, the belief is that, if you collect the input of enough intelligent and frequent observers of player defense, then by averaging all of their results, you approach the true answer.
Whether or not you believe in the Fan Scouting Report is entirely up to you. However, it is not without merit, and the results are most definitely interesting. By looking at the results, we can see which players fans believe have the best hands. We can see which players fans believe have the worst first steps. We can see which players fans believe have the most accurate arm. And, of course, we can see which players fans believe are the best and worst defenders in baseball.
The 2010 Fan Scouting Report is still going on. Ballots are still being submitted, so some of the results may change a little bit. Enough ballots are in, though, that we can take a look to see how people are thinking. And, without further ado, these are the players that the fans think are the best defensive players in the league:
Evan Longoria, 88 rating based on 79 ballots
Ichiro Suzuki, 87 rating based on 112 ballots
Carl Crawford, 86 rating based on 72 ballots
Troy Tulowitzki, 86 rating based on 33 ballots
Brandon Phillips, 85 rating based on 60 ballots
The Fan Scouting Report is positionless. Fans are asked to vote independent of position. What we can interpret from this, then, is that, among the ballots submitted so far, fans believe that Evan Longoria has the best blend of defensive skills in baseball, narrowly ahead of Ichiro and a pack of other talents.
Is it bulletproof? It isn't bulletproof. Longoria, for example, finished well in the 2009 Fan Scouting Report, but he didn't finish at the top. There's always the question of fan bias, as Rays fans or Mariners fans might be a little more biased in favor of their players than, say, Mets fans, or Orioles fans. And there's the matter of visual observation being subjective, and therefore unfalsifiable. These ballots aren't proof that Evan Longoria is the best defensive player in the league.
But they do serve as evidence. For Longoria, and for all players rated, these ballots are evidence. The statistical metrics we have for defense these days are riddled with holes. The Fan Scouting Report helps contribute to a better understanding of just how good or bad some players really are.