Margin of error is a huge component of statistics. When you see a political poll with candidate A leading Candidate B by 6 points, with a MoE of 3 points, you can be pretty confident that Candidate A is actually ahead. 

Why don't we think in similar terms for baseball projections? Witness this exchange:

Anderson — 3.63
Cahill — 4.53
Mazzaro — 4.53
Braden — 4.58
Tomko — 4.65
Outman — 4.75
Edgar — 4.79
Eveland — 5.22
Gio — 5.88
Mortensen — 6.28

 

So ZiPS thinks that Cahill will be just fine in the near future, no matter how bad he was in the beginning of the year. It predicted that he’d improve in the second half and he did, by K-rate and K/BB.

Clearly he’s one of the 5 most talented starters on the team and talent mostly wins.

This is very typical analysis at AN (and I love it--I learn from this kind of stuff all the time), but I have a question. The conclusion above is that Cahill is "clearly" one of the 5 most talented starters, since his ZIPS_ros is .26 lower than our 7th best starter (E-Gon). That seems compelling, but what's ZIP's margin of error?

Citing a statistic/projection without stating its confidence level strikes me as basically meaningless. For example, if a player is projected for a .760 OPS, but with a confidence interval of [.660 .860] my conclusion would be that the projection is 100% meaningless.

Unfortunately, I downloaded the Zips_ros spreadsheet, and it is literally a black box. This is akin to saying Obama leads McCain by 5 points, but refusing to specify any sort of margin of error. I'm sure Zips has some explaination of Margin of error somewhere, but I couldn't find it in 10 minutes of searching...

I've noticed that we talk about the components of the metrics a lot but we never talk about their precision. And, when the projections are wrong, we shrug, and say, well, that's luck. And that's fair- no model can project the future perfectly. But Confidence Intervals and Margin of Error are tools designed to let us know how reliable a model is. I think we should give more scrutiny to arguments that cite fairly small differences in projections and fail to provide any sort of margin of error analysis. Lets look at what the Margin of Error for two projection systems are:

Chone provides some interval analysis, via its "10th percentile" through "90th percentile" projections. By the way, for Chone, the daring prediction for Giambi was that he would OPS somewhere between .700 and .960...that's a HUGE interval. It had Rajai between .550 and .770 on OPS. With intervals that large, I'm going to (inwardly) laugh every time I hear Chone cited.

PECOTA  (using Phillies because I'm not a subscriber)

Raul Ibanez: 10th percentile: 740 OPS,   90th Percentile: 946 OPS

Jimmy Rollins: 10th: 735 OPS     90th: 915 OPS

Chase Utley: 797-987

Matt Stairs: 809 OPS (50th percentile)

These ranges are so large that you can't make a meaningful conclusion that any player is better than another. These projections should be read as "Rollins is projected for an 824 OPS, plus or minus 100 points. LOL. Since Stair's median OPS is within Utley's 80% confidence interval, statistically speaking, Pecota cannot be used to conclude that Utley is a better hitter than Stairs.

What am I missing? I like statistical analysis as much as anyone, but these intervals suck. Citing these stats is akin to citing a poll that lists Obama ahead of McCain by 3 points, plus or minus 15 points.

And, if I'm not missing something obvious, why are these metrics more reliable than a 40 year veteran scout? I bet he could tell you Utley is likely to OPS between 800 and a 1000. I bet he could have told you Rajai would hit better than a 550 OPS, but that 770 was near the peak of his potential.

And you know what? Lots of the ANers who go to games could have told you that too.

Disclaimer: This is a long post, with a lot of text, and some charts that require interpretation. It also has ambiguous conclusions, because of some hard-to-account for errors. There's a solid chance you will feel this is a waste of your time. I do believe it can provoke good conversation, and I also want to post this anyway, since I did it. Please do skip this if you don't want to think about it. Thanks.

The A's have played some good teams, including Toronto, Tampa Bay, Boston, New York, Texas, and the Angels. These teams may represent a stronger early schedule than typical for the AL. On top of that, with injuries, call ups, and random luck, each player on the A's has faced an extremely varying set of opposing batters and pitchers. For example, Anderson has pitched against tougher lineups than Cahill.

When we evaluate the performance of the A's personnel, we don't correct for "strength of schedule". For example, we look at Braden's .743 Allowed OPS, and think "Ok, above average". The AL average OPS is .766 in 2009. What we don't account for is the fact that the batters Braden has faced are actually averaging a .792 OPS so far in 2009, making Braden's pitching exceptionally good.

In this diary fanpost, I'm going to use Baseball Prospectus' data on the Average OPS of batters faced for our pitchers, and compare it to the OPS our pitchers have actually allowed. This will allow us to correct for the possibility that the A's have faced tougher lineups, and for the possibility that some A's have faced particularly tough tasks. I'll also do a similar analysis on the quality of pitching our hitters have faced.

The conclusions of the analysis (here ahead of time for clarity) are:

1. The A's have faced tougher than average (or hotter than average) lineups, and better than average opposing pitching.

2. Some A's pitchers performances thus far, such as Bailey, Braden, and Anderson are better than they appear, since they faced tougher opponents.

3. Cahill and Wuertz have faced below-average opposition, and thus are  worse thus far than they appear.

4. Powell, Holliday, Ellis, and most of the A's lineup have faced exceptionally tough pitchers, making them look worse than they are.

5. Nomar was not as good as he appeared, as he faced weak opposing pitching.

Finally, these conclusions are seriously dependent on a few key assumptions. A list of these assumptions, and defenses of them, are in Comment #1  (in order to avoid bogging down here). I expect that AN will find many more flaws in this work than I cover in comment #1. On to the real analysis...

Continue reading this post »

First, this is generally not a response to Nico's post. I'd be hard-pressed to answer most of his well-stated points. What I really disagree with is the overstated criticism I saw in the comments of that post. I also think there's a reasonable defense of Beane's treatment of 3B.

Beane made a foreseeable mistake with Geren. Unlike Nico, however, I think the 3B question was murky, and that Beane's approach is defensible--I think that Beane would be wrong to trade an asset for a 3B before he knew Chavez was done for absolutely sure, and that Garciaparra would not stay healthy. He can still trade the exact resources he would have last season. He can trade Gio, Gallagher, or Outman for a 3B solution. It is a mistake to make it an absolute requirement that he acquire a 3B last year, and to say he was short sighted for not doing so. FIrst, it's always a mistake to have absolute requirements; if the best offered talent was elsewhere...take it, and deal w/ 3b later. Second, and I can't stress this enough: we're not out of fungible assets just because we traded Haren, Harden, and Blanton. We STILL have resources- the talent Beane got in the trades I outline below.

The motivation for this diary comes from the fact that much anger is being directed at Beane, when, in fact, he's the only thing standing between us and the Royals(where one good start is huge news). I understand the frustration at 3B, and with the Holliday trade. However, some things (like the success of a team) require a larger sample size than a 3 year period to evaluate. A confluence of bad luck can happen to any GM, and make any 3 year period look good or bad. Think about how many bad breaks the A's have had. NOBODY disliked the Chavez deal at the time it was inked. The recent trades and deals, to me have been "fucking A" trades...and most people saw it that way at the time. The recent strain of thought is that those who defend Beane are in denial. To them, I say: Check the recent track record. It's not perfect but it's far above average. To suggest that Beane has checked out of baseball ignores just how good the "Rebulding" trades really were.

If you don't want to see a summary of trades you've already read about ad nauseum, now is a good time to skip to the comments. I just thought a reminder of recent history might be useful a this dim hour.

Continue reading this post »

So, this all started with my dislike for the "Playoffs are a crapshoot". I don't like the argument, because

1. I like the thought that the playoffs select the best team.
2. I think that the playoffs aren't completely random...the better team has a better chance of winning
3. I fear that thinking of the Playoffs as a crapshoot causes complacency.

The feedback I got is that those 3 arguments are semantics, obvious, and/or wrong. I'm perfectly fine with that- This was my gut feeling, and actually, I'm pretty much done with those arguments. 

The best explanation for the "Playoffs are a crapshoot" argument, I thought, came from WhiteElephants (near bottom)

the best strategy might not necessarily be to put together the best team in any given season if it means sacrificing playoff berths in future seasons. I think the main point in Moneyball is that the best strategy seems to be: just get to the playoffs as often as possible.

Anyways, I agree with this strategy...definitely want to get into the playoffs. So why am I writing a diary? Well, I wanted to look into how well regular season preformance predicted playoff outcomes.

Continue reading this post »

After all the signings, a look at the salary cap situation for the next 6 years.

Continue reading this post »

So I've been wondering why Pitchers like Ervin Santana dominate us, and at the same time, we hit decently against better pitchers such as AJ Burnett. My theory, which may not be surprising to many of you, is that pitchers with good control (who do not issue walks) absolutely destroy the A's. Even though this may be obvious, I want to take a closer statistical look at the issue.

Continue reading this post »

In my opinion, recent diaries have shown some of the ugly side of this community. The best example is the number of personal attacks on Iggy in recent diaries. He's been called a "numbskull". He's been accused of slinging BS, called a joke. He's been told to stop talking because he keeps getting shut down.

And honestly, the sad part is that most of those comments came from highly regular, respected members of this community.

This community should value attacking the argument, not the person. That's the only thing that makes this place different from an ESPN board.

I'm not sure what can be done, besides a general plea for people to explain why the other person's argument is wrong, rather than calling them out personally. I also think that regular contributors need to build that mentality by never making ad-hom arguments themselves. This is not purely an issue of moderation, but really, an issue of commitment from regular community members to help create a stigma against ad-hom arguments. The next time you hear someone insult another community member, ask them to debate the argument, not the person.

Although this issue has to be self regulated by the community, there are couple things I think we should try to do in terms of site policy/organization:

Continue reading this post »

Foyle's contract pays 8.9 million this year, 9.8 million next year. There is a 10.5 million team option after that.

In the event of a buyout, the sum total of the buyout is divided over the remaining years of the deal, and applied to the salary cap evenly.

The Warriors have less of a salary cap crunch in 2009-2010 then they do in 2008-09. This is because Monta and Bean's new contracts first hit in 2008-2009, and by 2009-10, Foyle is gone, and Baron is gone (pending an extension). Now undoubtedly, if the W's can squeak under the Lux Tax in 08-09 after signing Biedrins and Monta, they'll make it in 09-10. I believe using that fact can help the Warriors buy out Foyle.

Continue reading this post »

The 10 million exception is a dilemma. One of the biggest advantages of sending JRich packing was the cap space created. That space can be the key to resigning Monta and/or Biedrins next summer.

Continue reading this post »

The Salary Cap affects our decisions regarding the Draft, including whether or not we trade up.

Continue reading this post »