SBNation.com User Blog: Systematic
http://www.sbnation.com/users/Systematic
Posts made by Systematic on SBNation.com 36 Game Check-up
http://www.athleticsnation.com/2013/5/11/4322728/36-game-check-up
SystematicSat, 11 May 2013 19:54:13 -0400
<p>So, a lot of us are wondering what the hell is going on so far. This team leads the league in runs scored, but also has been shutout more than any other team. I seriously can't recall a "normal" game where we won 4-2, pitched well, manufactured a couple runs, and hit one (singular) homerun... just a quality start and an offensive performance that mirrors the way a "good team" would win ball games day in and day out. I wrote an article in the offseason about how many wins/losses you could expect from a team that distributed runs the way the 2012 A's did, and the amount of variance you could expect from chance alone.</p>
<p>So what's the problem, well it's still early... the variance of our run differential is high (mostly because we've only play 36 games), but let's take a look at the difference between this year and last year.</p>
<table cellspacing="0" border="1" cellpadding="0">
<tbody>
<tr>
<td valign="top" width="343"></td>
<td valign="top" width="138">
<p align="center">2012 Oakland A's</p>
</td>
<td valign="top" width="157">
<p align="center">2013 Oakland A's</p>
</td>
</tr>
<tr>
<td valign="top" width="343">
<p>Average Run Differential</p>
</td>
<td valign="top" width="138">
<p align="center">.5568</p>
</td>
<td valign="top" width="157">
<p align="center">.1351</p>
</td>
</tr>
<tr>
<td valign="top" width="343">
<p><a href="http://en.wikipedia.org/wiki/Standard_deviation">Standard Deviation</a> of Run Differential</p>
</td>
<td valign="top" width="138">
<p align="center">4.17</p>
</td>
<td valign="top" width="157">
<p align="center">4.92</p>
</td>
</tr>
<tr>
<td valign="top" width="343">
<p>Correlation of Runs Scored/Allowed</p>
</td>
<td valign="top" width="138">
<p align="center">0.027</p>
</td>
<td valign="top" width="157">
<p align="center">-0.16</p>
</td>
</tr>
</tbody>
</table>
<p>What stands out? On average we're winning by less than we were last year, sign of a team that isn't as good (don't worry, at this point this doesn't mean much). There's more noise in our data- wins/losses are by a larger margin, which makes it hard to tell how good we really are, and oddly the correlation of our runs scored and allowed is negative- when we score a lot, we give up a little, and vice versa. If you're pitching performance has nothing to do with you offensive performance on the same night, you'd expect a correlation close to zero, like we saw last year.</p>
<p>What to expect? Well, nobody knows what the run differential will be by the end of the year. I'd love to see .5568 again but time will tell. Here's an example of how our uncertainty in what our current run differential changes throughout the season due to the increase in sample size. Remember, this year's run differential is really just a sample (36 of 162) and even a whole season is really only a sample of a team that plays an infinite number of games.</p>
<p><a href="http://cdn3.sbnation.com/imported_assets/1601973/8730353648_76dd5b58af.jpg"><img class="photo" src="http://cdn2.sbnation.com/imported_assets/1601973/8730353648_76dd5b58af_medium.jpg" height="271" alt="8730353648_76dd5b58af_medium" width="595"></a></p>
<p>In short, we'll have about twice as much confidence after 162 as we do after 36 games. Right now the <a href="http://en.wikipedia.org/wiki/Standard_error">standard error</a> is about .75 runs per game and after 162 games it will be .35 runs per game. You don't need to understand this chart, just visually seeing the impact of an increased sample size should help conceptualize things. If you're asking wouldn't we gain twice as much confidence going from 81 to 162 games? No, because the confidence doesn't change linearly, but rather by the square root of time.</p>
<p>If we assumed this year's team is exactly as good as last year's team we probably would have won 22 games (most likely). But at the same time we'd have 18 wins or less 19.90% of the time - this is assuming we simulated 1,000 seasons with a run differential and variance of run differential identical to last year's team. We're a little unlucky thus far, but it's not exactly like being struck by lightning either. If we keep scoring and giving up runs the way we currently are, we're probably going to win 86 games with about a 5% chance that we win 94 games or more... but that assumes we don't play any better than we have been, I think this team has better baseball in it than what the first 36 games have shown. These estimates change with every game we play, and if we did the same thing at this point in time last year we probably would've been on pace to lose 100 games.</p>
<p><a href="http://cdn1.sbnation.com/imported_assets/1601979/8730353606_ba88d58b18.jpg"><img class="photo" src="http://cdn2.sbnation.com/imported_assets/1601979/8730353606_ba88d58b18_medium.jpg" height="366" alt="8730353606_ba88d58b18_medium" width="617"></a></p>
<p>So, what to take from all of this? Nothing really, I just wanted the waste some of your time and make you think a little bit. There are a few odd things thus far that should change by the end of the season, we're a little unlucky thus far and it's too early to really tell much. Hopefully the starting pitching stabilizes a little bit, that's been the difference from a fundamental standpoint. We know the offense strikeouts a lot, and hits a lot of homeruns and the bullpen is lights out. Our starting pitching is just distributing runs too wildly. Our wins should be a function of the offense's wild run distribution with the pitching being fairly constant- instead they're both wild leading to the uncertainty and prolonged win/loss streaks. If/when this happens we'll starting playing more of those 4-2 games that help us sleep a little better at night. It's really not that bad of a start thus far. We're doing a lot right, just lacking some consistency.</p>
Separation Anxiety- luck vs skill
http://www.athleticsnation.com/2013/1/13/3872748/separation-anxiety-luck-vs-skill
SystematicSun, 13 Jan 2013 13:32:57 -0500
<img alt="" src="http://cdn2.vox-cdn.com/uploads/chorus_image/image/6526885/152573584.0_standard_400.0.jpg" />
<p>Dogs, babies, needy spouses- they all get it, and to tell you the truth, I get it too... separation anxiety, overwhelming fear and cold sweats cover my body when I think about separating how much of 2012 was luck vs skill. Why do I feel this way?</p>
<p>2011- we sucked</p>
<p>2012- we should have sucked to the 10th power, but we were demigods (coco reference).</p>
<p>Hence, I kind of don't want to know how lucky or skillful last years team was. To preface, I work in finance- the data are extremely volatile, it's noisy, it takes 20+ years of data just to prove that stocks have higher expected returns than one month t-bills or bank CD's. In short, its a lot like baseball. Every event is really a series of odds, probabilities, coin flips- and you need to flip the coin to point you that your right thumb is the size of Yoenis' deltoids to have any sort of a clear picture of what is luck vs skill.</p>
<p>So how do we look at the data? I ran <a href="http://en.wikipedia.org/wiki/Monte_Carlo_method" target="_blank">monte carlo simulations</a> (binomial wins and losses, and normally distributed run differentials) with the same characteristics as a .500 team, 94-68 team, and the 2012 A's. I pretend these teams play 1,000 seasons with identical odds each game of each season and check for the variation in wins/losses by chance alone. These number's represent constant odds- they don't change, if a simulated season starts with 5 wins you have no better or worse odds of winning the next game than the assigned odds at the beginning of the season, there's no mean reverting, which I meet with healthy skepticism even though every post I read has some B.S. about players x,y, & z "regressing to the mean".</p>
<p><b>Simulation 1: A team with 50% odds of winning each game plays 1,000 seasons:</b></p>
<p>Obviously, the mean median, and/or mode are 81 wins.</p>
<p>Only 2% of the time did this team win 94 or more games. Those are the odds that we were a .500 team last year that was incredibly lucky. Yep, there's that much noise in wins/losses under these assumptions.</p>
<p>68% of the seasons the team won between 87-74 games, 95% of the seasons it was 94-68 wins.</p>
<p>You might think that it's crazy that a .500 team can win 68 or 94 games- I think the data are just that noisy, if mean reversion exists it'd tighten these confidence intervals, but that's a whole other topic.</p>
<p><b>Simulation 2: A team with 58.02% odds of winning each game plays 1,000 seasons:</b></p>
<p>This is a 94-68 team, obviously because the expected win percentage is 58.02% 94 wins is the center of the distribution.</p>
<p>3.7% of the time this 94-58 team wins 81 or less games.</p>
<p>68% of the seasons the team won between 101-88 games, 95% of the seasons it was 107-81 wins.</p>
<p>Considering the <a href="http://www.sbnation.com/mlb/teams/seattle-mariners" class="sbn-auto-link">Mariners</a> won 116 games in 2001, I don't think the 107 win probabilities are any cause for concern. I mean, if maybe 8 teams per year have expected winning percentages as high as 58.02% and we have 50 years of data with 162 game seasons and no regime change in the level of competition we only have 400 seasons in history, where as we have 1,000 in these simulations.</p>
<p>OK, now we have some feel for how much wins can vary by chance alone.</p>
<p>Now lets test seasons based on simulating run differentials rather than the probabilities of the games themselves. Each game that has a positive run differential is a win, while negatives are losses. I know you can't score a partial run in baseball, but if we round the simulated numbers our conclusion would be the same, we just can't round them before hand because it'd bias the simulation greatly.</p>
<p>The 2012 A's had an run differential of +0.5569 runs per game with a standard deviation of 4.27 runs per game (this measures how much the run differential varies per game, we use it in the simulation so that the behavior of the simulated games matches that of real life..</p>
<p><b>Simulation 3: A team with a .5569 run differential and standard deviation of 4.27 runs plays 1,000 seasons:</b></p>
<p>If we simulate games based on the SAME distribution of the 2012 A's run differential the expected out come is an 89 win season and there's a 26.5% chance that this team wins 94 or more games. I'd probably wager that this is a better expected value than the 94 wins we saw, the question is, is run differential a good predictor of wins? And 26% isn't exactly winning the lotto. There's a 12.9% chance a team with this run differential distribution wins 81 or less games too.</p>
<p>What if we simulate a .500 team (zero expected run differential) with the same standard deviation as the 2012 A's?</p>
<p><b>Simulation 4: A team with a 0.00 run differential and standard deviation of 4.27 runs plays 1,000 seasons:</b></p>
<p>Obviously the center of the distribution is 81 wins, half the time you do better, half the time you do worse. Only 2.10% of these teams won 94 or more games, so it's pretty safe to say that we're probably better than a .500 team.</p>
<p>What's interesting is that simulations 1 & 4 are the same team (.500 record) over 162 games they both have expected wins of 81 games and about a 2% chance of winnings 94 or more games.</p>
<p>The A's run differential doesn't predict their record however, its only predicts 89 wins. If (and that's a big "if") run differential is the best estimator of how good a team is, the A's should have won 89 games and they were slightly lucky (about 1 in 4) to win 94 games.</p>
<p>What to take from this? Baseball is a series of coin flips- some of which are in your favor and some against. Even after 162 games we're left with a lot of noise, hence many decisions are based on point estimates(or expect values) rather than the variation of those values. You also have to understand your assumptions and their impact. For example, the A's were basically 2 different teams before and after the All Star break, maybe we should weight those periods 70/30 in-favor of the post break period, rather than 50/50 if we want to predict next year? Its most important to use statistics to understand the data- not try to predict everything under the sun. Hopefully this adds a little color for some of you here. Just understand that plus/minus 6 games can happen very easily based on bloop singles and bad calls. The other teams in the division's data are probably just as noisy, and nobody knows what each teams true expected winning percentage is, that's why we play the games, else we'd have no need for them. The team with the best record is probably the best team, you just can't prove it.</p>
<p> </p>
<p>-Systematic</p>