SB Nation User Blog: Bleed Blue 'n White

In Defense of Statistics: In Defense of Polls

Bleed Blue 'n White — Mon, 02 Nov 2009 19:40:01 -0000

It's been a long while since I posted an In Defense of Statistics post. The idea behind these was to explain exactly how the statistical rankings work. I know, crazy people (including the BCS) call them computer rankings, but the computers aren't evaluating games. For the most part, the computers are just solving a system of equations that simple statistics ("the better team is likely to win") sets up.

This one is a little backwards. I'm defending the human polls. That's right. The horribly biased, terribly flawed, human polls. You know them, you hate them, right? I mean, there's that crazy love for the SEC and USC, there's the weird preseason bias where if you're highly ranked at the beginning, you've got an easier path, and then there's just the all-around stupidity that comes from the Harris poll. It's nuts, right?

Strangely, no. Believe it or not: the polls are surprisingly good, and you can prove it. With math.

Yes, People Actually Study This Stuff

There's actually been a fair amount of research into the AP/AFCA (Coaches') polls, mainly because it's a good example of human decision-making in aggregate - the "wisdom of crowds" - and this is, of course, interesting to people in economics, where failures in the "wisdom of crowds" costs actual money.

AP Voters Do Overreact

A lot of the most interesting stuff just came out this year. D.F. Stone, a professor at Oregon State, did research into AP voters' decision making methodologies, and found that voters tend to base their decision primarily on the scoring margin of the game, but they tend to overreact. The "money quote," in my opinion, is this:

I find that the voters do not respond sufficiently to relatively subtle aspects of game results (the
signals), namely, home status and margin of victory over unranked opponents. The voters respond
in a more Bayesian way to margin of victory over ranked opponents and margin of loss against all
opponents.

This, to me, is interesting, because I think this is how almost all of us would like teams to be ranked. We don't care how you pummel creampuffs. We care how bad you lose to a creampuff, and how good your victories over good teams are. This may not be the most predictive result, but ranking teams isn't about predicting performance, it's about evaluating a team's season, and I don't think anyone would really argue with that methodology.

Stone also found evidence that voters tend to overreact immediately after a game - this is almost certainly where the "it's better to lose early" belief, which seems to be real, comes from. That overreaction is common in human decision making. His conclusions appear to support the hypothesis that in general, the polls work well, but the problem is that the AP voters are overtaxed - ranking accurately takes too much work, so they take the easy route out.

The question, though, is how bad is the problem? If the 'easy route' works almost perfectly, is it really that bad? When you consider that the BCS uses a merger of human polls and statistical rankings, maybe the "easy route" is all you really need.

... but there's no preseason bias, and Penn State readers are about to kill me

The other interesting paper that came out recently was a paper by Ross, Larson, and Wall, which looked for conference biases in the AP/AFCA poll, as well as a possible preseason bias. They looked at the AP/AFCA poll from 2003-2008, so the sample size is pretty large.

The results may shock you. First off, again, "the polls are pretty good." The higher-ranked opponent wins ~80% of the time. But there are a few biases, and they're not what you would expect.

The first was presented elsewhere: Campbell, Rogers, and Finney had shown that there's a bias in the AP/AFCA polls towards television coverage. This isn't surprising - when you can see the game, you have a stronger reaction towards the victor and against the loser because you understand the game more. That bias gets confirmed via a proxy of population size, although the real source is almost certainly television coverage.

The other interesting thing is this: there's no weekly effect. That is, the poll was roughly equally accurate at all points in the season. Teams which are ranked high early don't tend to linger and be overrated throughout the season - there is no preseason bias. This agrees with Stone's findings: the polls overreact, and so the preseason rankings don't have a real effect. Also note that this semi-contradicts a finding elsewhere, but the study there was fairly simplistic, and even that author noted that the effect was very small. If preseason rankings are not entirely inaccurate, adding them to a model to predict a final ranking will add information that might not be present. This does not indicate a preseason bias - it indicates that voters aren't stupid. You could likely get a similar result by adding a ranking midway through the season.

As for conferences: the conference that the poll was most biased for? The MAC. Yes, the MAC. I'm not kidding. When polls rate a MAC team, they tend to be overrated. This is probably not really a bias for the MAC. The MAC's had only two teams that ended the year ranked, back in 2003, and the lower-ranked team actually upset the higher ranked team. Other than that, the only ranked teams have proved to be unworthy of that ranking. The authors acknowledge this - it's akin to the "long shot bias" in betting, where heavy favorites win more than you would expect.

But there was a major conference with a statistically-significant bias, and you wouldn't expect sample size to be an issue here: AP/AFCA polls were biased against the SEC, at about a 5-10% likelihood level.

I know, I know. I'm dead to you now. But don't kill the messenger - this is just statistics here. And the size of the bias isn't that large. To be honest, I can come up with an explanation that might further get me killed - quite a few SEC teams have been good over that period, and rankings are ordinal - that is, you can't tie teams. Since there are fewer "top" spots, that'll produce a downward bias in the rankings.

It should also be noted that this might be cyclical - that is, the SEC was underrated early this decade, and now it may be being overrated because it had been underrated for a while. Might the Big Ten experience the same thing when its big-money teams finally stop being mediocre (cough Michigan cough)? Quite possibly.

Can we make them better?

Now it's the opinion part. Clearly, the polls aren't that bad. There's no preseason bias, there's negligible conference bias, and in general voters seem to be prioritizing what we'd like them to do.

Can we do better? I can't say for certain. I can say that the authors of the research pointed out important points - first, the AP Poll acted as you would expect if the voters were overworked. Makes sense - they have other jobs. So it might make sense to create a panel of voters who have the time - that is, they're paid for it, who can see any of the games they wish (free ESPN GamePlan!), and whose performance is periodically monitored to avoid bias or poor voting. It might make sense to do the same for coaches, too - instead of using active coaches, use, say, retired coaches.

Will this work? I have no idea. It might not be feasible. It might be more inaccurate than what we currently have. And finally, it might not be worth it, because as noted above, surprisingly, things aren't that bad.

Down 14, 3 minutes to go...

Bleed Blue 'n White — Fri, 02 Jan 2009 01:56:35 -0000

Yeah, OK, we lost. But I'm not going to complain about any of the players, or the playcalling, or the interception. I don't care about that - what happened, happened. It's a game.

What I am going to complain about is the announcing. Down 14, with 3 minutes to go, with the ball, Brent Musberger makes a comment that "they're trying to make it a one score game," and half chuckles at it. Kirk Herbstreit says something like "they're playing for pride."

No, idiot. Down 2 scores, with the ball, with 3 minutes to go, you're still playing to win. If they hit a quick TD, it's a one score game and USC's unable to kneel out the clock, so you still might not even have to kick an onside kick. Yeah, it's Hail Mary desperation time, but who knows - maybe USC's long snapper fires it over the punter's head just like he did again.

Later, Herbstreit says something like "USC's gameplan changed..." and he's right. It did. They went from trying to continue to score to trying to run out the clock. That's how you win football games - you throw to gain a lead early, and run to finish the game late. And USC couldn't do that against Penn State. That's dangerous. You can lose games like that - even ones where you're up big. USC didn't "dominate" the game. If they "dominated" the game, they would've been able to gain a single first down on the last few drives so that they could kneel out the clock.

USC knew that they were letting Penn State have life. Sanchez was pissed on the sidelines, trying to get the defense to focus and the skill players to stay in bounds. But instead of the announcers noticing any of this, we got to listen to the announcers blather on about nothing.

Ranking the BCS games(' gift packages)

Bleed Blue 'n White — Thu, 11 Dec 2008 20:14:18 -0000

It's bowl time, and that means one thing. "$500" worth of bling for the players! Whoo

Unfortunately, not all bowl gift packages are created equal. Some are downright lame, so let's take a look (you can see a full list here). It's important to realize that that $500, for a lot of bowls, is totally made up, and the content can be worth a lot more. For some, however, it's way less than that.

This year's BCS bowls are actually pretty amazingly poor, with one exception.

Note: retail values are "intelligently guessed." Yes, of course you can get stuff cheaper than retail, but I'm using pretty similar standards for all of them - find an MSRP, if that's not available, take the high end of end-user sales prices. Models with ranges are mid-to-low range.

FedEx Orange Bowl/BCS Championship Game

$300 of Sony electronics (recipients choose from available electronics)
Tourneau watch (retail value ~$200)
New Era 59fifty hat (retail ~$35)
Ogio duffel bag (retail ~$30)

Why is this the best of the BCS bowls? Easy - selection! Already got an MP3 player? Go for the DVD camcorder. Got one of those? Go for the PS3. You can't beat selection, and even the "secondary" stuff is pretty decent - a watch, a trendy hat, and a duffel bag to carry your souvenirs home in. Quality. Bonus points for a likely retail value over $500, fudged by creative retail values.

There's a downside, in that they don't get the gifts right away, but that's also a bit of a benefit - nothing to carry back.

(note that both the championship game and Orange Bowl get the same gift packages)

Tostitos Fiesta Bowl

Myvu Personal Media Viewer (retail ~$200)
Apple iPod Nano (retail $150)
Kenneth Cole watch (retail ~$100)
"Hat"

Sigh. Second place for this? Well, OK, the Myvu thing is a bit goofy - but "iPod Nano plus Myvu equals I can watch whatever I want on the return home." So it's got definite "usefulness in the near future" factor.

In 2005, the Sugar Bowl (I believe) gave players a mountain bike. I called this one of the worst swag packages ever - how ridiculously awkward is it to get a bike home? And it's the middle of the winter - you won't use it for what, another 3-4 months, at which point you've probably forgotten it.

So definite props for giving something that the kids can use quickly. But a "hat?" And no bag to carry souvenirs in? Lame.

Allstate Sugar Bowl

Sony MP3 Walkman (retail ~$75)
Sony Blu-ray Player (retail ~$250)
Timely Watch Co. watch (retail ~$75)
New Era cap (retail ~$20)

You might ask "this doesn't seem so great." It certainly doesn't to me. Two pieces of electronics: cool. Giving away a non-Apple MP3 player: not cool. Seriously, these are college kids. They have iPods, and if they don't, they're not going to be thrilled with a non-iPod MP3 player.

The Sony Blu-ray player is cool, but not that cool - again, college kids. A Blu-ray player that's not the PS3 is, well, a bit of a waste.

The watch is a fair step down, and they don't even give you a bag to carry things in. But still, two pieces of electronics - there's a decent chance they'll like something here.

Rose Bowl

Sony DVD Camcorder (retail $250)
Fossil watch (retail ~$75)
Oakley sunglasses (retail ~$75)
Ogio backpack (retail ~$30)
New Era 59fifty hat (retail ~$35)
1 year subscription to ESPN the Magazine (zero with just about any trial offer!)

Geh! Where to start on this? First, a DVD Camcorder? Seriously? College kids! It's not about 'preserving memories for your kids' or something. It's about Youtube. C'mon. The Flip digital mini-camcorder is more hip, and way more common among the bowls.

The ESPN the Magazine subscription just ticks me off. They're athletes. I'd imagine most, if not all, of them already have the subscription, and if not, ESPN pretty much gives it away if you even look at a trial offer.

All in all, it's pretty ordinary. The bulk of the money was probably in the trendy sunglasses and watch No cool electronics, doofy subscription to ESPN the magazine - OK, it's not the "mountain bike" catastrophe, but it's still pretty blah.

Best of the Others

It's really striking how blah the BCS bowls' offerings are when you look at the some of the others:

Alamo Bowl: 80 GB PS3 with game (!), plus Fossil watch
Insight Bowl: 26-inch Vizio LCD HDTV (!!!), plus Bulova watch and hat
Holiday Bowl: 19-inch LCD HDTV/DVD, Tourneau watch, hooded sweatshirt, $45 Best Buy gift card
Champs Sports/Capital One Bowls: $400 in gifts from Best Buy at party, commemorative watch

That's not to say the BCS bowls are the worst - heck no, but some of the other bowls are struggling, so they've cut back on swag. But the BCS bowls are top-of-the-top.

My personal "best of the entire set" is probably the Holiday Bowl. It's a tight contest between them and the Insight Bowl or Alamo Bowl, but you can never have too many flat-screen TVs, and adding a $45 gift card means a bit of variability - you can go out and buy a bunch of DVDs to watch while you're there.

Other opinions?

In Defense of Statistics: Dear Mr. President

Bleed Blue 'n White — Mon, 24 Nov 2008 19:56:58 -0000

(note: it's been a while since I posted one of these - life's been pretty busy. Previous editions are here and here).

Better than a Playoff

I know several BSD readers saw President-elect Obama's statement that he'd like to see a playoff in college football, and I know that the vast majority of you would support it. Especially now, because, y'know, it'd benefit Penn State.

Or would it? Look at the BCS standings: We're #8. The only way we'd get into a playoff right now is in an 8-team playoff. Things are a bit weird in the BCS standings right now: the Harris/Coaches poll have us at #6. Our average statistical ranking is #7, but we're at #8 (barely) because the statistical rankings and the human polls disagree with the relative ranking of USC/Penn State and Texas Tech/Utah.

So would an 8-team playoff do everything we want? Hardly. You've got 3 contenders from the Big 12: Oklahoma, Texas Tech, and Texas. Those three are linked in a circle of death, and between unbeatens, that essentially means that you can move all 3 as high as you want, so long as you move all three. You've got 1 contender from the PAC-10, two contenders from the SEC, one from the Big 10, and then three contenders from outside the BCS. One viable one (Utah) and two very poor ones (Ball State and Boise State) given their schedules. But they're all contenders. And we've essentially ignored the Big East and the ACC champs, because, y'know, they kinda suck. That's 10 contenders if you leave the Big East and ACC champs out entirely.

You can come up with more and more convoluted playoff scenarios - larger ones, ones that seem "more fair" (conference champs plus wild cards) - but when you go to implement them, you'll frequently get a situation where some team has a valid complaint that the system is arbitrary.

The thing is, we don't really want a playoff because we want teams to prove it on the field. We want a playoff because we want the possibility of Oklahoma facing Alabama, or watching Penn State smash Utah. We want more good football games. A playoff, depending on how it's seeded, could give us Georgia Tech vs. Ball State. Be still my heart. More good football games - that'd be better than a playoff.

All About the Connections

So what's really wrong with college football? After all, Division IAA doesn't have these problems. College basketball doesn't have these problems. What is it about Division IA? It's easy: Division IA college football is the worst connected major sport in the US. Basketball has more teams, but more games, and the games are better distributed. In addition, it is impossible to achieve major-league connectivity in Division IA football.

What do I mean by "connected" and "major league connectivity"? Think of it this way: Penn State is first-order connected to Oregon State. We played them, directly. Penn State is second-order connected to USC (twice over, in fact) - we played Oregon State (or Ohio State) who played USC. We're third-order connected to Bowling Green - we played OSU who played Minnesota who played Bowling Green (or Ohio State through Ohio). There are even 9 teams that Penn State is only fourth-order connected to.

I define "major-league connectivity" as "every team is at least second-order connected to every other team." The NFL is constructed to be this way. MLB is this way since interleague play and a crazy number of games. I don't know enough about the NBA, but given the number of games it's unlikely that it wouldn't be.

Why is this important? Easy: because with major-league connectivity, you're only ever talking about the play of each team. We can't say from direct experience on the field that Penn State is better than USC, but we can look at the way that both of them played Ohio State, and the way that both of them played Oregon State, and say "OK, this could be a tossup, but Penn State might have a slight advantage." Here, you're comparing a team's performances based on a common benchmark - how they did against common teams.

How can you tell if Penn State is better than, say, Texas Tech? Penn State is fourth-order connected to Texas Tech: Penn State lost to Iowa who beat Iowa State who lost to Texas A&M who lost to Texas Tech. How the heck am I supposed to figure out anything from that? You can't - not really. You can't really compare teams head-to-head in college football. There's not enough information.

Simple Suggestions, and why they're Not So Simple

So if I'm saying that we need something more than just a playoff, what do we need? Can't we just say "look, you can't keep scheduling cream puffs all the time"? Isn't that what everyone wants? There are sites that come up with an elaborate playoff system that looks completely feasible. Why can't we just do that?

The problem is that Penn State, like all Division IA teams, has to keep scheduling cream puffs. Because we have to have 7 home games. Otherwise we don't make enough money. Just that simple. Because we have to have 7 home games, we have to schedule a team that doesn't care if they get 7 home games. Which is a cream puff.

The Big East has problems because sometimes they need to schedule four out of conference home games. This breaks down how important home games are to Boise State (who's small enough that they can't always schedule 7 home games - but they have to schedule 6). Paterno's said repeatedly that Penn State needs to schedule seven home games for financial reasons. I could include quotes from the LSU athletic director, from Texas Tech's, etc. - pretty much whomever you want. The answer's the same. You need 7 home games at least for the BCS conference level programs. Which means we need cream puffs.

But don't get me wrong - college football isn't doomed. There are ways things could be improved, and if President-elect Obama really has so much free time he can look into college football, this is what I think he should do. #1-2 are pretty much controversy-free and easy. #3-4 are less so.

Don't allow Division IAA games (FCS) to count towards bowl eligibility. At all. There are plenty of cream puffs in Division IA. If you have to play an FCS school for financial reasons, you have to win 7 games. Period. The end.
Don't allow Division IAA games to count at all in the BCS statistical rankings. Not at all. Not wins, not losses. If a team (Michigan!) loses to a IAA team, let the human polls strike the wrath of the heavens upon them. Statistically, Division IAA teams are too poorly connected: you can end up ranking James Madison #24 (Sagarin) or #105 (Wolfe) - yes, that's right. A team that 4-7 Duke beat 31-7 is ranked above LSU. This is why including IAA games is insane.
Try to get rid of situations where interconference play is dominated by two conferences - i.e. the Big Ten playing 13 games (!!) against the MAC, and only 1 game against the ACC. This is far, far easier said than done - but the easiest method is probably to encourage conference-collective bargaining with major networks.
As a return for that, encourage the major cable carriers to include the conference-only channels (i.e. the Big Ten network and friends) in the standard cable tier or a lesser alternative than the full "mega-package" that's usually required.

Like I said, #1 and #2 are straightforward: you have to get rid of any incentive to schedule a IAA team over a IA team. Period.

#3 and #4 are complicated. It's just not easy to force teams to schedule games where they won't make as much money. And that's the driving problem behind scheduling.

So, Mr. President, I really strongly suggest - if you're going to bother mucking around in college football, don't just come in and decree a playoff. A bad playoff can make things far worse than what we have now. What needs to happen in college football is that scheduling between Division IA teams needs to be fixed.

I think it's hard to argue that Penn State played a weak schedule this year. Oregon State is a Top 25 team right now, and playing a Top 25 team in your out of conference schedule is usually considered a strong team. The problem wasn't Penn State's schedule - it was every other team in the Big Ten's schedule (well, that and the fact that those teams couldn't, y'know, win those games).

The Lost Path to Victory

In the second In Defense of Statistics, back when Hell hadn't frozen over and Penn State was undefeated, I put forth a list of things that could happen that would cause Penn State to jump an undefeated Alabama. Obviously, this is all moot since Penn State lost to Iowa, but it turns out it still quite wouldn't've been enough. It would've been very very close, however.

The main point of that discussion, though, was to point out the fact that preseason expectations were really what was driving the viewpoint that the SEC is better than the Big Ten. That's really still there. We still don't know if the SEC is better than the Big Ten or not. Two of their biggest out-of-conference games are next week, versus Georgia Tech and Florida State. If they lose both of those, it's hard to argue that the SEC is better. Winning both is a solid statement that they're a good conference - but it hasn't happened yet.

LSU is really they poster child; for some insane reason, they stuck around in human rankings forever. Statistical rankings knew that LSU wasn't that good. If you look at the week-by-week graph of Colley's ranking for LSU, you'll see that by Week 7, they were basically where they are now. It didn't take people long to realize Auburn sucked. Why did it take so long to realize LSU was really not that good? I have no idea.

For those BlogPoll (or any other voters) who still have LSU on the ballot: you're insane. Iowa would be a better choice. They've at least beaten someone decent.

One step closer to BCS #2...

Bleed Blue 'n White — Mon, 03 Nov 2008 19:25:07 -0000

BCS Standings, Week 1, Penn State distance from #2: 0.0821

BCS Standings, Week 2, Penn State distance from #2: 0.0242

BCS Standings, Week 3, Penn State distance from #2: 0.0086

Notice a pattern yet? One thing to note is that our former opponents have resume-builders next week, believe it or not: Illinois vs. Western Michigan (!!), and Ohio State vs. Northwestern. Iowa's also not a terrible team, and so it's entirely possible that we might jump Texas in a few of the statistical rankings. I don't think it's likely that a Texas Tech victory over Oklahoma State will leap them much higher, since the gulf is pretty large between them and the team directly above them in the rankings (except in Sagarin, where they're on top already, thanks to iterative convergence).

Which means even with Alabama winning, and Texas Tech winning, Penn State could still end up #2 in the BCS. That would make people in the media go "huh?" - but, of course, the BCS isn't supposed to be a poll, but an aggregate of polls. So weird stuff like that can happen.

Yet another moron sportswriter

Bleed Blue 'n White — Fri, 31 Oct 2008 19:55:50 -0000

Some idiot out of Detroit is once again making an idiot of himself by flinging the "Penn State's schedule sucks!" thing around. He's even making things worse for himself by beating the "Coastal Carolina" drum again.

Listen, Bob. Did you even bother to check Penn State's out of conference schedule? I doubt it, considering you only listed Temple and Coastal Carolina. Temple's kindof interesting, of course. Because Temple happens to probably be just about as good as Alabama's best out of conference win, Clemson.

Of course, you didn't bother to list Oregon State. No one ever does anymore, even though Oregon State is a far better team than any of the teams on Alabama's schedule, and certainly mind-blowingly better than every out-of-conference team on Texas Tech's schedule.

And then, of course, he absolutely didn't bother to check the schedule strength on any of the statistical rankings, where he would've found that Penn State's schedule compares pretty favorably to Alabama's, and depending on how the rest of the year goes, could end up being better.

Nah. Every year there has to be some patsy trying to go undefeated due to "weak scheduling," even though we didn't schedule weak, and no one's even talking about the teams who are the paper tigers: Ball State, Utah, and Boise State.

Aren't you glad these guys don't actually contribute to the national championship anymore? Good riddance, AP poll.

Edit: And here's another crazy thought, this time embedded in an otherwise-sane article. The article's talking about the Georgia/Florida game, and noting what their resume will be if they run the table: "Beat a possibly undefeated and likely still wobbly Alabama team for the SEC title and the winner will have two wins over teams like to finish the season in the top-10."

You know what else Georgia and Florida will have? A loss. Florida's will be a loss to Ole Miss, which I cannot, for the life of me, understand how people could excuse. Georgia would at least have some semblance of an argument, but an earlier loss to Alabama should pretty much cancel out a victory over Alabama, leaving them with a resume that'll likely be slightly weaker than Penn State's.

Florida just still makes me go "wha?" When someone says "well, they will have beaten two Top 10 teams!" and then forgets that they weren't able to beat all of the cupcakes on their schedule, that's just... insane. If a victory over a good team is more important than a victory over a bad team, a loss to a bad team has to be more important than a loss to a good team.

In Defense of Statistics: Go Florida State!

Bleed Blue 'n White — Mon, 27 Oct 2008 09:28:44 -0000

The Only Time You're Allowed to Root for Bobby Bowden

Right now, Penn State and Alabama are close enough in the human polls that the statistical polls - which have Alabama a universal #2 and Penn State a universal #3 - are the deciding factor between the two. Switch the statistical ranking portion for the two teams, and Penn State comes out ahead.

People around the country are saying "there's no way that would happen." Don't believe them - they're wrong. The Big Ten is not nearly as weak as people tend to claim, but probably more importantly, the SEC is not nearly as strong as people tend to think. The Big 12 is pretty clearly the premier conference this year, and so I won't even suggest that Penn State could end up ranked over Texas due to weird flukes. Won't happen.

But it is possible for Penn State to end up ranked over Alabama, but it will require a few things to happen - the combination of all of which is probably unlikely, but possible.

Remember one very important thing: statistical polls have no inertia (Billingsley omitted for insanity reasons). Alabama does not have to lose to move down - what has to happen is that their opponents have to lose. So, I mentioned in the first installment of IDS that good teams are pinned most by their best wins. So, how can we imagine Alabama's best wins eroding? Mainly by eroding the SEC as a whole, since Alabama's strength (much like any BCS team) is in the conference's strength. But there are a few teams Alabama's disconnected to.

This is just one possibility. Hopefully you can see the pattern:

Georgia losing to Florida.
Florida State beating Florida.
Vanderbilt beating Kentucky.
Georgia Tech beating Georgia.

Florida State's actually very important, since Florida's the most likely opponent for Alabama. One other thing you might note here: all of these wins are plausible. I'm not suggesting the Citadel beat Florida or something stupid. Georgia Tech is a borderline Top 25 team, FSU is a Top 25 team, Vanderbilt and Kentucky are about the same, and Georgia/Florida are both Top 25 teams.

But in addition to that, we can also boost Penn State's wins and possible future wins:

Oregon State beating Oregon and Cal: this would put them in the Top 25, and suddenly our out-of-conference schedule is much stronger than Alabama's.
Ohio State winning out: Northwestern is still a decent team, and winning out would likely put them in the top 10.
Michigan State winning out (minus the obvious): they won't really move up, but losing to Wisconsin or Purdue would just pull them down to about that level, and that wouldn't help.

The pattern's straightforward: you want Alabama's opponents to lose to teams that Alabama doesn't play, and you want Penn State's opponents to beat teams that Penn State hasn't played.

Colley's rankings have this awesome little script which shows movement of the rankings based on hypotheticals that I wish every ranking system had. Keep in mind that Colley's system will probably tend to show the least movement of any of the rankings for an unbeaten, so moving us above Alabama in Colley's rankings probably moves us above them in all of them.

It's not really possible to get Penn State over Alabama (with an equal number of games won for Alabama/Penn State) with just their limit of 5 inserted games, but the combination of all of the above happening probably should do it - probably sprinkled with LSU losing somewhere else again.

If a few completely bizarre results happened in the SEC - Vandy ending up in the SEC Championship Game or something - that's definitely enough.

But all these results are plausible, and in the end, this isn't gerrymandering or some weird magic formula. Alabama's ranking is built on its opponents. If they go down, so should Alabama.

The Myth of a Weak Schedule

Probably the most common thing that you've heard this week is that Penn State's played a weak schedule compared to those above them. In Texas's case, that's true. Texas is having an insane year, and if they keep winning, you can't make any argument.

But Penn State vs. Alabama? According to most of the statistical ranking systems, Penn State's strength of schedule is just behind Alabama:

Colley: PSU #81, Alabama #70
Sagarin (*): PSU ~70, Alabama ~60
Massey: PSU #70, Alabama #44
Anderson-Hester: PSU #46, Alabama #33

(*: estimated from ELO-CHESS ratings only)

People simply haven't noticed the fact that the SEC is much weaker this year than last (more a statement of how good they were previously), and therefore also haven't noticed Alabama's rapidly weakening schedule. Auburn started off as a preseason #10 - Colley has them currently at #72. Also, Alabama's early big OOC win - Clemson - now looks like garbage, as Clemson, a preseason #9, is ranked #80 by Colley.

And for all the people complaining that Penn State played Coastal Carolina:

Penn State out of conference opponents:

Oregon State, #30 AP, #43/17/27/25 in Colley/Sagarin/Wolfe/Massey
Syracuse, NR AP, #104/112/121/125
Temple, NR AP, #83/87/80/98
Coastal Carolina, NR AP, NA/178/196/217

Alabama out of conference opponents:

Clemson, NR AP #80/74/88/84
Western Kentucky, NR AP #102/123/108/124
Tulane, NR AP #103/129/110/136
Arkansas State, NR AP #90/#106/#95/126.

Yes, we played Coastal Carolina. So what. Everyone plays cupcakes. Some of them are in Division IAA. Some are not. All of them are a functional "bye" week for the teams in question, unless you're Michigan (zing! hey guys, two in a row, you gotta expect some ribbing).

Please join me in telling the national media to go stuff themselves when it comes to scheduling cupcakes. We. Don't. Care. No one thinks that Texas is better than Penn State because we played Coastal Carolina and they get to play Baylor. We ignore those games. Statistical rankings ignore those games. So should the national media's criticism.

Home-field advantage

In the first IDS, I explained the basic way that rating systems work:

Come up with a game-output function, which takes the rating of two teams A and B, and outputs the probability that A will beat B
Find the ratings which maximize the combined probability (given by the game-output function) of all of the results of the season.

If you read Colley's mathematical description, it'll sound a lot different, but it's the same (For the mathematically inclined, it's the difference between a maximum likelihood and a chi-squared approach. The two end up the same so long as the distributions are ~normal-ish, which is why Colley's approach differs the most when there's a bunch of undefeated teams). Colley doesn't include HFA, because it's not easy to do in his approach.

So how do we include home-field advantage in this? Easy. You modify the game-output function and give a boost in probability to the home team, or reduce the probability of the visiting team. All of the rating systems save Colley include HFA, but you may notice a problem: how do you decide how much of a boost to give? Well, the answer is that you do it the same way that you find the ratings - find the value that explains the year's results the best.

The only problem with this is relatively obvious: as far as I know, all of the statistical ranking systems use the same HFA for all teams. That means that going to Penn State, with 110,000 screaming fans, lowers your chances just as much as going to Indiana, with a guy with a hand-puppet. There's no real way to fix this without biasing the results against a team based on historical evidence (which isn't really in the spirit of rating a season), so it's just a limitation.

Which means, again - just like with margin of victory, it's up to the human voters to recognize certain environments as harder to win in than others.

Silly Statistics: The Over-Criticized Offense vs. OSU

Bleed Blue 'n White — Sun, 26 Oct 2008 04:41:45 -0000

"The defenses dominated at OSU." "Penn State's vaunted offense was struggling." "Both teams looked out of sync."

Sound familiar? Too bad they're all wrong. Here's one thing you won't hear: Penn State had a grand total of zero three and outs. Zero. They had only one drive in which they gained under 20 yards. (Ohio State had one three and out.)

If there was one thing I wish football fans would understand, it's this: Points aren't all about offense, and offenses aren't just about scoring points. We held OSU to six points - and a good portion of that was because our offense was keeping them deep in their territory, where our defense had a long field to work with.

Points usually happen when bad things happen on defense: missed tackles, blown coverages, etc. Defenses typically have an advantage in numbers in a football game, and so disciplined defenses will keep any offense from scoring points on long fields, because sooner or later in the drive, the defense will guess right on a play, and force a negative situation, and end the drive.

The other reason for the low-scoring game is simple: Penn State had only 8 possessions. A normal game usually is 12 possessions long. The game was simply short.

A low-scoring game does not mean a bad offense. Bad offenses have turnovers, penalties, and bunches of 3-and-outs. Penn State had none of those. This was simply a short game in which both defenses played extremely disciplined football.

Thanks, ESPN, for a fantastic game!

Bleed Blue 'n White — Sun, 26 Oct 2008 03:16:51 -0000

Major kudos to ESPN for an incredibly well-produced game. I don't care if you're not a Penn State fan - that pictorial history of JoePa was just fantastic.

Also major kudos to the Buckeye alum in the booth. Great job. You never got the sense that he was partisan at all.

And my God, what a game. The Big 12, PAC-10, and hey, this week, even the SEC can keep all the high-scoring goofball shootouts filled with sloppy tackling, poor fundamentals, and dumb penalties. Give me a disciplined game filled with great football, and I'll eat it up. Zero accepted penalties for Penn State, and only one called.

I have to say, I'm a bit biased, as I lived in Columbus for a while, but I love the Penn State-Ohio State games. Hate on the Buckeyes all you want, but they're a disciplined, good team. Penn State-Ohio State 2005 and 2008 were just awesome thrill rides.

I'll be posting Silly Statistics about this game next week, along with another installment of In Defense of Statistics. What a season. What a team. What a coach.

In Defense of Statistics

Bleed Blue 'n White — Mon, 20 Oct 2008 23:31:02 -0000

This is the first in (hopefully) a series of posts to try to explain the way that the statistical rankings work in the BCS.

The first thing I'm going to say is this: Statistical rankings are easy to understand. Very easy. There's no magic about them, and with a few crackpot exceptions (Billingsley) they're all basically doing the same thing.

The basic idea is this: first, assume all teams have an intrinsic "true strength" - one number that determines how likely team A is to beat team B. That "true strength" is how the teams are ranked. Note that this means that we're assuming that matchups can't play a significant role - that is, you can't have a situation where Penn State is more likely to beat Wisconsin, who is more likely to beat Ohio State, who is more likely to beat Penn State. This may seem like a big assumption, but first off, it's necessary, and second, you can do some tests to determine if it's likely that loops like that are real, and they don't seem to exist in football. At least, not significantly.

OK. Now, with that rating, you need a "game output function." You need some function that gives you the likelihood that team A will beat team B. Whatever function is chosen, it's going to look something like this, where the X-axis is something like "A-B", where A is the true strength of team A, etc.

Now, armed with a game-output function and the results of each game, you can form a rating. You do this by figuring out what ratings make the total likelihood of the entire season highest, formed by multiplying everything together.

Since the BCS is out for the first time this week, there are some interesting questions we can ask by looking at the computer rankings:

Why is Ohio State so high? They're ranked 5th, which is universally higher than USC, who beat them. Quite soundly.

Because Ohio State's only lost one game, and that was to a good team. USC lost one game, too, but that was to a much weaker team. Losses to weak teams pin a team much more than losses to good teams. This is where human polls and statistical polls differ: a human sees the pounding USC gave Ohio State and does not see it as a fluke, but sees the USC-Oregon State loss as a fluke. The statistical polls see *both* as an upset, but the USC-Ohio State loss as a much more likely upset.

Why is Penn State so low? In some cases (Wolfe's rankings) they're below Ball State!

Undefeated teams are a kindof "gotcha" for ranking systems. If a team's true strength rating is "infinite", then all of its wins will have a GOF of "1", and that maximizes the total probability. So really, the biggest difference between all of the ranking systems is how they treat undefeated teams. Most of the ranking use something called "maximum likelihood," which is basically what I described above. When they do that, they correct undefeated teams by using something called "Bayesian inference," which starts by assuming a likelihood that a team that good actually exists. You don't get an infinite ranking, because no team in college football is that good. So it's a combination of "how likely is it that they're this good?" with "how likely is it that they would've won all their games if they're this good?"

This ends up being a "floor" rating, typically - which means that frequently, undefeated teams will get ranked lower than defeated teams, simply because the prior distribution (the likelihood of a great team existing) is low.

An important opinion note here: I hate this method. Frankly, it's stupid: because you can't bias yourself and say "Penn State's more likely to have a great team than Ball State," you have to assume the prior distribution is the same for all teams. This is violently, and provably, wrong. Bayesian statistics is just not a good model for separating undefeated teams. The problem is that it probably looks good, on average. But it underrates "major" programs, and overrates "minor" programs, which is what's happening for Ball State.

I can't really justify Ball State's ranking at all, though. The wins we have, straight down the line, are better than all of Ball State's, by Wolfe's rankings (who has Ball State above Penn State). The only advantage Ball State might have is that they've got one more away game (thus converting a "godawful team" into a "might not be so bad" team). But even there that doesn't make a lot of sense, considering the opponent advantage we have is very large.

OK, so why is Penn State so much higher in Colley's rankings?

Because Colley uses a different method of finding the ranking that maximizes the likelihood for each teams. Basically, in the end, Colley's rankings end up putting unbeaten teams midway between the "worst" and "best" rankings. I like this method a lot more, since you don't have a preassumed prior distribution. Also, winning more games helps to pull you higher and higher. With a Bayesian approach, winning doesn't help much unless you win a better game than your "best" game, because the prior distribution is likely to be the one that's pinning you back.

Shouldn't the statistical rankings be allowed to use scoring margin?

No. Very much so, no. Penn State's 46-17 victory over Michigan is a resounding victory in terms of scoring margin, but I'm sure that a lot of people in the human polls looked at that victory and said "hmm, Penn State might have some weaknesses." If not, they should've. I wouldn't blame them. The point is that you play to win the game - period. That's the only fair way to measure how well a team played. A statistical model cannot include all the variables that go into a game, so rather than use a flawed model to get imperfect information, it's best to use a simple model to get perfect information.

Does this suck for Penn State? Yes, and no. Yes, in the sense that the statistical rankings don't help us. But if the human polls worked the way they're supposed to, Penn State should be fine - because the human polls shouldn't get worked up about the fact that PSU has a weak schedule. We beat the crap out of our opponents. That's all they need to know - the statistical rankings deal with the weakness of the opponent.