clock menu more-arrow no yes mobile

Filed under:

College Football Rankings, Human Bias And The Case For Computers

AP poll voters are biased and weird, in an endearing and ultimately human way. They also hold far too much sway in determining college football's national champions.

On November 22, 2008, nearing the end of what really was a fun season of college football, here are how three Big 12 teams ranked in the Harris Poll:

2. Texas Tech (10-0, 32 first-place votes, 2,737 overall points)
4. Texas (10-1, 2,476 points)
5. Oklahoma (9-1, 2,375 points)

In the BCS standings, Tech was second, Texas third and Oklahoma fifth. Tech's last-second win over the Longhorns had taken them to within two games of the Big 12 South title and within three games of a spot in the national title game. It was not to be, of course. That Saturday night, Oklahoma massacred Tech, 65-21, on national television and predictably made a nice jump. Here's how the poll took shape the next week:

3. Oklahoma (10-1, 2,598 points)
4. Texas (10-1, 2,577 points)

In destroying the team that actually beat Texas, Oklahoma made up 122 points on the Horns in the Harris poll. Texas held onto a narrow lead in the BCS standings, however: they were second, Oklahoma third.

That next week, Oklahoma went to Stillwater to face No. 11 Oklahoma State, a team Texas had beaten just 28-24 in Austin. In an incredibly fun game, the Sooners took out the Cowboys, 61-41. Meanwhile, the Horns hosted 4-7 Texas A&M, a team that had lost to Oklahoma at home by 38 points a few weeks earlier. They dilly-dallied a bit, leading only 7-0 midway through the second quarter, but they caught fire late and won, 49-9. Both the Sooners and Longhorns looked good, but Oklahoma clearly had the better week with their big win in Stillwater.

Here are the Harris Poll rankings that came out a couple of days later:

3. Texas (11-1, one first-place vote, 2,575 points)
4. Oklahoma (11-1, 2,569 points)

Texas gained 27 points and a first-place vote. Why did this happen? Because a) word came out that the computer rankings involved in the BCS standings were probably going to start favoring Oklahoma if they beat Oklahoma State, and b) Texas head coach (and politician-in-another-life) Mack Brown went on every television network that would have him to advocate for his squad. And to at least a small degree, it worked. And I was horrified. But we'll get back to that.

I thought about the developments of late-2008 when I began playing with CBS' wonderfully entertaining, interactive AP voting bias tool this week. You can lose hours of your day both looking at how voters from certain regions perceive certain teams, and you can lose even more hours trying to figure out the narrative behind it. Why is Texas A&M overrated on the east coast? Why is Clemson underrated in their backyard and Top 10 in the Pacific Northwest? Why does New York love Texas so damn much? Why did the West Coast overrate Oregon before the LSU game, and why do they underrate the Ducks now? It is a fun way to both reinforce your own assumptions -- an East Coast bias, a hometown bias, etc. -- and create new ones.

It is also a way to prove the fallibility of the folks who have the biggest impact on who gets to play for (or in some cases, win) the national title.

In theory, the marriage of human polls and computer ratings makes an incredible amount of sense. There are some really good computer ratings systems out there -- I even have one of my own that I like quite a bit -- and I trust them to make evaluations more than I do my own, or anybody else's, lying eyes. Computers look at a team's performance over an entire season, and they don't naturally overreact to late losses over early ones (unless they are programmed to do so). They aren't swayed by external criteria like "Coach A shook my hand and sure is a nice guy" or "Coach B is a total jerk, and his team didn't look very good that one week I watched them." They do a much better job of painting the big picture than we do.

At the same time, however, there is a certain level of context that often gets lost with computer rankings -- injuries, head-to-head results, and so forth. And for that reason, I like that human voters play a role, too. I'm a numbers snob, but I'm not a complete numbers snob. But the way the BCS rankings have unfolded in recent years, humans have had too much on-the-fly control. When they see a ranking they disagree with, they change their own vote to rectify it. When computers produce a result they don't like, they change the formulas or reduce the weight of the computer rankings. And that ruins the entire marriage.

(The same thing happens when a voter makes an unconventional vote, of course -- he or she gets mocked and ridiculed for not thinking like the rest of the pack, and his or her legitimacy as a voter is questioned. It is almost as if polls are a waste of time. If conventional wisdom is going to be so rigorously enforced, we might as well just appoint a rankings committee of six or eight voters.)

To be sure, I was not appalled about Texas' strange, Thanksgiving 2008 poll improvement because I am a closet Oklahoma fan or I thought Texas didn't have a case -- clearly they did. I was dismayed simply because voters were changing their votes because of factors other than what happened on the field that week. Either they heard computers were doing something they disagreed with, or they were swayed by Mack Brown's glad-handing, but in either case, they abused the system. Their job is to rank the 25 best, or most accomplished, teams in the country. Their job is not to right the wrongs of other pieces of the equation.

At this point, you might as well just remove computers from the BCS equation altogether. The formulas themselves have been marginalized to remove margin of victory, which just so happens to be one of the most telling, evaluative pieces of data you can glean from the data most BCS formulas use: points scored and points allowed. Their impact on the overall BCS standings has shrunk as well. And when the computers do make an impact in one way or another, humans just figure out a way to neutralize them regardless. Computer rankings are treated as an impediment instead of a contributor.

At some point down the line, perhaps when the BCS standings are first released this year, I will talk about what I would do to fix the current BCS rankings system. (And no, "PLAYOFF!!!" is not an acceptable "fix," if for no other reason than similar formulas would probably still be used to determine who participates in a supposed playoff.) For now, I will just highlight the fact that human pollsters are weird, biased and entertaining, in the most human way possible.

Then I will remind you how important football polls are. If Clemson goes undefeated, they may be prevented from reaching the national title game by the voters in their own backyard. A 13-1 LSU team might (but probably won't) get screwed over by the fact that voters in the midwest don't respect them quite as much as everybody else.

We are all biased, and while that is okay, we need to be protected from ourselves sometimes.

(Note: AP poll data was originally used in the 2008 example above, but the Harris Poll is what the BCS standings were using in 2008, so the switch was made to Harris Poll data.)

The Morning Tailgate runs weekdays. Check out the archives.