College Football Playoff committee is BCS-esque at avoiding helpful stats

On average, up to three College Football Playoff teams will be painfully obvious. I wrote as much a year ago, and it's been worth repeating, mantra-style, almost every day since then.

To maintain faith in the Playoff selection committee, you might occasionally need to remind yourself that its job is relatively easy. Because every time we learn more about the process, the more we learn about the flaws of the process.

Fox Sports' Stewart Mandel told us in September that the committee would not be using any sort of strength-of-schedule variable. ("We could build a fancy algorithm, but kind of how the Supreme Court said you know pornography when you see it, you just know a hard schedule when you see it.")

And today, via CBS' Jerry Palm, who took part in last week's media mock selection process, we learned more about that and margin of victory, once eliminated from BCS formulas because of the unsporting nature of huge wins, not being part of the Playoff discussion.

Of course, you can see the schedule itself. The opponent, the opponent's record and the result of each game are listed. They also separate out wins against teams with records above .500 and losses to those below .500. Other than that, all you get is the old BCS version of each team's collective opponents' record, and opponents' opponents' record. There are no other ratings.

[C]ommittee members get a litany of stats about each team that not only fail to help determine strength of schedule, they don't even help determine a team's ability to win games, and that's important too.

Instead, team sheets have data from 26 statistical categories, about 2/3 of which are yardage based. The others are points based, and there are a couple that combine points and yards but one can't use any of that to determine strength of schedule. [...]

[A]n even more meaningful stat isn't even allowed in the room. Margin of victory. Nothing correlates as well to winning percentage as MOV, but that isn't politically correct.

When we were discussing various data points in our mock meeting last week, a couple members brought up stats from other sources, and real committee chair Jeff Long stopped us from using them. He said since the formulas for stats that were brought up were unknown, they would not be allowed to use them as discussion points in the room.

A few quick points here:

1. "They also separate out wins against teams with records above .500 and losses to those below .500." Your eye is going to naturally gravitate toward whatever information you are given when attempting to differentiate between Team A and Team B. Because it is available, the committee will look at the "above .500" and "below .500" records.

And they'll be using a number that values a win over a 7-5 team (by one point or by 50) the same as a win over a 12-1 team (by one or by 50). If you lose to a team that went 5-7 (by one or by 50), that's bad. If you lose to a team that went 7-5 (by one or by 50), that might be OK.

This might be even worse than simply providing a team's record.

2. "Instead, team sheets have data from 26 statistical categories, about 2/3 of which are yardage based." Palm confirmed to me on Twitter that some of these measures are at least per-play measures. That's good, as yards-per-play is the most descriptive of any primary statistic.

But the fact that any raw per-game stats are going to be used is depressing. It's 2014. We are in the era of pace. Baylor games produce completely different yardage and point expectations than Florida State or Stanford games. Per-game stats have become irrelevant.

I heard from another mock committee member that the tools given to them are useful and impressive when trying to compare one team to another. That's good. But that only matters so much, if you're looking at the wrong data.

3. "Nothing correlates as well to winning percentage as MOV, but that isn't politically correct." Football is a man's man's game. It's about grit and fortitude, leadership and sacrifice. It's a battle. It's a war. No sensitive souls need apply.

But my goodness, if you don't call off the dogs in the fourth quarter, somebody might get their feelings hurt! And we can't take that chance!

4. "He said since the formulas for stats that were brought up were unknown, they would not be allowed to use them as discussion points in the room." Palm is a well-known figure in the stats world, in part because of his creation of the Ratings Percentage Index (RPI), which was basically the at-a-glance measure of choice for college basketball for nearly two decades. It is as transparent as transparent gets, comparing your win percentage to your opponent's win percentage to your opponent's opponents' win percentage.

As one would expect, Palm is pretty big on transparency. But his measure is only an at-a-glance résumé measure, one that isn't intended to be evaluative of actual play or predictive of future play. And those types of formulas don't work very well with 12-game seasons. You might have to dial into the per-play or per-possession data, like Ken Pomeroy does for college basketball. And when you do that, things get messy -- particularly the opponent-adjustment part -- and sharing formulas loses its purpose.

That's the bad news. The good news: no matter what goes into a given measure, you can judge a formula based on its performance. Jeff Sagarin's numbers go back to 1998. Ours at Football Outsiders go back to 2005. Along with sharing the components I use, I personally try to list the components that go into my ratings, and I share weekly/yearly results at Football Study Hall so you can see how they're measuring up against the actual schedule. (They're pretty good for gauging win probability and creating a hierarchy. They're not great if you're looking to break Vegas.) There are countless other formulas out there with varying complexity and all sorts of variables. Pick some out. Give them a test drive. Grill the creator of the measure for more information. Find the one that suits your needs.

Any number of them will be more useful than saying in December that Florida State is 7-0 against teams above .500.

Never mind given measures, though. With fancy tools and all sorts of gravitas in the room, the committee is being asked to actively avoid information that would be helpful to the cause of picking four playoff teams. In 2014, with infinite information available at fingertips, that is depressing.

On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious. On average, up to three Playoff teams will be painfully obvious ...