45 reasons to care about the next wave of college football advanced stats

I began tinkering with college football play-by-play data in 2007 because nobody else was.

There was no such thing as the college football data community, no shared knowledge; nearly eight years later, there still isn't, at least to any organized degree. I created a mostly dormant Google group a few years ago, and the CFB Analysis group on Reddit is slow but occasionally produces some interesting conversation. There is no SABR for this sport (SACFR?). CFB-specific data analysts probably cannot fill a two-deep yet.

Each year we pick up stragglers. It's safe to say that, while we are still years behind other sports, more people than ever are paying attention, and even more will be next year. So perhaps we need to set the table.

Below is a mission statement, a list of topics we need to pursue to get the most out of stats in college football. Using this year's Football Study Hall glossary and the 2015 team preview countdown as means of communicating where we've come to date, let's talk about where we could go once the two-deep is fleshed out.

Some of these items are intended to help writing and analysis, some are for coaching, and some are for the health of the sport. There is nothing here about predictions, rating systems, or college-to-pro projections. There is already plenty of work in those arenas.

These are based in part on the language and scope of Keith Woolner's "Hilbert Problems" baseball piece published 15 years ago. Nate Silver drew reference to it in a recent piece for 538. The topics Woolner discussed included much of what sabermetricians have pursued since the turn of the century. They also include this:

The industry of baseball encompasses more than just the action on the field. To be relevant to the sport as it's practiced today, baseball analysis must expand to explicitly consider the economic, social, technological, competitive and governmental contexts in which the game operates. ... Numbers alone are not data, and solving equations is not analysis. ... We should not abandon a line of reasoning for lack of quantification or a failure to find a tidy formula.

Woolner wrote this more than 20 years after Bill James began writing his annual Abstracts and years after dedicated MLB data communities began sprouting up on the web. The man hours that had gone into playing with baseball stats at that point dwarfed what have gone into college football in March 2015.

Some of the themes below are similar to those Woolner referenced for baseball, and others share no similarities whatsoever.

Offense

1. Separating passing effectiveness into quarterback success and receiver success

2. Separating rushing effectiveness into runners and blockers

3. Separating sacks into quarterbacks and blockers

The first item on Woolner's Baseball Prospectus list was separating defense into pitching and fielding. The separation game is a large part of what makes football statistics so challenging. Still, as proven by the work that went into Football Outsiders' Adjusted Line Yards statistic, we can work toward pretty strong generalizations.

Item No. 2 would seem to be satisfied by the existence of the Line Yards stat, but any time an NFL measure is used for college purposes, it would make sense to ensure that the definitions should remain the same for the college game.

4. Evaluating play concepts by efficiency

What is the corner three-pointer of the college game? What concepts and plays are underutilized in college football? As the number of college games charted increases, we can pursue this type of information.

5. Quantifying the value and impact of a quarterback over other players

We agree quarterbacks are the most important players. But how much more important?

6. Using charting data to differentiate between playing styles

Using the combination of efficiency and explosiveness measures, run-pass ratios, and standard/passing downs splits, we can get a pretty good feel for an offense's personality. But charting data can take us further.

7. Examining the value of "stretching the field"

Call this the Brandon Coleman rule. We hear a lot about how important it is to be able to "stretch the field" in the passing game, but what if you're rarely successful at completing those long passes? How successful do you have to be to truly impact the defense?

8. Determining optimal red zone strategies and practice

Finishing drives is one of the more underrated aspects of winning football; what are the best tactical approaches? What do successful teams practice that others are not?

Defense

9. Determining individual credit for defensive success/failure

10. Generalizing the value of each defensive position/unit

Consider these catch-alls, the defensive versions of items 1, 2, and 3 above. We will almost always be forced to make generalizations, but we can make pretty good ones.

11. Further development of defensive personality stats

Defensive alignment, level of aggressiveness, zone vs. man, et cetera. Here is where an increased amount of charting data can come in handy.

12. Quantifying frequency, type, and effectiveness of blitzing

Defensive tactics are often difficult to catch and quantify, but we can easily count pass rushers and determine whether a blitz qualifies as a zone blitz or not.

13. Quantifying the value of positional flexibility

One of today's defensive trends is the hybrid, an athlete big enough to play one position and fast enough to play another. How does having flexibility impact a defense from a numbers standpoint?

Special teams

14. Determining special teams' impact

My old F/+ calculations say special teams has between a 12 and 14 percent impact on a team's quality. But there are deeper ways to figure that out. And if we can figure that out ...

15. Determining optimal special teams practice strategies

... we can begin to determine how much focus coaches should spend. Virginia Tech and Beamerball became synonymous with special teams dominance -- how much effort does it take to play at a high level on special teams, and it is worth it?

16. Exploring optimal kickoff and punt strategies

I'm thinking depth, direction, etc., here.

Strategic decisions

17. Quantifying each coach's impact on winning

What are an overall program's effects on winning and losing, and how much credit should a coach receive for the caliber of recruiting? How much credit should a coach get for developing talent as opposed to simply landing players with more talent? What role do play-calling and tactical decision-making (things that receive a disproportionate amount of praise/scorn from fans) play in team success?

Once we figure out answers to questions like these, we can more properly evaluate coach performance. We can also use this as a jumping-off point for why coaches succeed at one job and fail at another.

18. Optimizing effects of recruiting, development, and deployment

I've long held that success in team sports comes down to the three things listed above. They represent three disparate areas of skill. How much might one matter over another? And what are some best-practice examples of each?

19. Examining effects of home-field advantage and its potential strategic impact

20. Examining effects of luck and its potential strategic impact

Getting a better feel for how to quantify these will allow us to explore how coaches attempt to offset the negative impact of either one.

Tactical decisions

21. Determining proper fourth-down strategy

We can come up with a proper set of percentages and create our own bot, sure. But how does the strength of an offense or opposing defense play into the percentages? That's not something we've explored much.

22. Determining proper third-down strategy

We always view third down as a 1-or-0, success-or-no-success proposition. But once you have worked the ball into your opponent's territory, is there an area where coming within one or two yards of a conversion should be deemed a success? After all, you get four downs, not three, to gain 10 yards.

23. Better understanding patterns in play-calling

24. Developing deeper methods for analyzing play-calling and other decision-making processes

There is serious game theory potential when it comes to analysis stemming from charting data, and I bet taking a hardcore, quant-friendly, pattern-recognition look at play-calling from different coaches would give us some fascinating results.

25. Further exploring upsets and their causes

There was a panel at Sloan this year called "Anatomy of an Upset" (has a nice ring to it, yeah?). The panel missed the mark by focusing on pro examples instead of delving into the college ranks (where the organizers of the panel most comfortably reside); at the pro level, the difference in talent between favorite and underdog is going to be drastically small compared to college sports, and the tactics used to pull off an upset are going to be far less drastic than what you might find in the NCAA.

Still, it is a forever fascinating topic, one we should explore more. It's not like there's suddenly going to be an even playing field in college football, right?

26. Exploring in-game win probability and its potential effect on risk-taking and play-calling

Matt Mills created a fun, semi-interactive win probability system last summer and has plans to take it further. There are other in-game systems like this as well. But as they become better, how can we best use the data these systems produce?

Recruiting

J.J. Watt was hardly the first Wisconsin player to apparently go underrated by recruiting services. Jeff Hanisch, USA Today.

27. Examining causes of inaccuracy in recruiting rankings

Instead of simply acknowledging that recruiting services miss on certain prospects, let's start figuring out what types of players they miss on and why.

28. Determining which positions are overvalued or undervalued based on geography

My example is always how three-star Wisconsin linemen tend to play like four-stars, and we have maps that are more anecdotal than scientific, but going along with No. 27, where and how are we over- or undervaluing pockets of prospects?

29. Stripping apart talent and potential to create more detailed ratings

I explained this one in more detail here.

I would love to see recruiting rankings move in a direction like what you see there: ability and potential. [...] Current recruiting services tend to try to toe the line between the two. They look at both a player's instant-impact potential and the raw athletic potential that could be coaxed out three, four, or five years down the road. If I were designing a new system of evaluating prospects, I would love to try to take my ratings in two different directions. [...]

This would allow us to say that a player like Florida State signee Derwin James maybe has 4.0-star ability and 5.0-star potential while a player like Missouri signee Franklin Agbasimere, a Nigeria transplant who has only played football for a year and a half but has crazy athletic potential (his highlight film is fun and screams "RAWWWW"), maybe has 2.0-star ability and 4.5-star potential.

30. Examining potential ratings value of high school statistics with opponent adjustment

My white whale is figuring out how to make something useful of high school stats. Yes, almost every FBS prospect is going to look great in high school, but the district and state in which you play likely have impacts, yes?

31. Determining optimal playing style based on location

If we know that certain areas of the country produce certain types of prospects, how far should coaches (and the athletic directors who hire them) go in crafting geography-friendly systems?

32. Evaluating the effect of short- and long-term competitiveness on attendance and recruiting success

This title comes from Woolner's piece, but the intent here fits well with recruiting. How does success impact attendance? How does success impact recruiting? How does attendance impact recruiting?

33. Examining most effective recruiting pitches

How deeply do coaches and their staffs evaluate what is working (or not working)?

Player development

34. Assessing developmental strategies for young players

Among the three silos of success -- acquisition, development, and deployment -- we think we have a pretty good feel for the former and latter. We use recruiting rankings to evaluate acquisition, and we use stats and eyeballs to evaluate deployment. But we don't see much of the developmental side of the game, do we?

35. Understanding proper communication techniques from coach to player

This comes up every year at Sloan. It's one thing to determine best practices, strategies, and goals. It's another to communicate them. And if the analysis side is to become more complex, how does that impact the communication process?

36. Maximizing effects of GPS and other emerging technologies

More programs are using GPS and related technology to measure output, effort, etc., in their players. It's still a new field, and there's potential opportunity for competitive advantage for the coaches who use these technologies better.

Health

37. Determining optimal injury prevention practices

Practice and recovery time. Varying intensities. Use of emerging technology. It's one thing to project what a team can do with its first string on the field. It's another to figure out how to keep said first string on the field.

38. Implementing and measuring optimal prevention of head injuries

The popularity and existence of college football in 50 years depends more on this than any other. Over the last 10 years, coaches have become more educated on the topic, but prevention of and proper recovery from head injuries will continue to rule a good percentage of the conversation.

39. Identifying optimal roster design

Which positions deal more with injuries than others? More turnover (besides quarterback)? More shuffling? Are coaches constructing their rosters properly to account for this?

Economics

Conference USA's UAB dropped football for 2015, citing a lack of funds. Chuck Cook, USA Today.

40. Clarifying the win/dollar trade-off preferences for athletics/football programs

41. Clarifying the win/dollar trade-off preferences in recruiting

This is a take-off of one of Woolner's items. As the cash flow for college sports gets larger, those better able to locate inefficiencies, both in their own spending and in others', might be able to derive a competitive advantage. (And if you're a smaller school, it is an outright imperative that you do a good job in this regard.) This becomes an even larger issue as the power schools figure out how to go about better compensating players.

42. Optimizing the competitive ecology of the game

This also comes from Woolner. Here's what he wrote about it:

Some issues are bigger than any single team's problems. The long-term survivability of 'small-market' clubs has made headlines in the past couple of years. One theory is that 'small-market' teams can't hold onto their own farm-developed talent, which supposedly departs through free agency for major media markets like New York and Los Angeles. The current argument is that the Minnesotas of the world can never retain the players produced by their farm system long enough to contend. However, if we went back to making it easier for teams to retain their own players, you risk creating long-term dynasties like the Yankees of the 1950s, which diminishes interest in baseball in the cities without the dynasty. So what's the best way to achieve league-wide competitiveness?

You can replicate that, nearly word for word, for college football.

43. Maximizing the supply of college football opportunities

As more money gets filtered to power conferences, the imbalance between that top tier and everything else diminishes. To a degree, that's not a bad thing. When more noteworthy teams play in more noteworthy games, bigger audiences tend to watch. Alabama-Texas, for example, drew more eyeballs in 2009's BCS Championship than Alabama-Cincinnati would have.

But while there's a general fairness issue that the guilty liberal inside me rages against, there's also a practical issue that comes with favoring the most powerful schools. If power schools = ratings, then limiting the number of powerful schools to those who already have money or a good conference alignment seems silly. Yes, Alabama's a huge draw, but maybe Random Team A could become one and is currently limited from doing so.

Beyond that is a simple numbers issue. When UAB shuttered its football program, that removed 85 scholarships from the FBS pool. If the top tier of the sport goes further in strangling off the money flow for lower levels, it's conceivable that football becomes less of a cost-effective resource, and other schools either fold or move down a level. Every time that happens, the supply of big-time football scholarships diminishes. And at some point, you run the risk of decreasing demand. If there's less of a chance to make money and/or a scholarship from a sport -- one that is all sorts of risky from an injury standpoint -- there might be less reason for you to pursue the sport in the first place.

The money in college football is great, but a specific number of schools see most of it. That might not be in college football's own long-term interests.

Historical data

44. Extending play-by-play data collection

45. Extending per-game data collection

When Woolner wrote his list in 2000, baseball's availability of historical data was immense. It's even better now.

For college football, we have only a complete set of play-by-play data going back to 2005 and full-season box scores dating back to the early-2000s or late-1990s, depending on the school. Sports Reference has done an amazing job of pulling older data together, but there is still so much that needs to be done to make data available to researchers, analysts, writers, coaches, et cetera.

Having the most complete data set possible increases the range of questions we can both ask and answer. And for college football, a sport with so little shared history among programs and regions, more complete data creates more a shared understanding.

This is an ongoing and urgent pursuit.