SB Nation User Blog: cyberwulf

Clustering the college QBs

cyberwulf — Tue, 31 Mar 2009 16:41:19 -0000

Predicting QB success in the NFL is a very challenging problem: among other difficulties, it's hard to define what "success" is, and properly account for the fact that earlier picks are given many more opportunities than lower picks.

So I decided to look at the problem from a different angle, and instead ask the question:

What current NFL QBs are Matt Stafford, Mark Sanchez, and Josh Freeman most similar to, based on their college stats?

One way to try to answer this question is to apply a clustering method to the data. Here's what I did:

1. Get the final-year college stats (Completion %, Yards, Yards/Att, Int, TD, Rating, Attempts/Game, Yards/Game) from QBs drafted in 2005-2007, as well as from Stafford, Sanchez, and Freeman

2. Apply the k-means clustering algorithm to the normalized statistics. Basically, the algorithm figures out the groupings which yield the smallest within-group variances.

Here are the results. The algorithm requires that you pre-specify a number of clusters; I chose six. Reported for each group are its members as well as the average college statistics for that group:

"Group 1 : Brady Quinn, Jason White, Matt Leinart, Matthew Stafford, Mark Sanchez"

Pct. Yards Yards.Att Int TD Rating Att.G

65.2 3555.2 8.8 9.0 31.0 159.2 31.5

Yards.G

281.5

"Group 2 : Andrew Walter, Jay Cutler, Kyle Orton, Omar Jacobs"
     Pct.     Yards Yards.Att       Int        TD    Rating     Att.G
     59.5    2988.7       7.4       7.9      28.2     138.2      38.0
Yards.G
    287.5

"Group 3 : Dan Orlovsky, Derek Anderson, John Beck, Jordan Palmer, Kevin Kolb"
     Pct.     Yards Yards.Att       Int        TD    Rating     Att.G
     60.5    3511.1       7.4      17.0      26.0     133.5      38.6
Yards.G
    292.0

"Group 4 : Aaron Rodgers, Alex Smith, Jason Campbell, Troy Smith, Vince Young"
     Pct.     Yards Yards.Att       Int        TD    Rating     Att.G
     65.8    2588.1       9.3       7.0      24.0     164.5      24.8
Yards.G
    213.3

"Group 5 : Brodie Croyle, D.J. Shockley, David Greene, Isaiah Stanback, JaMarcus Russell, Reggie McNeal, Trent Edwards"
     Pct.     Yards Yards.Att       Int        TD    Rating     Att.G
     58.6    2356.4       7.7       7.0      16.0     136.5      25.6
Yards.G
    204.1

"Group 6 : Brad Smith, Bruce Gradkowski, Charlie Frye, Charlie Whitehurst, Jeff Rowe, Kellen Clemens, Josh Freeman"

Pct. Yards Yards.Att Int TD Rating Att.G

62.0 2546.1 7.3 9.0 19.3 133.5 31.7

Yards.G

223.9

Observations:

- Stafford and Sanchez profile similarly, and closely resemble (in terms of college stats) Quinn and Leinart. Good completion %, lots of yards, TDs, etc.

- David Greene = JaMarcus Russell? Uh, OK. But the rest of that grouping seems to make sense.

- Group 4 is interesting. High completion %, but relatively low yards and attempts per game. Could call these guys the "dinkers". Interesting to see Aaron Rodgers in with a couple of highly-regarded busts and potential busts-to-be.

- Gradkowski. Frye. Clemens. Freeman. Yikes.

Caveats:

- The groupings aren't totally stable, since the algorithm isn't guaranteed to find the optimal solution; if I ran things again, the groupings might change a bit, but not dramatically. Same goes if you change the number of groupings; a few names might change groups, but the overall structure would be similar. For example, for all the settings I tried, Sanchez and Stafford ended up being grouped together.

Are small-school players over/undervalued in the draft?

cyberwulf — Sat, 21 Feb 2009 03:08:28 -0000

Based on some commentary in this thread, I became interested in exploring whether small-school players have tended to be over- or under-rated on draft day. Basically, the question is:

Do players coming from non-BCS schools perform better/worse in the NFL than comparable big-school (BCS) players?

LantermanC was kind enough to provide me with his awesome Excel spreadsheet containing every player drafted from 1984 through 2007, along with position played, games started, Pro Bowls, and a plethora of other information. We'll be looking at two outcome variables, compiled by him:

Success = 1 pro bowl or 3 starting seasons, or 64 games played.

Wild Success = 2 pro bowls, 7 starting seasons, or 125 games played.

I restricted my analyses to the years 1993 (when the draft went to the current 7-round format) to 2005 (though data is available through 2007, recently drafted players haven't generally accumulated enough stats to be classified as successful). Players were categorized as being "small-school" or "large-school" according to whether they played for a BCS or non-BCS college. Undrafted players were excluded.

The knee-jerk reaction to answering the initial question is to compare the Success and Wild Success percentages of BCS and non-BCS players. But non-BCS players tend to be drafted in later rounds:

Average pick, BCS players:116

Average pick, non-BCS players:147

It's not a fair comparison. So, what we'd like to do is to somehow adjust for draft position. A more refined version of our initial question is then:

Given a BCS and non-BCS player selected at the same spot in the draft, which has performed better in the NFL?

Fortunately, we can answer this question using a basic logistic regression. Without going into too much detail, we model the probability of Success (or Wild Success) as a function of BCS/non-BCS and draft position. This allows us to compare the success probability of BCS and non-BCS players given that they were selected at the same place in the draft.

Running the analyses, we get that:

1. Adjusting for draft position, players from non-BCS schools are roughly 4% less likely (95% confidence interval: 20% less likely, 15% more likely) to have Success than BCS college players

2. Adjusting for draft position players from non-BCS schools are roughly 1% more likely (95% confidence interval: 20% less likely, 29% more likely) to have Wild Success than BCS college players

I re-ran the analyses, adjusting for position played; the results are basically identical.

So, bottom line:

There is little evidence to suggest that non-BCS players exceed or fall short of their expected performance based on where they were drafted.

In other words, NFL GM's do not appear to have systematically downgraded (or upgraded) players based on whether they attended a BCS or non-BCS college.

Final notes:

- If small-school players are systematically denied playing time (due to their lack of pedigree) when they have reached the NFL level, then this analysis could be misleading. In fact, if this is the case, you can't even answer the initial question.

- This analysis refers to overall trends in the NFL; it doesn't claim to be able to predict a specific GM's behavior in the future. However, it does seem to indicate that there is no substantial "market inefficiency" that can be easily exploited.

Interested to hear what people think. Thanks again to LantermanC for the fantastic dataset.

Top 10 pick "bust rate": QB vs. Tackle

cyberwulf — Wed, 10 Dec 2008 16:32:32 -0000

Given the vigorous debate about this post, I went to NFL.com's Draft History site and pulled the list of QBs and Tackles who were selected in the Top 10 picks over the past 10 years. Here's what I found:

QBs

Ryan, Russell, V. Young, Alex Smith, Manning, Rivers, Palmer, Leftwich, Carr, Harrington, Vick, Couch, McNabb, Akili Smith, Manning, Leaf

Out of the QBs, how many can you qualify as busts? Clearly Akili Smith, Leaf, Couch, Carr. Arguably Harrington, Alex Smith. Jury's still out on Young, Russell, but not looking so good. Jury's in on Vick. Overall, out of 16 QBs drafted in the Top 10 (most in the Top 5, actually), somewhere between 4 and 8 qualify as busts, for a "bust rate" of 25-50%.

Tackles

Long, Thomas, Brown, Ferguson, Gallery, Gross, Williams, McKinnie, L. Jones, Samuels (1997: Pace and W. Jones)

Some of these guys aren't household names, so I looked at games started. Mike Williams (#4, 2002) started all games for the Bills for three years, then dropped off the face of the earth. Probably a bust. Robert Gallery? Well, he *has* started all but one game for the Raiders since he was drafted, but it *is* the Raiders. Maybe a bust. Jury's still out on Long, maybe Brown. So, out of 10 tackles drafted in the Top 10, we have one probable bust, with perhaps one or two other possibles. The "bust rate" 10-30% (or about 7-25% if you add in '97)

It might be interesting to look at this for other positions as well; but this preliminary analysis seems to suggest that Top-10 tackles have a higher chance of being regular contributors than Top-10 QBs.

First FanPost; I'd be interested in hearing people's thoughts.