An intellectual spat recently broke out in the New York Times and spilled over into the blogsphere, and one of the wedges of division is a recently published journal article by a pair of distinguished sport economists.
The work of David Berri, an associate professor of economics at Southern Utah University, and Rob Simmons, a senior lecturer of economics at Lancaster University and the editor of the International Journal of Sport Finance (IJSF), which is published by Fitness Information Technology (FiT), regarding quarterbacks’ NFL draft position and their subsequent performance was pulled into a dispute between journalist Malcolm Gladwell and Harvard psychology professor Steven Pinker. While they went back and forth in print, and later on the web, I was interested more in the analysis of Berri and Simmons’ research than the playground-like spat between Gladwell and Pinker (and later journalist Steven Sailor).
What I’d like to provide in this forum is not another medium to prolong the dispute but rather a chance for the Berri and Simmons research to be further examined. In order to do so, FiT contacted Berri, who has had four articles published in IJSF and who used his blog to defend and clarify his research, to ask him some questions about his and Simmons’ journal article titled “Catching a Draft: On the Process of Selecting Quarterbacks in the National Football League Amateur Draft,” which was published in a recent issue of the Journal of Productivity Analysis.
Q: For those who haven’t read the article, can you give them a brief summary of the research?
Berri: In this paper, Rob and I are examining the NFL draft. The motivation behind this institution is that league parity will be enhanced if the worst teams in the league are given access to the best amateur talent. But can NFL teams identify the best amateur talent? From previous research we had already seen that quarterbacks in the NFL are very inconsistent across time. In other words, identifying the best NFL quarterbacks in the future—even when you have already seen the quarterbacks in the NFL—is difficult.
With respect to the NFL draft the problem is even harder to solve. College football isn’t the same as the NFL. So we shouldn’t be surprised that the drafting of quarterbacks is difficult. Here is what we specifically found:
1. We did find several factors that predict where a quarterback will get drafted. Specifically, we find that taller, faster, and smarter (i.e., better Wonderlic scores) quarterbacks get drafted first.
2. The factors that predict draft performance, though, don’t predict NFL performance.
3. Given this result, we shouldn’t be surprised that where a quarterback is drafted doesn’t predict how well a quarterback will perform in the NFL.
Q: In the conclusion you and Simmons write:
“Our analysis revealed that there was a relationship between aggregate performance and where a player was chosen. But when we looked at per play performance, the relationship between production and draft position was quite weak. In contrast, a much stronger relationship existed between how many plays a quarterback ran and where he was selected. In sum, draft position can get a quarterback on the field. But quarterbacks taken higher do not appear to perform any better.”
I think it’s that last sentence that has gotten a few people a little stirred up. Can you clarify your position?
Berri: I’m not sure there is much to clarify. Per play performance is not related to where a quarterback is taken in the draft. We looked at a host of statistics. And we looked at various points in a quarterback’s career. Draft position doesn’t seem to predict completion percentage, yards per pass attempt, touchdowns per pass attempt, interceptions per pass attempt, the NFL’s quarterback rating, or Wins Produced per 100 plays (or any other per play metric from the Wages of Wins).
Q: In the article you also examine the factors that determine where a quarterback is selected in the draft. The article stated that too much emphasis is placed on NFL Combine measurables like height, 40-yard dash time, and the Wonderlic score, which you claim aren’t proven predictors of future performance. Based on your research, what are one or two things the NFL should focus on more closely, such as, for example, completion percentage, when examining where to select a quarterback in the draft?
Berri: Completion percentage in college is correlated with NFL completion percentage. But we did not find that completion percentage predicts where a quarterback is selected in the NFL draft. Furthermore, the link between completion percentage in college and the NFL—although statistically significant—is not very strong.
Other than completion percentage, there isn’t much that we were able to see that could predict future performance. The point of the article is not that we can do better. The point is that with respect to quarterbacks, the NFL draft doesn’t seem to serve its purpose.
Q: I suspect having your article discussed in the New York Times has brought quite a bit of attention to the research you and Simmons conducted. You have written columns for the New York Times in the past, but have you been involved in any other published research in sport economics/finance that has drawn any similar type of reaction or attention?
Berri: The idea that scoring is over-valued in the NBA has attracted a fair amount of attention. Also, the finding that good-looking quarterbacks get paid more has also been noted in the media.