A few days ago I was watching a panel discussion on quarterbacks on the NFL Network. Or maybe it was at ESPN. Actually I am not sure. In fact, I am not sure who was on the panel. I do remember, though, the discussion.
The discussion focused on the value of Jay Cutler and Aaron Rodgers. And one person on the panel – who said he was a “numbers guy” – argued that Rodgers was clearly the better quarterback. Another member of the panel, though, argued that Cutler didn’t have the same quality of teammates. Therefore, it was not surprising that Cutler didn’t have the same numbers. Consequently, maybe Cutler was the better quarterback.
My response to this debate: I agree.
In other words, I sort of agree with both sides.
People have argued that the numbers in basketball are misleading because of the interactions between teammates. I have looked into this claim (repeatedly) and find that the size of these interactions appear quite small (see the FAQ page for some of this discussion – and see Stumbling on Wins for even more). So when it comes to basketball, we really can use the box score numbers to accurately evaluate individual players.
In football, though, these interactions appear to be much greater. At least, when we look at the numbers we see a great deal of inconsistency. A quarterback’s past performance is a relatively poor predictor (at least, relative to what we see in basketball) of a quarterback’s future performance. Part of this inconsistency is due to injuries (very common in football) and the small number of games football players actually play. But one suspects, part of this is also due to the fact a quarterback’s numbers depend upon the quality of his receivers, offensive line, and running backs. So if you are looking at the numbers of two quarterbacks, you have to consider who the quarterback is playing with (and let’s not forget, who the quaterback gets to play).
And all that means the conclusions we can reach with the numbers from football are going to be somewhat weak. That being said, since we have some numbers, let’s leap to some weak conclusions.
Before we get to the conclusions, let’s look at the numbers. The following table reports – for all quarterbacks who participated in at least 100 plays this past season – each quarterback’s QB rating (the NFL’s metric), total yards (including yards from passing, rushing, and sacks), total plays (pass attempts, rushing attempts, and sacks), all turnovers (interceptions and fumbles lost), Wins Produced (a metric detailed in The Wages of Wins, Stumbling on Wins, and other places), and Wins Produced per 100 plays (WP100).
And what do these numbers indicate? Whether we look at QB Rating or WP100, the top three quarterbacks in 2010 were Tom Brady, Phillip Rivers, and Aaron Rodgers. Although the two metrics agree on the top three quarterbacks, there are some disagreements after this point. Specifically, QB Rating appears to overrate the play of Matthew Stafford, David Garrard, Rex Grossman, Jay Cutler, and Joe Flacco (each of these players is ranked at least 10 spots higher by QB Rating relative to WP100). And QB Rating underrates the play of Troy Smith, Tim Tebow, Mark Sanchez, and Donovan McNabb (each of these players is ranked at least 10 spots higher by WP100 relative to QB Rating).
As one might suspect, I prefer the WP100 rankings (for reasons detailed in The Wages of Wins, Stumbling on Wins, and other places). So what are some of the stories this specific ranking tells?
- Of the 48 quarterbacks who saw at least 100 plays of action, Tim Tebow was a top 10 quarterback in 2010. Yes, Tebow was “good”. Does this mean Tebow will be good in the future? Again, quarterbacks are inconsistent. But his numbers this year were “good”.
- Of the two Smiths employed in San Francisco, Troy was a top 10 quarterback and Alex… well, he wasn’t. So maybe the Niners should have stayed with Troy.
- NFL quarterbacks participated in 19,784 plays in 2010 and produced 90.56 wins. Given these numbers, per 100 plays an NFL quarterback produced 0.458 wins. And that means, Alex Smith – with a WP100 of 0.445 — produced at a rate that was only slightly below what we saw from a typical NFL quarterback. So maybe Alex Smith can help someone in the future (i.e. a team that employes a substantially below average quarterback like the Carolina Panthers).
- Fourteen teams have more than one quarterback listed. For some teams – like the Arizona Cardinals (Derek Anderson and John Skelton), Carolina Panthers (Jimmy Clausen and Matt Moore), Denver Broncos (Tim Tebow and Kyle Orton), Detroit Lions (Shaun Hill and Drew Stanton), and Minnesota Vikings (Joe Webb and Brett Favre) the overall rankings of each quarterback on the same team are virtually the same (within three spots in the rankings). That suggests (but only suggests) that each quarterback’s success or failure was about the players around the quarterback and not the person actually taking the snaps from center.
- For other teams – like the Buffalo Bills (Ryan Fitzpatrick and Trent Edwards), Philadelphia Eagles (Michael Vick and Kevin Kolb), and Washington Redskins (Donovan McNabb and Rex Grossman), the rankings of each quarterback on the same team differ by 20 spots or more. That suggests that the quality of quarterbacks employed by these teams was quite different.
- The best rookie quarterbacks – according to WP100 – were as follows: Tim Tebow (drafted 25th overall), Colt McCoy (drafted 85th overall), Sam Bradford (drafted 1st overall), Joe Webb (drafted 199th overall), John Skelton (drafted 155th overall), and Jimmy Clausen (drafted 48th overall). Obviously Sam Bradford played the most. And as we have seen in the past, those drafted higher got to play more in 2010. But on a per-play basis, those drafted higher did not seem to perform better. Of course, those drafted higher did get paid a great deal more.
- By the way, if we look back on the 2009 draft, the top quarterbacks in 2010 (of those who got to play) were as follows: Josh Freeman (drafted 17th overall), Mark Sanchez (drafted 5th overall), and Matthew Stafford (drafted 1st overall). So would the Lions have been better off drafting Josh Freeman or Mark Sanchez? Certainly each choice would have been cheaper if the Lions managed to trade down to make the selection. Of course, one could argue that a healthy Stafford is still the better choice. Then again, one could also argue that Stafford has yet to be healthy.
Once again, one has to remember that whether you are “numbers guy” or not, the numbers we see in football don’t allow us to reach strong conclusions about the play of individual quarterbacks. One suspects that given a change in teammates, quality of competition, coaching, etc., the numbers we see from any quarterback could go up or down considerably. And that means, we should be hesitant to conclude that any of these quarterbacks are substantially “better” or “worse” than any other quarterback (and therefore, paying one quarterback much more than you pay another – in many cases (although perhaps not all) – is not wise).
Okay, enough about quarterbacks. Let’s talk about the Lions (my team).
Across the last four games of the season – which is the only sample that matters (since games in December, as I think I have heard people say, are where we separate the true contenders from the pretenders) – the Lions were 4-0. And those four victories came against somewhat okay teams (Green Bay Packers, Tampa Bay Buccaneers, Miami Dolphins, and Minnesota Vikings). So the Lions are clearly one of the best teams in the NFL (at least, I think so).
Of course, some might say “why not look at the entire season?” Okay, let’s do this. Pro-Football-Reference.com provides the Simple Rating System, which considers a team’s margin of victory and strength of schedule. The Lions average margin of victory was -0.4 while their strength of schedule was 2.3 (in other words, the Lions played teams that were better than average – since average strength of schedule would be 0.0). If we put these two numbers together we get and SRS of 1.9. And that mark ranks the Lions 7th in the NFC; and also ranks the Lions ahead of the Kansas City Chiefs and Seattle Seahawks (two teams in the playoffs).
So the “Roar has been Restored” in Detroit. Okay, maybe that’s a bit too strong.
Let me close by noting the Lions performance in 2010 – according to SRS and Pro-Football-Reference.com– was the 17th best performance by a Lions team since 1960. And if the Lions can get their SRS to 5.0 next season– a mark exceeded by six teams in 2010 – the 2011 Lions will do something that only one Lions team (in 1995) has done since 1981. Yes, the Lions history is not one of amazing teams (unless we go back before I was born or more importantly, before the Ford family bought this team).
Update: The quarterback numbers (i.e. the stats that comprise yards, plays, and turnovers) were found at Yahoo.com.
– DJ
Wages of Wins Journal 
question
January 11, 2011
Football statistics seem to (finally) be differentiating between different types of interceptions. It’s obnoxious to see a poorly thrown pass that goes right to a defensive back count the same as a ball that deflects off a receiver’s hands and into a defense backs. Do your numbers do the same? When you use a more sophisticated version of some of these numbers, do you see less fluctuation in the data?
tywill33
January 12, 2011
Have you ever found any connection between quarterback performance and coaching? I bring this up because neither Aaron Rodgers nor Matt Flynn were equipped to quarterback Tiny Football League teams when they came to Green Bay and now one is top flight and the other is competent, and I tend to credit Mike McCarthy. Anyone who saw Rodgers in his first couple of preseasons under Mike Sherman will know that he was absolute junk. He looked clueless. Now he sits back in the pocket and throws and runs likes “Jolly” Roger Staubach in his prime (I even saw him do the Staubach “shoulder fake” on a recent run).
Mo
January 12, 2011
It’s obnoxious to see a poorly thrown pass that goes right to a defensive back count the same as a ball that deflects off a receiver’s hands and into a defense backs.
But you need to differentiate between balls that deflect off of a receiver’s hands as well. If the pass is well behind or over the head of the receiver and only substantial action (like a leap or a dive) causes the receiver to be able to reach the ball. A ball 5 feet over the receiver’s head and is deflected to a DB is just as much the QB’s fault as a pass that goes right to the DB.
ilikeflowers
January 12, 2011
Innate QB quality seems to be a particularly tough nut to crack:
[a] The better your receivers – the less accurate the QB needs to be
[b] The better your strategies – the less accurate the QB needs to be (i.e. they are more open)
[c] The more varied your strategies – the less accurate the QB needs to be
[d] The better your offensive line – the more accurate the QB can be
It’d be nice to be able to at least rank these things in order of importance.
Anyone want to crowdsource this? :)
I have: d, a, b, c
welbilt bread machine
January 13, 2011
Wow, Troy Smith a top 10 QB. I hope Jim Harbaugh and Trent Baalke doesn’t read this article and don’t draft a QB. -alfredowelbilt bread machine
Jim Glass
January 17, 2011
Tim Tebow and Troy Smith ahead of Peyton, eh?