And then there was one. As the Week Five QB Score Rankings indicate, after five weeks we now have only one quarterback who has played more than once and been above average (an average QB Score per play is 1.1) every time he played. The last man standing has been maligned by conservatives, liberals, one famous teammate, and more than a few residents of the city of Brotherly Love. Yes, the only quarterback to be above average every week he has played this year is Donovan McNabb. All other quarterbacks have been below average at least once. Not surprisingly, the Overall QB Score Rankings list McNabb as the top quarterback in the NFL in 2006.
Although only one player has consistently posted above average numbers, two signal callers have been below average every game played – Ben Roethlisberger and Andrew Walter. Walter is a rookie on a bad team. So his numbers may not surprise. Roethlisberger, though, is the quarterback for the defending champion Pittsburgh Steelers and in 2005 finished second in the NFL in QB Score per play. As the Overall Rankings indicate, now only Walter prevents Roethlisberger from being ranked last in 2006. Once again we see evidence that the numbers signal callers put up can be quite inconsistent. Perhaps that is good news for Pittsburgh fans. It is entirely possible that Roethlisberger will play much better in the weeks to come.
Two more stories from Week Five…
Damon Huard is second best? According to the NFL’s quarterback rating, the top two quarterbacks are David Carr and Damon Huard. QB Score per play ranks these two signal callers 20th and 11th respectively. Last week I detailed the problems with Carr’s performance. Like Carr, Huard has also had three fumbles lost. And since fumbles do impact outcomes, it is likely Huard is over-valued by the NFL’s metric.
Chad Pennington posts the worst mark of the year. Pennington, who was ranked 6th in QB Score per play entering Week Five, posted the lowest QB Score per play of any quarterback in the NFL this year against Jacksonville. Jets fans should note that in 2004 Pennington’s QB Score per play was 2.09. In 2002 his mark was 2.96. As we note in the book, the 2002 effort was the 12th best performance from 1995 to 2005. Of course in 2003 his per play number was only 0.85. So it is possible for Pennington to put up below average numbers, although one would think what he did against the Jaguars is not likely to be seen again soon.
– DJ
QB Score has been discussed previously in the following posts:
Football Outsiders and QB Score
Consistent Inconsistency in Football
The Value of Player Statistics in the NFL
Simple Models of Player Performance
Wages of Wins Journal 
TK
October 11, 2006
I’ll add one longtime gripe, and this isn’t really with the QB Score but rather with NFL stats.
Turnovers, as has been discussed here, are maybe the single most statistically significant part of the game and are given heavy weight in calculating both the QB rating and the QB Score. But the NFL system for assigning fumbles and interceptions is dreadful.
– Fumbles: If an exchange with a running back isn’t successful, the QB is assigned the fumble, no matter whose fault it was. So even if the RB just drops the ball, the QB Score — poof! — goes down 50 points. (See Favre’s handoff to Morency as an example of a perfect handoff dropped.)
– Interceptions: Same issue. Sometimes a QB puts a ball right on his receiver’s numbers. But it pops up, gets picked off, and the QB gets the blame.
[I’m not talking here about totally subjective calls, like interceptions where the QB and the receiver miscommunicate. The receiver sees a blitz and cuts off his route, while the QB gets adequate protection and makes the longer throw. That’s going to get put on the QB, and I can live with it.]
Solutions:
– Why not assign the blame where it actually goes? At least errors in baseball are assigned to the right people. If a shortstop throws wildly in the dirt and the ball squirts away, he gets the error. If he throws perfectly and the first baseman drops it, 1B gets the error.
– Or you could go the sack route, and assign full and half turnovers. RB drops the handoff? He gets the full credit for the fumble. Receiver delivers the pick to the DB? He gets the full interception. The throw is high, catchable, but tipped to the free safety? They each get a half.
It would better reflect reality, and the stats could then be used to better reflect performance. The current system just exacerbates already volatile turnover stats by exaggerating the QB’s culpability. No wonder year-to-year performance varies so much…
dberri
October 11, 2006
TK,
I agree with your point. The NFL should do a better job of assigning responsibility. I would add that it is often very difficult for us to know on a given play what went wrong. The coaches know what was supposed to happen, after all they designed and called the play. So they know if the outcome was due to the receiver running a poor route or the quarterback making a bad read and/or throw. Without this insider knowledge it would be very difficult for a scorer to know who is truly responsible. And coaches probably do not want to share this information, if for no other reason than blaming individual players in the media is probably not the best way to promote team unity.
In the end, I think we are just going to be frustrated by the quality of data we get from the NFL. This is why I repeat over and over again that these statistics do not necessarily represent the quality of the quaterback.
Dennis Coates
October 17, 2006
It seems to me that some of this uncertainty or shared blame washes out, at least as regards passing. In other words, assigning blame for tipped passes or drops strikes me as of second order importance in evaluating quarterbacks. Tipped balls are sometimes intercepted, sometimes fall incomplete, and sometimes, like in the Ravens-Panthers game on Sunday, caught and turned into touchdowns. Receivers make great catches on poorly thrown balls, and drop balls that hit them in the numbers. These events all seem likely to balance out over time.
With fumbles, it is hard to see how these would balance out. As relates to handoffs, if every botched handoff is recorded as a quarterback fumble, then the system has no way to allow random events to balance out. However, it seems that approach will tend to make quarterbacks on teams with lots of different running backs appear worse than they really are. With more backs taking handoffs, there is greater likelihood of unfamiliarity with each other and with the plays, and more likelihood of fumbled handoffs. This is a simple thing to control for in any quarterback rating system.
Likewise, fumbles in the pocket always count against the quarterback, but there is no chance of their effects balancing out as there is with tipped passes. Even if the offense recovers the ball, it still counts as a quarterback fumble. In fact, even if the fumble squirts out and is advanced by the offense for a big play, it still goes in the books as a quarterback fumble. There is no random event that balances a positive against the negative of a fumble while setting to pass.
There may be a bias inherent in this accounting too. For instance, left tackles are often argued to be the best pass protectors on the offensive line because they protect right-handed quarterbacks on the blind side. If right tackles are, therefore, routinely less proficient pass blockers, then left handed quarterbacks may get blind-sided, and therefore fumble in the pocket, more often than right handed quarterbacks. Do the data say anything about propensity to fumble among quarterbacks by right versus left-handedness? If there is a disparity, then one can address this in ratings as well.
One last issue that occurs to me concerns quarterbacks stumbling coming out from under center. Here in Baltimore we have seen over the last several years what must be some sort of record for the number of times the quarterback gets tripped by the center or a guard as he steps back after the snap. Some of these get recorded as sacks, I think, and others as rushing attempts. How do these figures affect QB ratings? Again, there is no countervailing positive random event to offset the negative event.
dberri
October 17, 2006
Dennis,
You bring up a great point about offensive tackles. Michael Lewis has a new book called “Blind Side” where he talks about the increasing value of left tackles. If the market for left tackles is correct, then left-handed quarterbacks should systematically perform worse over time. As I read the latest from Lewis — which is a wonderful book — this research project immediately comes to mind.