Why does the quarterback rating measure of the NFL persist?

Posted on October 27, 2007 by

13


The last few weeks I have been just posting the QB Score and RB Score rankings for the week with just a brief story.  Today I will again post the rankings.  But I want to offer a discussion of the NFL’s quarterback rating that will note the limitations of this measure and why – from an economics perspective – this measure does not go away.

Week Seven QB Score Rankings

Overall QB Score Rankings

Overall RB Score Rankings

Dr. Z’s Campaign Against the NFL’s Quarterback Rating

Dr. Z at Sports Illustrated has gone on the warpath.Apparently he really, really, really dislikes the NFL quarterback rating system.  As he notes in this article (and his mailbag for the week), the NFL’s metric is based on how the game was played in the early 1970s, when this metric was put in place.  Obviously the game has changed a bit in the past three decades, so maybe it’s time for a new formula.

Beyond the change in standards overtime, though, is a problem for decision-makers. It turns out – as Dr. Z notes – that quarterbacks, coaches, and general managers confess they do not know how it is calculated.

As Dr. Z notes “…it’s a prehistoric monster that no one understands, an illogical piece of antiquity that influences so much of the game when it shouldn’t. It affects what is written, what is discussed, what becomes the basis, in some cases, of salary structure and bonuses for players and coordinators.

Steve Young, who has the highest career passer rating in history, admits that he’s “not quite sure how the system works.”

Charley Casserly, who as Redskins general manager was quite aware that some clauses were built into contracts that reflected the rating points, says, “No, I couldn’t tell you exactly how they determine the ratings.”

Bill Parcells, whose 11-point dictum to quarterbacks came from years of study of the position, says, “I don’t know how they arrive at their ratings and I don’t care. I don’t pay any attention to them. I have my own system for evaluating quarterbacks.”

The confusion is understandable since this may be the most complicated metric commonly cited in all of sports.  Here is how the calculation of the quarterback rating is described in The Wages of Wins:

First one takes a quarterback’s completion percentage, then subtracts 0.3 from this number and divides by 0.2. You then take yards per attempts subtract 3 and divide by 4. After that, you divide touchdowns per attempt by .05. For interceptions per attempt, you start with .095, subtract from this number interceptions per attempt, and then divided this result by .04. To get the quarterback rating, you add the values created from your first four steps, multiply this sum by 100, and divide the result by 6. Oh, and by the way, the sum from each of your first four steps cannot exceed 2.375 or be less than zero.

When you look over the steps it is pretty easy to see why people have trouble figuring out what this is measure is actually measuring. 

Although it may look like a random collection of steps, the calculation does seek to compare a quarterback’s performance relative to the average player at this position.  Of course, as Dr. Z notes, the average values are from the early 1970s.  So maybe that isn’t the most relevant reference point.

A Better Measure: QB Score

There is of course one other flaw in the NFL’s measure.  It’s only a passer rating.  Quarterbacks also play with their feet, and none of that is included.  The NFL’s measure ignores sacks, yards lost from sacks, rushing attempts, yards gained from rushing, and fumbles.  To see why this might matter, consider Jeff Garcia’s game this past week.

On Sunday Garcia completed 37 of 45 passes for 316 yards without throwing an interception.  His quarterback rating for the week was 110.7, the third highest mark posted by a signal caller in Week Seven. 

But his team lost. Now a quarterback could play well and still see his team lose. There are ten other players on offense and quarterbacks do not play defense (except when they throw interceptions).  But the Buccaneers loss was actually linked to two actions Garcia took.  Twice during the game he fumbled and the Lions recovered.  The NFL’s measure ignores fumbles, so although these fumbles are charged to Garcia, they are not charged when the NFL evaluates his performance. 

When we turn to QB Score, the simple measure introduced in The Wages of Wins, we get a different view of Garcia’s performance.  QB Score takes into account pass attempts, rushing attempts, sacks, passing yards, rushing yards, yards lost from sacks, interceptions, and fumbles.  It is also weights these factors – as the following equation indicates — in terms of how each impacts points scored and points allowed. 

QB Score = All Yards – 3*All Plays – 30*All turnovers

When we look at Garcia’s Week Seven performance from the perspective of QB Score we see a player who ranked 19th out of 30 quarterbacks.  In other words, a performance that was rated 3rd best for the week by the NFL’s measure is actually well below average. 

It’s important to note that the NFL’s measure and QB Score do not always reach such dramatically different conclusions.  Both measures tell us that Tom Brady is having an amazing season.  Both metrics tell us that Drew Brees and Marc Bulger are struggling. 

There are times, though, when the NFL measure gives us the wrong evaluation.  And when you factor in how difficult it is to calculate and understand, one might wonder why this measure is still in use.

Why the NFL’s Measure Persists

Let me try and explain this as an economist. For fans and members of the media an investment has been made in the NFL’s quarterback rating system.  Although many people don’t understand how this metric is calculated, they do understand that a measure of 100 is considered “good” and a measure of 50 is “bad.”  So the basic interpretation is understood.  And the measure provides an answer to the question: Who is the best? 

Although Dr. Z and I can note the flaws in the measure, as long as someone does the calculation (and the NFL’s metric is easy to calculate with a spreadsheet), the quarterback rating serves does allow fans and the media to evaluate quarterbacks. 

When confronted with a new measure like QB Score (or something from Football Outsiders) the fans and the media are presented with a choice.

1. They can invest in acquiring new knowledge.

2. They can conclude what they have is “good enough” and forgo the cost of learning the new measure.

Think of these two choices from the perspective of the media.  Even if members of the media read The Wages of Wins and decided it is the greatest book ever written, they might still be reluctant to switch to QB Score in their broadcasts.  Such a switch would require each broadcaster to explain QB Score during the broadcast of the game. And this lesson would have to be repeated each time a quarterback is evaluated in future games. 

And it’s not only the broadcaster who has to take on this cost.  The print media would have to devote newspaper and magazine space to teaching QB Score. Given a scarce supply of broadcast time and print space, it’s easy to understand why QB Score – a measure that is simpler, more complete, and more accurate — cannot supplant the NFL’s measure.

So Dr. Z, feel free to complain about this measure.  Yell about it.  Scream to the heavens for something better. But I think the simple economics of path dependence (I suppose I should explain this term but I am at the end of the column) tells us why your efforts are in vain. 

– DJ

For more on QB Score, RB Score and what these metrics mean see

The New QB Score

Consistent Inconsistency in Football

Football Outsiders and QB Score

The Value of Player Statistics in the NFL

Advertisement
Posted in: Football Stories