The New Orleans Saints and Green Bay Packers are both recent Superbowl champions. We might expect that their coaching staff knows what they are doing. However, when looking at a simple judgement call – going for it on fourth down – I question if even the best of the best in the NFL have a complete grasp of the game.
A fun tale from Stumbling on Wins and Scorecasting is the story of coach Kevin Kelly. He is a high school coach in Arkansas who hates punting or going for field goals. He blindly flies in the face of conventional wisdom. And Kelly’s team tends to win a large number of football games.
Kelly’s approach seem to follow from the research of economist David Romer. Back in 2006 (in the Journal of Political Economy) Romer employed some fairly sophisticated math and examined when a team should go for it on 4th down.
Romer’s research indicates that much of the time, going for it is the right decision. As the above figure indicates, if you are past midfield you should probably always go for it if you have less than four yards to go. And even if you are on your side of the field, going for it can be a good decision.
Let’s take a brief review of Thursday’s NFL opener and see how these teams did relative to what Romer uncovered.
- Green Bay punted from their own 20 with 10 yards to go – right call
- New Orleans punted from Green Bay’s 45 with 14 yards to go – right call
- Green Bay punted from their own 27 with 3 yards to go – wrong call
- New Orleans punted from their own 22 with 14 yards to go – right call
- Green Bay punted from their own 37 with 16 yards to go – right call
- Green Bay punted from New Orleans 37 with 4 yards to go – wrong call
How about Field Goals?
- New Orleans kicked a field goal from Green Bay’s 12 with 4 yards to go – wrong call
- New Orleans kicked a field goal from Green Bay’s 20 with 15 yards to go – right call
Finally how about going for it on 4th down?
- New Orleans went for it on 4th down with 1 yard to go on Green Bay’s 7 – right call
Now if we review Thursday’s game we find 9 situations where the tough choice of going for it came up. The current champs made the right call 50% of the time. The Saints — or the former champs — fared slightly better with a 80% correct rate. And that means — according to Romer’s research — the Packers and Saints were incorrect at least 33% of the time.
This game was expected to be a match up of two of “best” teams in the NFL. Fantasy fans were happy with the performance of Rogers and Brees. From Romer’s research, though, we see even the best teams in the NFL still make mistakes. In a game decided by less than a touchdown a touchdown and a two point conversion (editor’s note: thanks Alex!) there is no doubt mistakes like these could have impacted the final score.
Of course, the truth is that the coaches of both the Saints and the Packers will come away happy. They can say their team played a great game. And although their decisions didn’t conform to Romer’s research, it probably followed conventional wisdom.
Conventional wisdom, though, would seem to be reducing a team’s chances to win. And that means coaches may be making decisions that most fans like, but not the decisions that maximize the probability of winning. So the real truth may be that the coaches appear to know exactly what they’re doing, but keeping up these appearances might be hurting their team’s chances to walk away with a victory.
-Dre
Wages of Wins Journal 
Alex
September 9, 2011
I like Brian Burke’s extension/update: http://www.advancednflstats.com/2009/09/4th-down-study-part-4.html. It mostly agrees with Romer’s, except it makes the Saints’ field goal from the 12 more of a borderline call (I think the line goes across at the 5, making the FG on 4 yards to go technically wrong but close) and Green Bay punting from their own 27 a good call. Although things have probably changed a little bit for both analyses since touchbacks are now more likely on kick-offs, so the expected points following scores have changed somewhat.
Also, the game was decided by a touchdown plus the two-point conversion, not less than a touchdown.