Week 15 Football Rankings and a Sabermetric Debate

Here are the quarterback and running back rankings for Week 15:

Table One: Quarterback Rankings for Week 15

Table Two: Running Back Rankings for Week 15

As I have noted, I am a suffering Lions fan.  Given my allegiance, my desire to write about football in any great detail has waned a bit.

Normally a lack of interest in a topic would make for a very short column.  Although a short column might be a good idea, I have decided to resist the urge to keep my comments brief by shifting to a completely different story.

Rob Neyer Starts a Discussion

This story begins with a column written by Rob Neyer (of ESPN.com).  A few days ago Neyer noted that Sabermetrics make baseball even more interesting.  Some might think that adding math to baseball would reduce interest, since it would appear to eliminate the arguments that make sports so much fun.  Neyer argued — because Sabermetricians couldn’t agree — it appears that adding math to baseball simply added to the discussion (and hence enjoyment) of the game.

MGL and Bradbury Agree

The disagreeing Sabermetricians identified by Neyer were JC Bradbury and Tangotiger. Bradbury is an economist at Kennesaw State and author of The Baseball Economist.  Tangotiger is… well, I am not sure where Tangotiger is employed. He did co-author a book on baseball, called The Book.  

In response to Neyer’s discussion of disagreement between Bradbury and Tangotiger, Mitchel Lichtman – or MGL – posted the following:

JC, with all due respect, is an economist with some knowledge of sabermetrics and not a sabermetrician. As well, I don’t think he is an expert on projections by any means. There are plenty of experts with respect to projections, and I don’t think – in fact I know – that any of them will project his value at anything close to 14mm per year.

In sum, MGL says Bradbury is not in the Sabermetric club.  In response, Bradbury posted the following:

I agree with MGL, I shouldn’t be considered a sabermetrician. I have never claimed to be a member of this community. What I do is apply my knowledge from my economics training and experience with analyzing data to issues in baseball. While MGL believes I do not understand certain sabermetric principles, I believe many sabermetricians (MGL included) underestimate the difficulty in analyzing baseball phenomena, especially when it comes to valuing players. Sabermetricians have made important discoveries (Voros McCracken’s work on DIPS was an important and correct finding); however, much what passes for research within this community is not sufficiently rigorous to reach the conclusions often claimed. There are many academic researchers from a variety of fields who have significantly advanced the understanding of baseball that receive scant mention in the sabermetric community. For example, Michael Schell’s Baseball’s All-Time Best Sluggers is the most thorough treatise on hitting ever written; yet, few individuals mention his work or attempt to replicate his methods. You rarely see economists Gerald Scully or Tony Krautmann mentioned when attempting to value players, despite the fact that their methods were published in reputable peer-reviewed economics journals, where established experts vetted their work. Academics are not always right, but I believe the checks ensure they are more likely to reach correct conclusions than informal online discussions.

As Sabernomics (Bradbury’s blog), Bradbury continued the discussion by once again noting that he’s not a Sabermetrician:

The sabermetric community is quite broad, and I’m not sure how to properly define it. But, I will say that I consider myself to operate outside of what most people consider to be the sabermetric community, but that I often study similar questions. It wouldn’t be incorrect to say some of what I do is sabermetrics, but my approach and methods are different than the community standard. This doesn’t bother me.

I’m an economist. I didn’t grow up reading Bill James, playing Strat-O-Matic, or participating in old Usenet groups. I wish I had been aware of these things, but I think that my outside perspective has been an advantage when analyzing baseball.

Others Comment

Bradbury’s post led to a comment by Rod Fort.  Rod Fort is also an economist, currently working at the University of Michigan.  He has also published numerous academic articles in the field of sports economics and is author of one of the leading textbooks (Sports Economics).   Here is Fort’s observation:

In my experience, it occurs to me this way (the difference between SABRmetricians and economists).  SABRmetricians are trying to determine the MP part of MRP = MP x MR.  This is important work and part of what economists also try to do when they work on MRP.  Indeed, there is much to learn everywhere from both SABRmetricians and economists on this part of the interesting problem.  What distinguishes economists, in the main, is the rest of the story on MR.

This leads to interesting discussions that, I think, miss the point of the importance of what the two types of researchers do!  Economists (partly in the name of parsimony) might not be as precise as SABRmetricians in capturing MP (although many economists are pretty good SABRmetricians in this regard-I think of you and Dave Berri).  In essence, if two MP measures are quite highly correlated and one is easier to get, then in the ultimate regression analysis OF MRP why worry about the more complicated measure?  This bothers SABRmetricians doing heroic battle over which MP measure best captures player contributions to winning.

And, learned first-hand in my interactions with them, SABRmetricians simply are not interested in the MR part.  So, no surprise, whenever they do bring it up, they do it in ways that bother economists.

But the idea that the analyses are at odds is counterproductive.  Both are important in determining MRP.  Economists doing MRP may sharpen their analysis by paying attention to SABRmetric results-some measurements may be worth the extra effort and vice versa. 

Fort’s observation was followed by a comment from PWHjort (and I don’t know who that is) which quotes extensively from Bill James.

Here’s a few quotes from a Bill James article entitled “Intro to Sabermetrics”

“Sportswriters discuss a range of questions which are much the same from generation to generation. Who is the Most Valuable Player? Who should go into the Hall of Fame? Who will win the pennant? What factors are important in winning the pennant? If Boston won the pennant, why did they win it? If Kansas City finished last, why did they finish last? How has baseball changed over the last few years? Who is the best third baseman in baseball today? Who is better, Mike Lowell or Eric Chavez?”
“The questions that we deal with in our work are the same as the questions that are discussed by sports columnists and by radio talk show hosts every day. To the best of my knowledge, there is no difference whatsoever in the underlying issues that we discuss. The difference between us is very simple. Sportswriters always or almost always begin their analysis with a position on the issue. We always begin our analysis with the question itself.”
“If you find a sportswriter debating who should be the National League’s Most Valuable Player this season, his article will probably begin by asserting a position on the issue, and then will argue for that position. If you find 100 articles by sportswriters debating issues of this type, in all likelihood all 100 articles will do this.”
“What we do is simply to begin by asking “Who is the National League’s Most Valuable Player this season?” rather than to begin by stating that “Albert Pujols is the National League’s Most Valuable Player this season, and let me tell you why.” That’s all. That is the entire difference between sabermetrics and traditional sportswriting. It isn’t the use of statistics. It isn’t the use of formulas. It is merely the habit of beginning with a question, rather than beginning with an answer.”
“We are no more statisticians than we are historians, or scouts, or accountants, or computer programmers. I suspect that everything we do is much the same as what many of you do. We look to the past, and we try to organize the things we have seen so that they make some sense. We ask ourselves “how many of those were there?” and “how many of those others were there?” and “How many of them ended well?” and “How many of them ended badly?”, just as I would imagine most of you do. “

PWHjort concluded that according to Bill James, Bradbury was quite wrong. Indeed he is a Sabermetrician.

My Comments

It appears to me that whether or not Bradbury is a Sabermetrician depends upon the definition of the term. Bill James focuses on the nature of inquiry.  Sportswriters — as James notes – frequently argue from conclusion to evidence. 

It’s important to note that sportswriters are not alone in this habit.  I have often thought that the primary difference between the nature of academic inquiry and what you see in non-academic forums is that academics are trained to argue from evidence to conclusion.  In contrast, non-academics often seem to begin with a conclusion and then seek out evidence to support their point of view.  And when that point of view is challenged, you often see non-academics resort to such comments as “your analysis defies common-sense” or “your analysis is not consistent with the views of a particular “expert” (or “experts”)”.  Academics tend not to be persuaded by arguments that purely rely on an appeal to a particular authority.  At least, we tend to prefer actual evidence.

To the extent that other Sabermetricians are actually following in the spirit of the Bill James definition, then one cannot separate Bradbury from other people who investigate baseball empirically.  Yet, it’s clear many of these people are somewhat hostile to Bradbury (and other academics).  I think this hostility stems from a second definition of Sabermetrics.  This definition – which I do not think is often articulated – would be as follows: “A Sabermetrician is someone who presents and discusses baseball within the forums of other Sabermetricians.”  In other words, if you participate in the group, then you are a Sabermetrician.  If you are not part of the club – even if you are doing legitimate analysis of baseball – then you are not a Sabermetrician.  By this definition, Bradbury is not a Sabermetrician.  After all, as he notes, he’s an economist who employs data from baseball to explore questions within the field of economics.

The comments from Fort highlight the nature of that inquiry.   The interest economists have in sports stems from the quality of the worker productivity data found in this industry.  Because we can measure the productivity of players in many sports, we can in turn address a variety of economic issues. In addressing these issues we wish to employ a measure of performance that is both accurate and parsimonious.  In other words, academics prefer to keep it simple if possible.

My sense is that many on-line Sabemetricians tend to have the opposite preference set.  In other words, given a choice between a simple or complex measure, the on-line Sabermetrics crowd prefers the latter.  I can illustrate this point by comparing John Hollinger’s Player Efficiency Rating (PERs) and the NBA Efficiency measure.  As I have noted in the past, there is a very high correlation between these two measures.  PERs, though, is much more complicated.  And consequently – following the above observation — you often see PERs cited on-line and NBA Efficiency ignored.

Both PERs and NBA Efficiency are good measures of how performance in the NBA is perceived by decision-makers (but not the best measures of how actual performance connects to actual wins).  For an academic to employ PERs, though, he/she would have to devote the limited space in an academic article explaining how Hollinger calculates this measure.  It’s simply far easier to use NBA Efficiency, even if some people outside of academia don’t appreciate that choice.

Let me close by noting that Tom Boswell — in The Hidden Game of Baseball – made a similar observation (an observation that I think is often ignored, although not by JC Bradbury): “the more ambitious the stat, the more complex and arbitrary it almost always becomes. What it gains in sophistication and the intuitive wisdom of its creator, it loses in simplicity and objectivity. How can you love a stat, or use it in arguments, if you can’t really explain it?”

– DJ

The WoW Journal Comments Policy

For more on the Wages of Wins football metrics see

The New QB Score

Consistent Inconsistency in Football

Football Outsiders and QB Score

The Value of Player Statistics in the NFL

Advertisement

Posted in: Basketball Stories