By Request, the History of the Efficiency Measures in the NBA

Posted on February 1, 2011 by

61


Some time ago, Andres (Dre) Alvarez – of NerdNumbers and the creator of the Automated Wins Produced page – asked me about the history of the NBA Efficiency measure.  As Dre noted on Sunday (during our podcast) – when this issue came up again — the NBA Efficiency measure is the primary statistical summary measure at NBA.com.  Yet where this came from is somewhat of a mystery.  So by request, I am going to discuss what I know of the origins of NBA Efficiency (and if anyone knows any more behind this story, please let us know). 

Back in the early 1990s (maybe 1993?) I had my first exposure to sports economics.  Much of the research at the time examined baseball (this has changed dramatically in the past 20 years).  Given my interest in basketball, I decided to see if I could employ the data generated by basketball players to answer various questions in economics. 

Such research seemed to require a summary measure of basketball performance (similar to batting average, slugging average, linear weights, etc… in baseball). So I started searching for a similar measure in basketball.

This search turned up two potential – and similar – candidates.  Dave Heeran offered the TENDEX model.

TENDEX = {[(PTS + REB + AST + BLK + STL – TOV – All Missed Shots)/minutes played]/game pace}*minutes played

In the introduction to the 1994-95 Basketball Abstract (there were four editions before this book appeared), it’s noted that “…Heeran invented his TENDEX system for evaluating players 35 years ago.”  So that means this model was developed around 1960. 

The TENDEX model is quite similar to Robert Bellotti’s Points Created measure.  The simple version of this measure (a more complex version adjusts for the average number of points scored per possession) – as detailed in The Points Created Basketball Book of 1991-92 – is as follows:

Points Created = PTS + REB + AST + BLK + STL – Missed Shots – TOV – PF/2

Obviously Points Created is quite similar to TENDEX.  And both are quite similar to NBA Efficiency.

NBA Efficiency = PTS + REB + AST + BLK + STL – TOV – All Missed Shots

Given these similarities, I would argue that NBA Efficiency has its origins in the work of Dave Heeran.  And that means it goes back about 50 years.

At first glance it would seem these measures are too simplistic to be of much value.  Bellotti, though, offered this defense of his Points Created model:

“Points Created is accurate.  … This contention is borne out by facts.  In the past eight years, the NBA’s Most Valuable Player has finished either first or secon that season in my Points Created rankings.  In the past 14 years, the MVP has finished first or second 12 times in Points Created.  In the other two years, the MVP finished third and fourth in Points Created, and in both years, the margin between the top three or four players was small.”

The MVP award is decided by members of the media, so explaining this vote may not prove the accuracy of the model.  One should note, though, that one can connect other player evaluations to the NBA Efficiency class of models and the story is essentially the same.  Whether we look at salaries (decided by NBA general managers) or voting for the All-Rookie team (decided by the NBA coaches), the NBA Efficiency model does a great job of explaining the evaluations we observe by NBA decision-makers.

Still, it is a simple metric.  And people tend to be more impressed by complexity.  So in recent years we have John Hollinger’s Player Efficiency Rating.  This metric is much more complex than NBA Efficiency, Points Created, or TENDEX.  But as noted in the Wages of Wins Journal FAQ page, this complexity doesn’t really change much of the story.  Hollinger offers a simple measure of PER called Game Score. 

Game Score = PTS + 0.4 * FGM – 0.7 * FGA – 0.4*(FTA – FTM) + 0.7 * ORB + 0.3 * DRB + STL + 0.7 * AST + 0.7 * BLK – 0.4 * PF – TO

As noted on the FAQ page, “for the 2008-09 season, PER and Game Score per 48 minutes for the 445 NBA players employed had a 0.99 correlation.”

And although Game Score doesn’t look like NBA Efficiency, these measures are also quite similar.  Again, as noted on the FAQ page, “for the 2008-09 season there was a 0.99 correlation between a player’s NBA Efficiency and Game Score.”

Why are these measures so similar?  Yes, this issue is also addressed on the FAQ page:

These measures all align because each tells a similar story about player scoring.  For example, imagine a player who takes twelve shots from two-point range.  If he makes four shots, his NBA Efficiency will rise by eight.  The eight misses, though, will cause his value to decline by eight. So a player breaks-even with respect to NBA Efficiency by converting on 33% of his shots from two-point range.  From three-point range, a player only needs to makes 25% of his shots to break-even.

Most NBA players can exceed these thresholds.  Therefore, the more shots most NBA players take the higher will be his NBA Efficiency total.  As a consequence, players who take a large number of shots tend to dominate the player rankings produced by this measure.

For Game Score the same problem exists, only the problem is a bit worse.  The break-even point on two-point shots for Game Score is 29.2%.  From three-point range a player breaks-even if he hits on 20.6% of his shots.  If a player surpasses these break-even points – and again, most players can do this – then the more shots he takes the higher will be his value.

Because these measures reward a player for just taking shots, they don’t tend to explain wins very well.  A team’s NBA Efficiency only explains 32% of the variation in team wins.  A team’s Game Score and PER explains 31% and 33% of the variation in win respectively.  One might note, though, that these measures don’t include the team defensive adjustment employed in the calculation of Wins Produced.  Unfortunately, if you add the team defensive adjustment to NBA Efficiency, Game Score, and PERs, explanatory power only rises to 58%, 60%, and 56% respectively.

One can go one step further and allow the individual components of the team defensive adjustment (detailed in Berri (2008) and employed in the calculation of Wins Produced) to vary. Such a step does raise the explanatory power of PERs to 82%. Wins Produced, though, explains 95% of wins, so even with the team defensive adjustments components added, the more popular measures come up short.

One should note that PERs –by itself – 0nly explains about 33% of team wins.  If you add in all the defensive variables – and you let the coefficients take on any value – you can raise the explanatory power to 82%.  But then, it is the team defensive factors that are offering the bulk of your explanatory power.   So what you learn about individual players from PERs is still not helping much.  Finally – as noted – even if you let the team defensive variables take on any value, you still can’t match the explanatory power of Wins Produced.

Let me summarize what we know about these measures:

  • The efficiency metrics seem to derive from the work of Dave Heeran, and that means these metrics go back about 50 years.
  • The story told by TENDEX, Points Created, NBA Efficiency, and the Player Efficiency Rating is quite similar.  Although these metrics look different, the measures are highly correlated.
  • These measures – as Bellotti notes – do a wonderful job of explaining player evaluation. So if this is your objective – and I have published work with co-authors (a paper with Tony Krautmann and Peter von Allmen looking at monopsonistic exploitation in sports is a good example) that have used NBA Efficiency – then these measures are quite useful.
  • These measures, though, over-value inefficient scoring. 
  • As a consequence, these measures are not a very good measure of a player’s actual productivity (i.e. actual contribution to team wins).  A point we can clearly see when we look at how well these measures actually explain wins in the NBA.

And that means, the search had to continue to find a metric that captured an NBA player’s performance on the court.  That search led to an article published with Stacey Brook examining trades in the NBA in 1999 (originally presented in 1997).  The model presented with Stacey was further revised for a paper I published in Managerial and Decision Economics in 1999.  That model was then revised for a paper published with Tony Krautmann in 2006 (which appeared in Economic Inquiry). And that model was modified for the Wins Produced model presented in The Wages of Wins (and yet another paper published in 2008).

After all this history, what will we see in the future?  Wins Produced – as noted – explains more of wins than any of the NBA Efficiency family of metrics.  So will the Efficiency metrics – after 50 years – start fading from use? 

No, these measures are still consistent with popular perception.  And I just don’t think popular perception – which focuses on scorers – is going to change any time soon.  So if you fear Wins Produced is going to take over the NBA… well, I don’t think you have to worry.  And if you want people to pay more attention to players like Landry Fields and less attention to Andrea Bargnani…. well, you are probably going to be disappointed.

Let me close by emphasizing that the Wins Produced metric was created because a measure of how a player contributes to wins seems necessary to address various issues important to economists (at least, important to this economist).  It was not created in an effort to change how people view basketball (although if it does this, I am okay with that) or in an effort to change how NBA teams make decisions (although if it does this, I am okay with that also).  Again this metric was designed to further research in economics.  And for the reasons stated above, the efficiency measures – and one might add, the plus-minus measures (for reasons stated in the FAQ page) – are not as helpful because they do not appear to be good representations of the productivity of individual players. 

– DJ

Advertisements