Some time ago, Andres (Dre) Alvarez – of NerdNumbers and the creator of the Automated Wins Produced page – asked me about the history of the NBA Efficiency measure. As Dre noted on Sunday (during our podcast) – when this issue came up again — the NBA Efficiency measure is the primary statistical summary measure at NBA.com. Yet where this came from is somewhat of a mystery. So by request, I am going to discuss what I know of the origins of NBA Efficiency (and if anyone knows any more behind this story, please let us know).
Back in the early 1990s (maybe 1993?) I had my first exposure to sports economics. Much of the research at the time examined baseball (this has changed dramatically in the past 20 years). Given my interest in basketball, I decided to see if I could employ the data generated by basketball players to answer various questions in economics.
Such research seemed to require a summary measure of basketball performance (similar to batting average, slugging average, linear weights, etc… in baseball). So I started searching for a similar measure in basketball.
This search turned up two potential – and similar – candidates. Dave Heeran offered the TENDEX model.
TENDEX = {[(PTS + REB + AST + BLK + STL – TOV – All Missed Shots)/minutes played]/game pace}*minutes played
In the introduction to the 1994-95 Basketball Abstract (there were four editions before this book appeared), it’s noted that “…Heeran invented his TENDEX system for evaluating players 35 years ago.” So that means this model was developed around 1960.
The TENDEX model is quite similar to Robert Bellotti’s Points Created measure. The simple version of this measure (a more complex version adjusts for the average number of points scored per possession) – as detailed in The Points Created Basketball Book of 1991-92 – is as follows:
Points Created = PTS + REB + AST + BLK + STL – Missed Shots – TOV – PF/2
Obviously Points Created is quite similar to TENDEX. And both are quite similar to NBA Efficiency.
NBA Efficiency = PTS + REB + AST + BLK + STL – TOV – All Missed Shots
Given these similarities, I would argue that NBA Efficiency has its origins in the work of Dave Heeran. And that means it goes back about 50 years.
At first glance it would seem these measures are too simplistic to be of much value. Bellotti, though, offered this defense of his Points Created model:
“Points Created is accurate. … This contention is borne out by facts. In the past eight years, the NBA’s Most Valuable Player has finished either first or secon that season in my Points Created rankings. In the past 14 years, the MVP has finished first or second 12 times in Points Created. In the other two years, the MVP finished third and fourth in Points Created, and in both years, the margin between the top three or four players was small.”
The MVP award is decided by members of the media, so explaining this vote may not prove the accuracy of the model. One should note, though, that one can connect other player evaluations to the NBA Efficiency class of models and the story is essentially the same. Whether we look at salaries (decided by NBA general managers) or voting for the All-Rookie team (decided by the NBA coaches), the NBA Efficiency model does a great job of explaining the evaluations we observe by NBA decision-makers.
Still, it is a simple metric. And people tend to be more impressed by complexity. So in recent years we have John Hollinger’s Player Efficiency Rating. This metric is much more complex than NBA Efficiency, Points Created, or TENDEX. But as noted in the Wages of Wins Journal FAQ page, this complexity doesn’t really change much of the story. Hollinger offers a simple measure of PER called Game Score.
Game Score = PTS + 0.4 * FGM – 0.7 * FGA – 0.4*(FTA – FTM) + 0.7 * ORB + 0.3 * DRB + STL + 0.7 * AST + 0.7 * BLK – 0.4 * PF – TO
As noted on the FAQ page, “for the 2008-09 season, PER and Game Score per 48 minutes for the 445 NBA players employed had a 0.99 correlation.”
And although Game Score doesn’t look like NBA Efficiency, these measures are also quite similar. Again, as noted on the FAQ page, “for the 2008-09 season there was a 0.99 correlation between a player’s NBA Efficiency and Game Score.”
Why are these measures so similar? Yes, this issue is also addressed on the FAQ page:
These measures all align because each tells a similar story about player scoring. For example, imagine a player who takes twelve shots from two-point range. If he makes four shots, his NBA Efficiency will rise by eight. The eight misses, though, will cause his value to decline by eight. So a player breaks-even with respect to NBA Efficiency by converting on 33% of his shots from two-point range. From three-point range, a player only needs to makes 25% of his shots to break-even.
Most NBA players can exceed these thresholds. Therefore, the more shots most NBA players take the higher will be his NBA Efficiency total. As a consequence, players who take a large number of shots tend to dominate the player rankings produced by this measure.
For Game Score the same problem exists, only the problem is a bit worse. The break-even point on two-point shots for Game Score is 29.2%. From three-point range a player breaks-even if he hits on 20.6% of his shots. If a player surpasses these break-even points – and again, most players can do this – then the more shots he takes the higher will be his value.
Because these measures reward a player for just taking shots, they don’t tend to explain wins very well. A team’s NBA Efficiency only explains 32% of the variation in team wins. A team’s Game Score and PER explains 31% and 33% of the variation in win respectively. One might note, though, that these measures don’t include the team defensive adjustment employed in the calculation of Wins Produced. Unfortunately, if you add the team defensive adjustment to NBA Efficiency, Game Score, and PERs, explanatory power only rises to 58%, 60%, and 56% respectively.
One can go one step further and allow the individual components of the team defensive adjustment (detailed in Berri (2008) and employed in the calculation of Wins Produced) to vary. Such a step does raise the explanatory power of PERs to 82%. Wins Produced, though, explains 95% of wins, so even with the team defensive adjustments components added, the more popular measures come up short.
One should note that PERs –by itself – 0nly explains about 33% of team wins. If you add in all the defensive variables – and you let the coefficients take on any value – you can raise the explanatory power to 82%. But then, it is the team defensive factors that are offering the bulk of your explanatory power. So what you learn about individual players from PERs is still not helping much. Finally – as noted – even if you let the team defensive variables take on any value, you still can’t match the explanatory power of Wins Produced.
Let me summarize what we know about these measures:
- The efficiency metrics seem to derive from the work of Dave Heeran, and that means these metrics go back about 50 years.
- The story told by TENDEX, Points Created, NBA Efficiency, and the Player Efficiency Rating is quite similar. Although these metrics look different, the measures are highly correlated.
- These measures – as Bellotti notes – do a wonderful job of explaining player evaluation. So if this is your objective – and I have published work with co-authors (a paper with Tony Krautmann and Peter von Allmen looking at monopsonistic exploitation in sports is a good example) that have used NBA Efficiency – then these measures are quite useful.
- These measures, though, over-value inefficient scoring.
- As a consequence, these measures are not a very good measure of a player’s actual productivity (i.e. actual contribution to team wins). A point we can clearly see when we look at how well these measures actually explain wins in the NBA.
And that means, the search had to continue to find a metric that captured an NBA player’s performance on the court. That search led to an article published with Stacey Brook examining trades in the NBA in 1999 (originally presented in 1997). The model presented with Stacey was further revised for a paper I published in Managerial and Decision Economics in 1999. That model was then revised for a paper published with Tony Krautmann in 2006 (which appeared in Economic Inquiry). And that model was modified for the Wins Produced model presented in The Wages of Wins (and yet another paper published in 2008).
After all this history, what will we see in the future? Wins Produced – as noted – explains more of wins than any of the NBA Efficiency family of metrics. So will the Efficiency metrics – after 50 years – start fading from use?
No, these measures are still consistent with popular perception. And I just don’t think popular perception – which focuses on scorers – is going to change any time soon. So if you fear Wins Produced is going to take over the NBA… well, I don’t think you have to worry. And if you want people to pay more attention to players like Landry Fields and less attention to Andrea Bargnani…. well, you are probably going to be disappointed.
Let me close by emphasizing that the Wins Produced metric was created because a measure of how a player contributes to wins seems necessary to address various issues important to economists (at least, important to this economist). It was not created in an effort to change how people view basketball (although if it does this, I am okay with that) or in an effort to change how NBA teams make decisions (although if it does this, I am okay with that also). Again this metric was designed to further research in economics. And for the reasons stated above, the efficiency measures – and one might add, the plus-minus measures (for reasons stated in the FAQ page) – are not as helpful because they do not appear to be good representations of the productivity of individual players.
– DJ
Wages of Wins Journal 
EvanZ
February 1, 2011
“These measures all align because each tells a similar story about player scoring. For example, imagine a player who takes twelve shots from two-point range. If he makes four shots, his NBA Efficiency will rise by eight. The eight misses, though, will cause his value to decline by eight. So a player breaks-even with respect to NBA Efficiency by converting on 33% of his shots from two-point range. From three-point range, a player only needs to makes 25% of his shots to break-even.”
This analysis is too simplistic. A player who takes 12 FGA will on average get 3.67 FTA. (The NBA average FTA/FGA is ~30%.) The average FT% is 76% so the player will on average add 2.8 points on FT shooting. Moreover, the average TOV% in the league is roughly 13%. This means a player with 12 FGA will turn the ball over on average 1.76 times. Finally, some of those missed field goal attempts will be rebounded by the offense, so the player should not be debited for a full possession lost each time he misses.
Take all these things into account, and you will find a different break-even point. I’ve calculated the “true break even point” for ezPM (click on the link) and it’s about 50%, which happens (not coincidentally I would argue) to be the league average eFG%. It also happens to result in 1.06 point per possession (PPP), also not coincidentally, league average. What’s the true break-even point for WP? I’ll leave it up to the professor to tell us that one.
Italian Stallion
February 1, 2011
EvanZ,
Could you explain your thinking here further? I may be missing something.
It seems you are saying you shouldn’t deduct as much for the misses because of potential for FTs and OREBs. But things like OREBs, made FTs etc… ARE added back and given credit to the players. If you don’t subtract from the misses but then you do add those values elsewhere, it seems like you are double counting.
To me, this debate is almost entirely about the relationship between scoring and usage.
IMO, there’s almost no question the very popular metrics like PER overrate scoring. The actual degree of overvaluation may not be as huge as the theoretical potential because players that aren’t very inefficient don’t get to play or shoot very much, but it’s there anyway. It also doesn’t recognize the high level of diminishing returns for scoring and the capability of other players to up their usage mildly with little or no impact.
If I had the time, energy, and mathematical knowledge to develop a model I could present to others for review, my own would value scoring approximately the same as this one but I’d tweak it mildly as usage moved away from the mean in either direction.
For gambling purposes, that’s approximately what I do now (I also look at specific details of the other players on the team to see how much I think the other players can alter usage) without worrying about minor errors or misjudgments. I don’t place bets unless I have a margin of safety large enough to compensate for my own ignorance on the exact relationship.
EvanZ
February 1, 2011
IS, search WordPress blogs for “point break even”. You’ll find my recent post. Apparently, I’m not allowed to post links here anymore.
Holland
February 1, 2011
Yes! Been waiting for this post for a while, wondering where all those other metrics came from in the first place. Crazy to think someone defended their scientific research via a popularity poll! (but sadly not surprising either)
tgt
February 1, 2011
EvanZ,
Yes, around 50% eFG is the break even point for scoring. The issue is that the break even points for PER, game score, etc are well below 50% eFG.
Greyberger
February 1, 2011
“At first glance it would seem these measures are too simplistic to be of much value.”
Forgive me for sounding snide, but that’s how a lot of people react to WP these days. Time marches on: I’m sure WP or proto-WP seemed revolutionary compared to metrics of the day, but it’s 2011 and linear box-score mash-ups aren’t taken as seriously.
Even Hollinger uses PER to summarize and as one tool among many. He’ll be the first to tell you it doesn’t capture defense, context, role etc.
I’d like to hear about the future of WP as well as its history – have there been any improvements in our understanding or tracking of NBA games since the latest major revision that have made you think of ways it might be better?
EvanZ
February 1, 2011
“The issue is that the break even points for PER, game score, etc are well below 50% eFG.”
Actually, the issue is whether that is really true. And moreover, what is the break-even point for WP? My guess is that it is actually greater than 50%. By all means, someone prove me wrong.
Greyberger
February 1, 2011
I thought the offense part of WP was more or less PTS + ORB – (FGA + 1/2FTA + TO). So a player helps/hurts his WP with every bucket over/under an eFG of 50%, ignoring the ORB and TO part.
EvanZ
February 1, 2011
“ignoring the ORB and TO part.”
Can teams “break even” and ignore those, too? If the answer is different for teams and players, then you’ve got a problem.
Greyberger
February 1, 2011
My mistake the coefficient for FTA is .47, not .50. Which seems weird to me I thought there were a lot more FTA that don’t lead to a possessions switch than that; the coefficient used by the possession statistic and usage and a lot of other advanced stats is .44. Something (else) to wonder about.
Italian Stallion
February 1, 2011
EvanZ
The turnover issue is complex because higher usage players tend to accumulate more turnovers. Even if their TO% is average or better and the model isn’t accumulating enough credit for their scoring, they actually get penalized for usage.
I’m not sure it’s the same with OREBs unless you want to track which shots are more likely to produce an OREB than others and give players credit for the types of shots they miss.
The way I see it , an OREB is simply an extra possession and the guy who gets it deserves the credit.
EvanZ
February 1, 2011
So, at the very least, we appear to agree that Berri’s criticism of PER’s break-even point is flawed. Right?
EvanZ
February 1, 2011
In other words, we can’t both be right. If you think my calculation is too complex to be right, then how can Berri’s approach be correct, if he doesn’t take into account TOV at all?
Joe
February 1, 2011
http://www.wagesofwins.com/CalculatingWinsProduced.html
Daniel
February 1, 2011
EvanZ- you’ve taken over the comment section without actually thinking about the issue at hand. Berri’s approach is correct because IT’S CORRECT. He may be missing some things here or there, and I’m sure he’d love to tweak his approach if he was provided with something new that could improve its accuracy, but it has a 95% correlation to team wins while the other methods have closer to a 60% correlation.
The issue you’re seeing with the break-even point in shooting efficiency doesn’t exist. “Break-even” may not be the best term here- it should be “average”– the issue is that the other metrics don’t use average shooting efficiency, they use a silly number that has no foundation in reality. With WoW, it’s the league average. Pretty crazy break-even point, eh? NBA Efficiency tells us that a team that shoots 33% on 2-pointers is going to be average. If you know anything about the NBA, this is INSANE, and makes it a poor metric, which is confirmed by its poor correlation to team wins.
If the statisticians responsible for these metrics (maybe the problem is that they aren’t really statisticians) could just fix the glaring issue with shooting efficiency, then their models would be more useful in describing the past and present, and more importantly, in predicting future results.
Mike
February 1, 2011
@EvanZ
PER has two major flaws:
1. A Player improves his PER when he shoots > 33% from 2, and > 25% from 3. Forget breakeven, you can shoot 41% and take lots of shots and improve PER. That is seems incorrect.
2. It doesn’t account for defense. That makes it, at best, a gauge of offensive ability with a rebounding, blocks and steals tweak, and at worst a theory so incomplete it is has no value.
WP’s suffers neither of these. And the flaws people bring up always seem to relate to people’s perception of the value of efficiency – with many arguing that less efficiency is better (WTF?)
But I have a rebuttal to that whole line of thinking.
It involves a scenario derived from the very common “if you had to have one player take a shot for your life, who would it be”?
My question is: if you had to choose a random player in a specific NBA game, and your life depending upon two things:
1. He has to score 20 points.
2. He has to make one, randomly selected one of those shots.
Who would you choose? Carmelo would usually hit part one (20 points) but part two? In his MVP season of 2000-01, who would you have trusted: Iverson or 2nd place Tim Duncan, if your life depended upon his shot selection? Imagine your life depended upon the next shot – would you really trust Iverson, and his oft times poor shot selection?
I think that scenario shows the difference between public perception – whihc favours great one off feats – and WP which favours the value of a player not on his ability in one (offensive) possession, but in all facets of the game and over the course of a whole game / season / career.
Greyberger
February 2, 2011
It’s rebounds. WP48 is infamous for mishandling rebounds – you should know this
EvanZ
February 2, 2011
@Mike,
Are you aware of my blog?
mystic
February 2, 2011
@Mike
WP has the same flaw regarding defense as PER has. The problem is that Berri used a trick to hide that a bit, but it is still there. Why? Because the boxscore stats aren’t representing defense enough to explain the overall defense of a team.
And claiming that WP has no issues with defense is some kind of a joke. According to Berri Kevin Love is by far the best player in the league right now. Did you ever watch him play defense? The guy is a mess, because he basically skips his defensive assignments for rebounding. The result is the Wolves are getting worse on defense. Kevin Love is making the team defense worse, that is not just a result we are getting by the eye-test, but also a result we are getting by EVERY +/- based metric.
According to the same metric (WP), the Mavericks should have never traded Kris Humphries for nothing, because apperently he is more valuable than Dirk Nowitzki in this season. Is there even ONE single person out here, who would want Kris Humphries over Dirk Nowitzki on his team? Heck, Humphries is doing better per minute than LeBron James. Who would have thought this? And what does happen in reality? Well, with Humphries on the court the Nets are getting more outscored than without him. Especially on defense the guy is weak. Using adjusted plus minus here we are ending up with an average player overall and a way below average on defense.
Seriously, the whole concept of WP is flawed, heavily flawed.
Greyberger
February 2, 2011
Re: Mike,
your example isn’t a good one. In 2001 Tim Duncan’s offense looked like this:
.536 TS, 15% AST%, 12% TOV.
Allen Iverson’s offense:
.518 TS, 23% AST%, 10% TOV.
Italian Stallion
February 2, 2011
mystic,
As I’ve been saying recently, I think a player’s value is a variable that depends on the makeup of the rest of the team.
I won’t bring up the usage issue again in detail again. It’s enough for me to say that IMHO Dirk is a bit better than he looks on this model and Humphries is a little worse because they are at opposite ends of the usage range.
However, the key point is that Dirk is a scorer and Humphries is not.
On a team that already had several efficient scorers and not enough rebounding, adding Humphries would probably be more valuable than adding Dirk. On a team that is already missing Butler and that has a PG that is not a scorer either (like Dallas), Dirk is much more valuable.
I think the idea that value is fixed or totally related to position is not correct. It’s dependent on the makeup of the team. The trick is getting the values correct and then putting the players together.
IMHO, Humphries has been a very valuable player this year and is very underrated by most analysts because they focus on scoring.
mystic
February 2, 2011
@Italian Stallion
Of course the makeup of the team will be a point for the decision makers, but we are talking about Dirk Nowitzki and Kris Humphries here. The one guy scores 24 points per 36 minutes on a higher efficiency than everyone else with a similar scoring rate (using everyone with 22+ per 36 minutes, which are 15 players right now, and ts% to evaluate scoring efficiency) and the other one is trading playing defense for rebounding. Humphries is rated above to slightly above average by nearly every boxscore based rating, he is rated as average to below average by every +/- based rating. The only rating which thinks Humphries would be an elite player is Win Produced (WP48).
You have to be blind to think that Humphries is even close to the value of a Dirk Nowitzki for EVERY team in the league. It would be different, if Nowitzki would be a low efficiency scorer, but he isn’t. He scores way above league average in terms of rate and efficiency. And that shows up in the results. Humphries impact isn’t seen in the results. That is a huge problem.
Kevin Love and Kris Humphries are prime examples of how overrated rebounding is in that metric. Neither of those players has any positive impact on defense, because both are skipping defensive assignments in order to grab the rebound. What the boxscore doesn’t show, how often that leads to an easier basket for the opponent.
Nerdnumbers
February 2, 2011
Mystic,
What the boxscore doesn’t show, how often that leads to an easier basket for the opponent.
Please explain how this a problem with Wins Produced. Is your contention that no advanced metric based on the box score can be trusted? If so I suggest you go to NBA.com and address the issue for EFF, ESPN for PER and Basketball-Reference for Win Shares.
mystic
February 2, 2011
Nerdnumbers,
it becomes a problem when rebounding becomes such a huge factor in a metric. If the value of a rebound is lower in comparison to other boxscore entries, it becomes less of a problem. Should be rather obvious. ;)
And I don’t evaluate other metrics here. I pointed out obvious flaws in Wins Produced. Especially when someone claims WP wouldn’t have an issue with defense.
Nerdnumbers
February 2, 2011
Mystic,
You use the word obvious twice here. I’m a little curious how you get that. I mean the values used in Wins Produced were systematically found and correlate well with what they are trying to model (the amount of Wins a team gets). Is it perfect for individual defense? No, but there are others who are working on it (See Courtside Analyst and Arturo’s Silly Little Stats). The way Wins Produced addresses defense is at the team level, which is a pretty good way to go as it turns out defense is a team activity.
Anyway I’m a little skeptical of things that are “obvious” without data. However, if I am wrong and you have proof as to why rebounds are overvalued I’d be very interested in looking it over.
EvanZ
February 2, 2011
“The way Wins Produced addresses defense is at the team level, which is a pretty good way to go as it turns out defense is a team activity.”
Isn’t offense a team activity, too?
Nerdnumbers
February 2, 2011
Evanz,
You clearly know the Wins Produced formula well. What team information does it use?
EvanZ
February 2, 2011
Dre, I’m not sure you understand my question/point, so let me be clear. Offense is a team activity just as much as defense. And defense is also an individual activity, as is offense. We don’t have as much information about the individual defensive efforts, but that doesn’t mean we can simply wish it away, or say those individual components are not significant. Therefore, it is a legitimate point of contention for critics of WP.
Alien Human Hybrid
February 2, 2011
EvanZ,
If WP “properly” accounted for team defensive contributions, how much more accurate do you surmise it would be in predicting wins?
mystic
February 2, 2011
Nerdnumbers,
first let me ask you a question: Do you think I haven’t read the paper? Or the book? Just asking, because somehow I get the feeling you are trying to imply that. ;)
If a=1 and b=2, which value will result in a higher value after a multiplication with 10? a or b? I thought it is obvious that b will result in a higher value, but maybe I’m a bit off.
Well, and Wins Produced correlates well on a team level, which doesn’t imply that it gets it right for the individual players. ;)
My own boxscore based rating has a 0.96 linear correlation coefficient (dataset used from 1984/85 season to 2009/10 season) to winning. Do I think it works well for every player or has no issues? Not really.
It is just funny when someone claims a metric has no issues with defense and than this metric ends up with guys like Kris Humphries being elite players. Especially when the author claims that the metric would explain wins the best, when the ability to predict future success was the lowest by any metric tested in Rosenbaum&Lewin (2007).
Btw., I didn’t say other boxscore based metrics have no issues.
Mike
February 2, 2011
@mystic who said defence “wasn’t an issue”? I wrote specifically (and nuance matters greatly):
I then wrote WP didn’t suffer that issue, which it doesn’t because it includes a defensive element in the calculations, which PER (steals and blocks aside) does not.
But it does reflect the quality of a defence overall. I don’t why this matters, from invalidating a metric. If this does matter, and you come up with an analysis that proves some rebounders hurt their team, this would supplant the current theory. It could very well be true, and if it then it is quantifiable. But without data, it is just an hypothesis which shouldn’t discredit a theory that is tested and replicable.
@Greyberger TS% accounts for Free throws. In 2000-01 Iverson shot 0.420, with 0.320 on threes, for an eFG% of .459. Still want him taking a random shot to determine the fate of your life? Even if you use ts%, and I said he had to make both free throws OR a shot, who would you take?
What about Carmelo vs Lebron? The point is that the real issue is who takes better shot most of the time, as opposed to in a one off situation. Carmelo has more ways to score than Lebron, but Lebron scores more efficiently and takes less awful shots.
The criticisms of defence as part of the metric revolves around an issue that many people misinterpret. The goal of WP is to explain victories. Not to evaluate a player fairly, but to explain wins and losses. A teams defence is a part of a teams record, and if a player is on a bad defence, the wins he produces will be reduced, no matter how good he individually is at defence.
Now if there was a way to improve that, to find a way to assign the true value of a defender, I’m all ears, but as a theory, WP DOES account for defence, no matter how inaccurately and unfairly.
mystic
February 2, 2011
Mike,
I pointed out TWO specific examples in which rebounders are not helpful for the team defense. In fact both are making the defense worse. You don’t believe it? Well, check out the OnCourt and OffCourt numbers for Kris Humphries and Kevin Love, check out their defensiv adjusted PM numbers. And I even explained why that’s the case.
I can add such a team factor to everything and I will not end up with a better evulation of the individual players while still having a higher correlation to winning on a team basis. And if WP doesn’t try to evaluate players fairly, what is the point of it? Explaining wins and losses on a team level can be made much easier by scoring margin. If you want a deeper look, check out the ORtg and DRtg. Why do you want to break it down for players, when the goal isn’t a “fair” evaluation of the players? Your statement makes no sense at all.
And what is the point of Anthony vs. James? I don’t know any metric which would claim Anthony is more valuable.
Mike
February 3, 2011
Then allow me to clarrify :)
The goal of a metric is make judgements between players possible. The question is what is the most effective way to do this, e.g. what metric helps decide which players are more valuable and productive, and which are less.
It isn’t meant to be “Fair” – it is meant to provide insight. The insight may be incomplete, and “unfair” in that is misses some elements, and a player might unfairly be mis-assigned because of some missing element, but that is nether here nor there. Is it an effective tool for deciding between two player’s worth and, more importantly, is there a better metric? If there is nothing better, an incomplete metric is by default the choice to make.
No, you have postulated an idea but not demonstrated it is true.
Besides, the argument that SOME rebounds MIGHT hurt defence (which again you have yet to show is true – you have just speculated) doesn’t make a rebound worth less in a metric, and certainly does not provide a mechanism to reassess the value of a rebound. To change the value of a metric, you need to correlate the thing being observed to a mathematical formula that can be shown to improve a metric’s ability to either predict or describe a phenomenon.
Two things:
1. So you are arguing against WP48, and yet your metric is 0.01 “better”? Fair enough!
2. When are you going to submit it to a journal for peer review? I’d happily choose a more effective metric if I have evidence and a reason. That is how knowledge works :)
mystic
February 3, 2011
Mike,
just let me ask you a question, because somehow I have a strange feeling here: Did you ever watch a game of the Minnesota Timberwolves or the New Jersey Nets? Because, if you would do that, you would actually understand the point really well. ;)
Anyway, I didn’t postulate an idea, I stated a FACT. I didn’t speculate, I know that this is true. But I can try it again:
When Kevin Love is on the court, the Minnesota Timberwolves have a DRtg of 112.1, that is basically as good as the WORST defense in the league right now (Cleveland Cavaliers with 112.3 DRtg). When he isn’t on the court, the Timberwolves are at 106.9 DRtg, which is basically equal to the league average.
Source: http://basketballvalue.com/teamplayers.php?year=2010-2011&team=MIN
See, the Timberwolves are playing worse defense with Love on the court than without him. Now you might argue that he has to play all the tough guys and the other time his teammates are only facing the bad players. That might be true, thus we can look at the defensive adjusted PM: http://stats-for-the-nba.appspot.com/ranking11
Kevin Love is listed with -1.2. He is actually making the defense worse in comparison to an average replacement player by 1.2 points per 100 possessions.
That is not an idea anymore, that is the reality.
The same thing for Humphries, 111.1 OnCourt DRtg, 4.6 points per 100 possessions than without him on the court. His defensive APM is -1.0. Again, the defense gets worse with him on the court and adjusted for the strength of teammates and opponents it ends up with a below average value (same sources).
Yeah, I know the arguments, it is noisy, it is … blabla. Matter of fact is in both cases the defense of their teams is horrible, while they can play decent defense when those players are not on the court. That is not an idea, that is the reality, Mike. You can ignore that and say something about blabla like you done that a lot in your previous post, but that will not change the reality.
And again, your statements about “fair” makes no sense at all. When I wrote “fair”, I meant it in a sense that the distribution of the team success to the players happened correctly. Which means a player gets credit for his contribution which really lead to the success. WP48 gives credit to Kevin Love to contribute more to winning than every other player in the league, but it doesn’t show up in the results. Is he just not compatible to his teammates? Or what is it? Why predicts WP48 the future success worse than other metrics (including minutes per game) according to Rosenbaum&Lewin (2007)?
My rating: http://bbmetrics.wordpress.com/player-ratings/
First of all, I didn’t say it is better, I just pointed out that it has a high correlation coefficient to winning. Which doesn’t prove that this rating gives credit to the right players. Even though defense is incorperated it features the same flaws regarding this as other metrics, due to the lack of informations in the boxscore for individual players. Second of all, unfortunately it will be rather tough for me to find some financial support to publish it somewhere., thus you probably have to live just with the results. I usually also don’t use it in an argumentation, because, as you pointed out, it isn’t published anywhere, thus it has no credibility.
And last but not least, I doubt that you are a decision maker in any kind in the NBA, thus you can use whatever metric you want and claim whatever you want. It is just sad when those things, especially when it is just based on one specific metric, aren’t compatible with the reality. ;)
Italian Stallion
February 3, 2011
Mystic,
“Kevin Love and Kris Humphries are prime examples of how overrated rebounding is in that metric. ”
I’ve read just about analysis on the value of rebounds I could find and I’ve concluded they are all probably wrong. I have some idea about how I would go about doing it I had the time, inclination, and mathematical skills, but it’s not going to happen.
In the mean time, you may think I am crazy, but I think the correct way to value them is the way it’s done here . For myself I simply make adjustments to a player’s value when his role, team, and circumstances change in a way that is likely to change his rebounds.
For example, when David Lee was with the Knicks IMO his value was very high despite the deficiencies on defense. On a team with Curry/Ellis taking all the shots and Biedrins also getting a lot of boards his value is lower. Same exact player (other than the injury issues that hurt his productivity and shot for awhile), but different values.
“Neither of those players has any positive impact on defense, because both are skipping defensive assignments in order to grab the rebound. What the boxscore doesn’t show, how often that leads to an easier basket for the opponent.”
I understand this issue, but until we have stats that measure how often this happens and who is doing it, I don’t know how to apply it properly. I don’t think you can or should globally reduce the value of rebounds because a handful of players are stat whores.
I think you can watch games and if you notice Kevin Love cheating on defense and costing the team easy baskets, then you know he’s not as good as he looks. You just can’t measure exactly how much unless you have loads of stats on it.
I don’t trust adj +/- to answer these things well because there so many examples that are so preposterously wrong it makes me not trust the ones that look right intuitively and based on box score. I look at the +/- numbers to reinforce what I think I already know, but not to change my mind about anything (if that makes sense).
tgt
February 3, 2011
@mystic,
Yes, the DRtg values are fact, but that Kevin Love is bad at defense is not.
Individual DRtg and +/- are wildly inconsistent. Do you think some players go from being one of the best in the league one year, to one of the worst the next, back to one of the best the following year?
You’re claiming an extremely flawed stat is The Word Of God. Maybe WP48 doesn’t match individual DRtg because individual DRtg is more random chance than reality.
mystic
February 3, 2011
tgt,
you didn’t read or understood my post well. I didn’t say anything about individual DRtg, I pointed out the RESULT, that’s what really happened on the court when those players where playing. The defense was horrible. Why is that? Maybe you can explain that?
Now, try to understand first what individual DRtg means, before you are talking about a “random chance”. ;)
EvanZ
February 3, 2011
On defense, I have Humphries #69 and Love rated #88 out of 125 (for players > 2000 posssessions). These are calculated using PBP data. Dave doesn’t let me post links to my blog, so you’ll have to take my word for it.
mystic
February 3, 2011
Italian Stallion,
synergy sports offers those stats you want to see about Kevin Love. And yes, he is cheating on defense. ;)
And you don’t need to trust anything at all, especially when you are trying to use just one metric to determine the value of a player.
Well, the role of a player might change, but overall a player who is elite according to a rating, should be able to contribute in a way that his team still improves. Why is that not the case with David Lee? Claiming that the coaches are stupid and give out the wrong minutes is most likely the worst answer you can give for that. Sometimes players don’t have the stamina to play that many minutes. Sometimes he isn’t working well together with other players on the court. If his skillset doesn’t mesh with the rest of the team, he might hurt his team more than he is helping. Thus more minutes to him will not improve the overall team play. Another point are matchups. The NBA is about exploiting matchup advantages, that is true since the 60’s and the Celtics. That’s what Red Auerbach used on those Celtics championship teams, that’s what Don Nelson learnt from him and used during his coaching career on the Bucks, Warriors or Mavericks. If a certain player has a matchup disadvantage on either offense or defense, he might hurt his team again. Unfortunately you can’t plug something like into just one number. I tried it since the late 90’s via multiple regressions, via simple linear models, via some non-linear models and I didn’t found that “holy grail”.
mystic
February 3, 2011
EvanZ,
thanks, both are below average. As I said, both aren’t good defenders at all. No surprise that their teams are doing worse with them on the court. It just surprises someone who thinks that WP48 actually can represent defensive strength on an individual player level.
tgt
February 3, 2011
@mystic
DRtg of individual players without accounting for who else is playing (on their team and the opposition) is completely useless. I assumed you were at least breaking it down like adjusted +/-. What you were doing is even worse than I thought. My point is only bolstered
mystic
February 3, 2011
tgt,
you still didn’t read carefully enough. Go back and try again. ;)
EvanZ
February 3, 2011
tgt, do you have in mind a metric that has Love as a “good defender”? If so, can you share it with us?
RAPM has him at -0.4. I have him at -0.18. Synergy has him at 0.9 PPP, which ranks #202.
So far, I can’t find any objective metric that says he is “good” on defense. But maybe you have some data that the rest of us don’t?
tgt
February 3, 2011
@mystic
I read it perfectly fine. You were comparing DRtg on and off the court. Pretty much useless for an individual player
@EvanZ
I didn’t ever say Love was good at defense. I was just pointing out that the stats mentioned to support his being bad at defense were worthless.
mystic
February 4, 2011
tgt,
no, you didn’t read fine. I used indeed defensive adjusted +/-. As I said go back and try again.
Complaining about the use of OnCourt ratings (again, we are talking about results which really happened in the reality!) while defending a metric like WP48 is somehow funny as hell. Not quite sure, but do you think Berri is adjusting his defensive factor for teammates or strength of opponents? Now, how useless is that to assume that the team is always playing defense like they are doing it in average with every set of players?
And dismissing the stated methods while providing NOTHING to disprove my statement is another big joke. Especially when it comes from someone who has obviously no clue what he is talking about.
Every possible metric shows that Love is a below average defender.
Italian Stallion
February 4, 2011
mystic
I look at Synergy Sports data on and off and it seems to me that Love’s defense isn’t nearly as bad as you are making out to be. Clearly, he’s not a good defender, but there are plenty of players ranked below him.
I think David Lee “has been” a solid addition to GS. It hasn’t been noticeable because GS has had a series of important injuries (including to Lee) . Last time I looked they were about .500 with Lee in the lineup despite the fact that he was playing hurt and very sub par for some of those games. In other games when he was playing Curry was out. IMHO, GS is underrated.
Aside from injury, the other reason that Lee hasn’t had as big an impact for GS is that Ellis is a chucker. He’s a talented chucker, but he’s a chucker. Lee has always been a highly efficient scorer. His touches are down and have been foolishly transferred to the less efficient Ellis. They would better off running pick and rolls etc.. with Lee/Curry or something like that.
The other way Lee tends to contribute value is with boards/assists.
He’s still contributing assists, but his rebounds are understandably down a tad playing next to Biedrins instead of guys like Harrington/Chandler/Jeffries etc…
Italian Stallion
February 4, 2011
mystic,
I should add one thing.
The Timberwolves are a dreadful defensive team. Last time I checked they were in the bottom few (27th?). So unless the entire poor defensive performance is Love’s fault (and it’s definitely not), I think we can be fairly certain that WP is not distorting Love’s value very much based on individual defense. If the whole team sucks defensively, then I assume that a portion of that suckiness would be applied to Love at the team adjustment level also.
Where WP could have a more serious problem is when there is one great defender on a terrible defensive team or vice versa. Then an individual could be carried or punished by his teammates.
EvanZ
February 4, 2011
IS, ezPM has the following for defense:
Brewer +2.04
Darko +1.93
Tolliver +1.20
Johnson +0.14
Love -0.16
Beasley -1.69
Webster -2.02
Ridnour -3.00
Telfair -4.19
Now, I’m not saying you have to believe those. It’s my model, so whatever, right? My point is that if you break down defense individually, as I do by using counterpart data, you can get a much larger variation than you would by “splitting it up” evenly. So…it “matters”, in the sense that you may be undervaluing or overvaluing players. (I might be, too, of course. The difference is that I understand and acknowledge that point explicitly.)
Italian Stallion
February 4, 2011
Mystic,
I acknowledge the issue and have even tried to encourage an improvement of some sort in the future. I also look at stats that try to capture individual defense. I just don’t trust them a lot because they seem to vary from year to year and even when players switch teams.
There are many things in this game I know I can’t value well, but I learned a way of coping with that from my experience with horse racing.
Sometimes it’s enough to simply rate things as “++”, “+”, “0”, “-“, “–” .
If what I know I can value suggests two players are equal, but in the things I can’t value I can at least determine that one is a + and the other a -, in some situations I already know enough.
mystic
February 4, 2011
Italian Stallion,
yes, Love looks better than he really is, because team defensive efforts aren’t that well captured by Synergy Sports. But anyway, he isn’t rotating very well and is leaving players open instead of closing out which results into a higher scoring efficiency for the other team. He still grabs rebounds, but his effect on the defensive end in terms of rebounding isn’t as big as his boxscore numbers are suggesting. The problem here is that in a couple of cases Love is just battleing with teammates for defensive rebounds. He gets the credit in the boxscore, while his teammate not.
Love should get really punished for that.
The excuses for the Warriors are lame. There is a reason why Lee doesn’t get as many touches, because he can’t create that offense as much for himself to give him more. It also is a fact that they are worse as a rebounding team with him than without him. His rebounding numbers have a similar flaw. Battling teammates for rebounds will not increase the team rebounding percentage.
Here is a recent analysis about rebounding: http://www.stats-for-the-nba.appspot.com/rebounding
As you can see the effect of the best rebounders in the league is mostly due to a higher chance on the offensive glass. Which also means they usually don’t get back on defense fast enough. The effect is also rather small in comparison to an average replacement player here.
I ran a regression and posted the results here: http://sonicscentral.com/apbrmetrics/viewtopic.php?t=2741
As you can see the correlation between actually boxscore derived rebounding values and those adjusted rebounding values isn’t huge. That means rebounding is also heavily depended on the overall team and not just basically a player’s effort. The basic assumption of WP48 that those values derived from the regression of team numbers can be used to determine the impact of a player is the big flaw here. Everyone who understands the game a bit knows that the overall team result is not just the sum of the parts. Five great rebounders for their position in terms of boxscore based Reb% aren’t making a better rebounding team per se.
And again, the whole point why I’m arguing here, is the myth someone wanted to spread that WP48 accounts for individual defense. It doesn’t do it, and it has no chance to do it either, because based on boxscore stats a correct weighting of the defensive impact isn’t possible. The informations just aren’t there.
It isn’t the first time that WP48 overrates rebounders heavily, we have the example of Marcus Camby, David Lee, Kevin Love and Kris Humphries. And we are not talking about a minor issue here, we are talking about players who are supposed to have superstar impact according to that rating while in reality that kind of impact can’t be found. No team will have a chance in the NBA, if they decide to build around Kris Humphries, his WP48 numbers suggest he is a piece to build around rather than being a role player on a lottery team.
tgt
February 4, 2011
@Mystic
So when I said you were using adjusting the data I was wrong, and when I said you were using unadjusted data, I was also wrong. That aside, you are still using statistics that vary wildly from year to year. The error margin is so large as to make those statistics worthless.
The stats are based on real results. That doesn’t mean the stats accurately measure contribution or ability. You can nitpick my exact explanation of your stats all you want, but they’re still +/- stats and still have the known issues. You cannot say they are reality by fiat.
tgt
February 4, 2011
@Mystic
“Anyway, I didn’t postulate an idea, I stated a FACT. I didn’t speculate, I know that this is true. But I can try it again:
When Kevin Love is on the court, the Minnesota Timberwolves have a DRtg of 112.1, that is basically as good as the WORST defense in the league right now (Cleveland Cavaliers with 112.3 DRtg). When he isn’t on the court, the Timberwolves are at 106.9 DRtg, which is basically equal to the league average.”
Clearly you did not mention individual DRtg at all. You also didn’t use a horrid stat and say it represented reality.
“See, the Timberwolves are playing worse defense with Love on the court than without him. Now you might argue that he has to play all the tough guys and the other time his teammates are only facing the bad players. That might be true, thus we can look at the defensive adjusted PM: http://stats-for-the-nba.appspot.com/ranking11
Kevin Love is listed with -1.2. He is actually making the defense worse in comparison to an average replacement player by 1.2 points per 100 possessions.
That is not an idea anymore, that is the reality.”
No mention of a horribly inconsistent stat as if it represents reality 100% accurrately. That definitely didn’t occur.
“The same thing for Humphries, 111.1 OnCourt DRtg, 4.6 points per 100 possessions than without him on the court. His defensive APM is -1.0. Again, the defense gets worse with him on the court and adjusted for the strength of teammates and opponents it ends up with a below average value (same sources).”
OnCourt DRtg. That’s just an unadjusted Individual DRtg. Clearly, you did not say this. I wonder where I quoted it from.
“Yeah, I know the arguments, it is noisy, it is … blabla. Matter of fact is in both cases the defense of their teams is horrible, while they can play decent defense when those players are not on the court. That is not an idea, that is the reality, Mike. You can ignore that and say something about blabla like you done that a lot in your previous post, but that will not change the reality.”
You say there’s noise, but then immediately ignore it. This is either intellectually dishonest or you don’t understand what happens with a noisy stat. In a noisy stat, it is likely for some good players to be listed bad and some bad players to be listed good. That’s the nature of noise. The reality is the stat is different based on who is on the court. As I said before, that doesn’t mean anything about the reality of their actual contributions to defense.
“no, you didn’t read fine. I used indeed defensive adjusted +/-. As I said go back and try again.”
So there’s adjusted data, and above that we see unadjusted data. So you used both, but claimed you used neither. If you don’t even know what you said, how can you keep telling me to read your post? I read your post. Your argument was horrible.
You said WP48 is bad because it doesn’t match reality, and then substituted in known horrible stats for reality. I told you that wasn’t much of an argument. You didn’t understand. You asked me to come up with better stats, instead of admitting the known flaws in yours. Wait, you did admit the flaws, you just have no idea what the flaws actually mean.
So, you don’t know what you said, and don’t understand the meanings of statistics, but claim I’m the ignorant one. If you refuse to listen to valid criticism, it isn’t worth my time to give it.
EvanZ
February 4, 2011
“Clearly you did not mention individual DRtg at all. You also didn’t use a horrid stat and say it represented reality.”
Right, he didn’t do either of those things. What are you reading? Because we’re clearly not seeing the same words. There was no mention of “individual”. You realize DRtg is a team stat, right?
“OnCourt DRtg. That’s just an unadjusted Individual DRtg. Clearly, you did not say this. I wonder where I quoted it from.”
No. It’s actually not. OnCourt DRtg is the team level +/- while that player is on the court. Adj. +/- brings it down to the player level.
Italian Stallion
February 4, 2011
Mystic,
Lee got plenty of touches for the Knicks in his last season there when he developed a mid range game despite his limitations for creating his own offense. He scored close to 20 points per 36 minutes at a high efficiency rate.
His efficiency is down this year because he got off to HORRIBLE start after injuring his arm in a Knicks game ironically. His TS% was below 50% for quite awhile, but it has been reverting to the mean since he’s gotten a little healthier.
He is getting fewer shots in GS because they have more options on offense than the Knicks had last year and because Ellis is a ball hog. Seriously, it’s that simple. I watch almost every GS game and have seen just about every Knicks game for the last 4-5 years .
IMHO every study ever done on rebounding is flawed. I think I’ve read just about every one of them. I don’t have a solution. IMO the value of a rebound or rebounding changes depending on the circumstances.
IMHO, Lee adds a lot of value with his rebounding even if some of them could have been captured by teammates.
I question the whole notion that one player can take a lot of rebounds away from his teammates anyway. I once charted about 200 games manually and it didn’t happen as often as some people think. The other thing I noticed was that there was an offset. Sometimes the supposed “rebound thief” was in a position to grab a rebound that a teammate got. So the “net steal” was typically not huge.
There are also things going on I can’t value.
If Lee/Love is the “rebounder”, then even if teammates “could have” gotten the rebound also, sometimes when they are not trying they are leaking out on breaks for easy baskets etc… and doing other things that have positive value that Lee and Love don’t get to do because they are rebounding.
Italian Stallion
February 4, 2011
Mystic
IMO even though there ARE diminishing returns for rebounds, it’s probably not large as some studies indicate. Teams generally don’t throw all good rebounders together because they understand that also. When a team has a surplus, the value of a rebound falls and each player gets fewer, but they still have a lot of value.
Everyone that is anti WP likes to harp on the value of a rebound being overrated (which I agree may be overstated), but no one seems to want to focus on the fact that scoring is very overrated by virtually everyone other than WP and perhaps Pelton. Pelton’s thinking most closely resembles my own because he adjusts efficiency slightly for usage and understands that the makeup of the team can also impact the adjustment.
tgt
February 4, 2011
@EvanZ
DRtg is a team stat. When you do DRtg when a particular player is on the floor, you are linking that DRtg to that player. Mystic was EXPLICITLY doing that by comparing it to DRtg when he was off the floor. If that’s not an attempt at individual DRtg, I take back my comment, and then lambaste him for quoting a team stat to talk about an individual player. That’s even worse.
mystic
February 4, 2011
tgt,
you just admitted that you couldn’t follow the post. Nice. Now it is even worse, because I used a team stat? Well, Berri will be happy that using a team stat to determine individual defense is even worse. Thanks for pointing that out. :)
@Italian Stallion
I can follow what you are saying, I don’t necessarily agree with everything, but I also don’t feel the need to discuss things further.
tgt
February 5, 2011
@mystic
I followed what you were saying. It’s not my fault your argument doesn’t hold any water and you can’t admit your own failings. Enjoy your begging the question, strawmanning, and general inability to understand what goes into a valid argument. I’m sure you’ll come up with extremely useful stats that way.
mystic
February 6, 2011
tgt,
the reality is proving me right. Didn’t you notice?
You can claim all you want, but in the end you didn’t show one tiny bit that the used metrics by me aren’t reflecting the reality.
And saying that you were able to follow while confusing several terms is a bit comical, but you may very well believe that you were able to follow.