My post on PERs a couple of days ago led to a few interesting comments. Let me respond with a few more observations about evaluating player productivity in Duck on a Rock – I mean basketball.
Evaluating the Average Player
Imagine a team had a player that was average in every respect. For his position, he was average in shooting efficiency, rebounding, creating steals and assists, blocking shots, and committing turnovers and personal fouls. If we looked at this player via Win Score or NBA Efficiency, we would see an average player.
Now we take that same player and we increase his shot attempts. He is still average in shooting efficiency and every other aspect of the game. But he is now taking more shots. Win Score still tells us that this player is average. NBA Efficiency, which over-emphasizes scoring, now tells us that this player is above average. In fact, a player can be below average in shooting efficiency, and if he takes enough shots, be considered above average in NBA Efficiency.
The problem for decisi0n-makers is that in terms of offensive and defensive efficiency, teams lose when shots do not go in. If you take ten shots at a below average rate you hurt your team. If you take twenty shots at a below average rate you hurt your team even more. But NBA Efficiency, and PERs – two metrics that set the break-even point on shot attempts at a very low rate – will tell you that the player is actually above average when he takes a large number of shots at a level of shooting efficiency below the league average. Again, I need to emphasize this observation. As long at the player exceeds the very low break-even points set by NBA Efficiency and PERs, the more shots he takes, the higher the player’s rating.
In fact, one can do nothing else but score – either at a below or above average rate of efficiency, and NBA Efficiency and PERs will tell you a player is great. And furthermore the NBA player’s market will likely give our one-dimensional scorer significant money. But our model of offensive and defensive efficiency tells us quite clearly that the one-dimensional inefficient scorer will not create many wins.
Creating Shots and Confusing How and Why
People will argue that there is value is creating a shot. And that is true, if the shots you create go in. If you are simply chucking the ball in the directions of the basket, though, you are not helping your team win games.
But isn’t it the case that the more you shoot the lower your efficiency, and the less you shoot the higher your efficiency? If this is true, a high volume shooter may be helping his teammates take fewer shots and become more efficient.
There are two answers to this question. First, as Marty detailed last June, there is very little evidence that shot attempts and shooting efficiency have a strong relationship. But let’s ignore that point.
Let’s say the argument is true, shooting efficiency and shot attempts are indeed linked. If it is true, shouldn’t we adjust our performance metrics to take this into account?
Those who think we should are confusing how productive a player is with why that player is productive (a point I made last June).
Consider baseball for a moment. Imagine a hitter with a 0.350 batting average. This is pretty good. What if I said he only has this batting average because his manager only has him face left-handed pitching, and if he faced right-handers he would bat 0.150? Does this fact change his 0.350 batting average? No, he did what he did. The information on who he faced allows us to understand why he has this batting average.
In making decisions you need to consider both questions. But to answer the “why” question you first must know “how.” And when people rely on PERs and NBA Efficiency to determine “how”, the picture of productivity they are using is flawed.
Over Valuing Rebounds, Again
Is our model an accurate picture of “how?” It has been said that our model clearly overvalues rebounds. A few days ago I wrote an essay addressing this argument. I would add to that story by referencing the wisdom of Milton Friedman.
In 1953 Friedman published Essays in Positive Economics. In this work Friedman argued that you cannot refute a model unless one has an alternative that provides better predictions. In Friedman’s words, “criticism of this type is largely beside the point unless supplemented by evidence that a hypothesis differing in one or another of these respects from the theory being criticised yields better predictions for as wide a range of phenomena.”
People have argued that our model overvalues rebounds because its conclusions are inconsistent with their prior beliefs. Specifically people believe Dennis Rodman is not a very good player. Because our model says he is quite productive they conclude our model must be incorrect. But prior beliefs are not evidence against our approach. Evidence against our approach would come in the form of an alternative model that offers a better explanation of the evidence. From this perspective, our model only over-values rebounds if another model with an alternative value for boards explains and predicts better. To date, I have not seen this alternative model.
I do see evidence, though, that whatever model the NBA employs has trouble explaining and predicting. Wins and payroll are not strongly linked in basketball despite the fact player performance is relatively consistent across time. Player salary is primarily determined by scoring, not other factors like shooting efficiency and turnovers. And the metrics people have offered – NBA Efficiency and PERs – confirm this bias towards scoring.
Taking Rebounds from Teammates
And to conclude this essay, let me address the question, do players take rebounds from their teammates? The answer is “absolutely.” Of course they also take shot attempts from their teammates. In fact, we show in The Wages of Wins that the more productive your teammates the less productive you will be. But people tend to exaggerate this effect. We found the interaction effect to exist but it appears to be rather small.
To see this point, consider the consistency in player performance across time. If your performance depended entirely on your teammates, player performance would fluctuate dramatically across a player’s career. This is indeed what we see in football. But in basketball we see much more consistency, suggesting the size of the interaction effects have been exaggerated.
– DJ
Wages of Wins Journal 
ZZ
November 19, 2006
This is a minor quibble, but I’ve never heard anybody say that Dennis Rodman was not a valuable player – even with his offensive shortcomings, his defensive and rebounding work were always well-recognized…
dberri
November 19, 2006
ZZ,
Read the King Kaufman column I cited a few days ago. Kaufman makes it pretty clear he thinks Rodman was below average as a player and he cites PERs to confirm his position.
dberri
November 19, 2006
By the way, I left more comments on this issues at my original PERs post.
Jordan Lichty
November 19, 2006
I completely agree with what you are saying about shooting, that taking a shot costs everyone else on your team the chance to shoot, so you better do so efficiently. You pretty much admit that rebounding is analogous: “do players take rebounds from their teammates? The answer is “absolutely.” Of course they also take shot attempts from their teammates.” But then why do you consider the average points per possession offensively to be the opportunity cost of using a possession, but not the average likelyhood of a defensive team (or offensive team) getting a rebound as the opportunity cost of getting the rebound. Isn’t the same logic applicable?
Using predictive power to justify you model is not valid here because you can change many of your coefficients and you will still get a model as predictive once you apply the team adjustment. Also, I have a model that is equally predictive, its called raw plus-minus, and its not that good although it predicts team wins better than yours. Point being that predicting team wins in and of itself doesn’t prove anything.
That said of course, I like your model but I don’t think its perfect yet, especially with regard to the rebounding question.
Jordan Lichty
November 19, 2006
One thing I forgot. I know for a fact that you came up with a model that was just as accurate at predicting wins but did not include blocks, assists or fouls. This effort passed the peer review process and led to a number of interesting results, but you went away from it at the urging of Dean Oliver. I assume you did this because it did not line up with conventional wisdom and you could not have sold many books with such perverse conclusions. This shows two things, one that the who peer reviewed your work don’t understand basketball nearly as well as you suggest they do, and two that you shouldn’t look down at Hollinger for bowing to conventional wisdom because you caved to it too.
Jason
November 19, 2006
“Jordan”, I don’t follow what you’re saying about the “opportunity cost” of getting a rebound. What exactly is this cost such that if the rebound isn’t secured something else can be gained? How does one realize a benefit from a rebounding opportunity without getting the rebound such that the rebound takes away from something else? “Jim Davis” was trying to make the same case but didn’t present a convincing argument then either. Perhaps you can elaborate more clearly here.
Jordan Lichty
November 19, 2006
Getting a rebound prevents a teammate from getting it in the same way that taking a shot prevents your teammate from taking it. Yes, it is true that someone has to get the rebound, but it is also true that someone has to take the shots. If you only get credit for scoring efficiency above average, then you should only get credit for rebounding efficiency above average. In the case of rebounding this would be 1 (you got the rebound) minus the average chance of a team getting, just like with scoring where you get whatever you scored, say 2 points minus 1 (the per possession average). I think this analogy is pretty clear, but if you don’t want to believe it you don’t have to. I can’t convince you of something you have decided you don’t like, and that is absolutely your prerogative.
Jason
November 19, 2006
“Jordan”, it appears to me that you are confounding the value of a rebound to a team with the value credited to an individual. Getting a rebound may deny a teammate the opportunity to grab said rebound, but it provides the same value to the team regardless of who grabs the rebound. There was never an opportunity for another player on the same team to do *better* than this and consequently, there’s no opportunity that the *team* has sacrificed. As far as the team is concerned (and this is where the value of the rebound is realized and thus where it should be calculated) it is a binary situation.
If your argument (whoever you are, and I’m presuming that Jim/Jordan are one in the same as you appear to take the same tone and position) is about the value of the actual rebound to an individual, this may not be the case, but that’s not where the value as a rebound relates to wins and losses is realized. It is realized on the level of the team. The team either gets it or doesn’t. I don’t see a need to subtract an ‘opportunity cost’ if there’s no *other* opportunity. Perhaps you do not expresss yourself well and you are quibbling about what that value is, but as far as the team is concerned, there are only two possibilities: get the rebound or do not get the rebound. That the rebound is worth “1” seems to be a point that you’re taking and that it needs adjustment from this point rather than compute the actual value of a rebound as it relates to the probability of a win and loss and use this value unadulterated.
I realize *you* believe that the analogy is clear, but your assertion notwithstanding, There is a difference between shots and rebounds. The competition for taking shots in a possession is between members of the same team. If a player takes a shot, a teammate cannot, but someone on the team will (turnovers and fouls withstanding). There are also different values a team can realize from a shot taken. It can be made, it can result in one or two points and it can also result in prolonging the possession if a foul accompanies the basket. Consequently, I can see how there’s an opportunity cost in taking a shot even when making one as a 2 point shot means that the team didn’t score three and the opportunity to do so is gone.
However, this is not the same case with rebounds. There is only one value a *team* can realize from a rebound. And importantly, rebounds are competitions *between* teams. No one on a particular team has to get the rebound. Unlike a shot where someone else on the team will take it in most circumstances, there is a real possibililty that the other team will get it, thus no value will be realized. The result is binary, again not holding to the analogy of a shot taken where the result is not binary.
I can’t convince you that you are not making a good case, but that is *your* prerogative. However, since you say that you have a model (raw plus-minus apparently, though I’ve been unable to find such a model credited to a “Jordan Lichty” anywhere suggesting it is either a secret model else you are someone else intent on hiding your identity while still trying to boast of your credentials in the argument) it does appear that you have a vested interest in trying to make your ideas known and understood. Writing off your impass as “something [I] don’t like” does yourself a disservice.
Jordan Lichty
November 19, 2006
I have to say I agree with you that we are quibbling about the value of a rebound to a player or to a team. I think Berri’s system is an excellent team evaluation system and I have never heard anyone disagree, you really can’t because it is quite good as far as teams are concerned. Team wins are certainly a function of offensive efficiency and defensive efficiency and Wins Produced accurately reflects this. My issue is with applying this team valuation model to individuals. The model was regressed at the team level and my point is that in terms of rebounding it does not fit for individuals. I think you see this, but maybe I didn’t clarify that it was at the indiviudal level that I was making my point.
In terms of raw plus-minus my point was that if you take the sum of every player on a team’s raw plus minus you will have that team’s point differential (or offensive and defensive efficiency if you prefer) from which you can predict win-loss record as well as any known system (as Wins Produced is also based on the idea that offensive and defensive efficiency predict wins). My point was simply that if raw plus-minus which we all agree to be flawed (some good reasons why are offered in the book) has the same ability to predict team wins then maybe this not the best or only measuring stick of a individual player rating system.
Harold Almonte
November 20, 2006
I would like to know, if we have a mechanical device skilled to shoot to a basket, the most attempts then less efficiency, and vicecersa? There´s no a machine performance standard?…Then if you add a circuit to make decisions with rules about good and bad shots and give some level of “green light”, what would be the results? Just take better shots and the same machine standard? A trouble with the program circuit and then some bad shots? –
I´m allways thinked the value of a rebound is relative to a pre defensive action and a post scoring, and so its point and win value. As shared as an assist. But I think to rebound above position average is a skill, and some positions/players have more chance to do it, but I think a lot of rebounders are also the best defenders, and that tend to believe they are win-overrated against scorers. –
I believe some players depens more on their teammates than others, because they don´t domain the isolated skills like others, or just don´t domain any skill at all and are a sum of half skills, and will be rated with different perception (useful-neccesary-productive-determinant) at win-ratings,on/off, or another point rating.
dberri
November 20, 2006
Jordan,
Where to begin? First of all, you misunderstand what is meant by “predictive” power. It is not simply a matter of connecting your measure to current team wins, but also your ability to forecast future player productivity. From what I understand, raw plus-minus does not forecast future player performance very well, which is but one more problem with this approach.
Second, to argue that Wins Produced is good for the team but not for the individual player simply reflects your belief that the interaction effects between the players is very large. And the evidence of this is very weak.
I also find myself in a position of not only defending my work and the work of my co-authors, but also the peer review process in economics. Let’s begin with the “fact” that you claim to “know.” Through numerous conversations with Dean Oliver I came to modify the model I published in 1999 in The Managerial Decision and Economics. The resulting modification was published in an article I published with Anthony Krautmann in Economic Inquiry. Dean did not have anything to do with what happened after the model in the Economic Inquiry paper was created. So apparently you have been very much misinformed.
I would add that although the Economic Inquiry model omitted assists, personal fouls, and blocked shots, I would hardly call it “perverse.” Is OPS a “perverse” measure of a baseball player’s performance because it omits stolen bases, sacrifices, and defense? The question you have to ask is whether including assists, blocked shots, and personal fouls would have changed our conclusions. I will save you the trouble, it does not.
In a few days I will try and post a longer essay defending my fellow sports economists. Hopefully this essay will dispel some of the silliness I have seen people offer about work on our field.
I would conclude by briefly noting that although the model in The Wages of Wins is a modification of what was in Economic Inquiry, this is not a reflection of me bowing to “conventional wisdom.” Rather it simply reflects further thinking on the subject.
RD
November 20, 2006
The meaning of shooting efficiency for players has to be interpreted in the context of the particular team. Does the team have a surplus of quality usage ability, i.e. shots at a reasonable eFG% or TS%, a bare adequacy or a shortage? On some teams you can rationalize, perhaps somewhat forgive fairly low shooting percentage of a high volume shooting star if you assert/prove the rest of the team doesnt not have adequate usage ability. Maybe this was the case for peak Iverson years of 76ers. The same is not necessarily the case for all high volume low shooting percentage stars. It depends on the ability of the rest of the team to handle a larger share of the usage as well or better. The case for ability to handle more shots is difficult and will probably remain in the realm of differences of opinion. Play where the star is out of the lineup may provide some data but also has issues (shortterm elevated play effect, different look perhaps harder for opponent to know what to expect) and play on other teams other years can’t be easily or fully proven applicable to the specific situation either. If you could isolate different periods of coaching direction for a team – when a star got play calls enough to produce 25 shots a game compared to 20 or 15 and other players were directed to shoot more and you judge individualk and overall team shooting efficiency under these scenarios on the same team with the same players in the same year then you could comment on the issue. For that team, situation. But every team will be different, no one answer fits all on this question in my view. This is an area where coaches and GMs have to bring their experience / judgment. I doubt the numbers will ever be enough to confidently answer this and make it automatic. Opponent defensive respponse is another factor not discussed. Game to game star and rest of team defensive emphasis will vary so results from every game can’t be treated generically. You have to watch or rewatch and sor t the games into different piles and evaluate who star and rest of team did in the face of different defensive strategies.
RockDuck
November 20, 2006
In many cases I think there is a surplus of usage ability on a team over a certain minimally acceptable level . The coach has some flexibility to divide up the shots at least in his mind and with the play calls but not only will the defensive strategy vary and affect results but some plays won’t quite produce the expected shot opportunity or may present a surprise opportunity. Players don’t always run plays to the desired shot option. The ability to see and exploit small advantages when a defender is a half step slow or wrong will vary by player. The stars think they see that more. They have more experience seeing that, but they all arent as good as the amount of shots they take. Some of them over do it. For the money and the fame that goes with “being the man”, for ego and / or low trust levels in teammates.