There is much debate about how to measure the productivity of an NBA player. There is NBA Efficiency, the Player Efficiency Rating, Win Score, Wins Produced, Plus-Minus, Adjusted Plus-Minus, and many more. One might wonder (at least I do) with all these choices, how do the sports media evaluate talent?
Consider voting for the Sixth Man of the Year (which is the award I was thinking about last night). For the 2007 award, 127 members of the sports media considered every NBA player who failed to start half their team’s contests in 2006-07. Each sports writer chose three players, with their first choice earning five points, second place earning three points, and their third choice receiving a point. When the voting points were counted, three players – Leandro Barbosa, Manu Ginobili, and Jerry Stackhouse – captured 92% of the available points.
What do these three players have in common? Each was the leading scorer off the bench for one of the three best teams in the regular season.
Aju Fenn (economist at Colorado College) and I have a paper which we presented at a conference some time ago. The paper needs to be revised and updated, but the basic story I think is pretty solid. In an examination of the voting for the NBA’s Most Valuable Player we found two factors to dominate – points scored and team winning percentage.
What this tells me is that the sports media does not consider sophisticated – or even unsophisticated – models of player performance. Sports writers tend to consider the leading scorers on the best teams. If you can score (or in the case of Steve Nash, create assists that lead to scoring) and your team wins, sports writers believe you are great. I’m pretty sure you don’t have to be an advanced statistician to know that the media is leaving out a bit of information in their evaluation of talent.
The winner in 2007 was Barbosa, who also was the leading scorer among all NBA reserves. When we turn to Wins Produced, though, we see that – for players who received any votes — Barbosa was only the seventh most productive player.
Table One: Evaluating the Top Sixth Men
The top sixth man in terms of Wins Produced was Manu Ginobili. He finished the year with 14.1 Wins Produced, and per 48 minutes [WP48], had a mark of 0.330. Another good choice was David Lee, who produced 13.6 wins and had a WP48 of 0.378. Despite Lee’s performance, 111 out of 127 voters did not place him in the top three. With the acquisition of Zach Randolph it does look like voters will get another chance to vote for Lee next year. One suspects, though, that the Knicks will not improve enough – and Lee will certainly not score enough – for him to crack the top three in 2007-08.
Looking past Ginobili and Lee we see that the next four players on the list – Josh Childress, Corey Maggette, Jose Calderon, and Antonio McDyess – either failed to score ten points per game or failed to play for a winning team. Again, scoring and winning appear to be the keys.
Of course these four should be happy that a few sportswriters actually placed their name in the top three. Yes, more than 100 of the 127 voters completely ignored the quartet, but a few writers thought something of these four. This can not be said by Dikembe Mutombo, James Posey, Brent Barry, Anderson Varejao, and Rajon Rondo. These five players also produced more wins than Barbosa. But none averaged double figures in points, and hence none received any votes at all.
What lesson do we learn from our study of the Sixth Man award? People have spent a substantial effort developing measures of performance for basketball players. Despite a plethora of measures to consider, though, the sportswriters – who are supposed to be experts – seem to be inordinately influenced by only two factors: Points scored per game and team winning percentage. Consequently, a player like Jerry Stackhouse – who has never been a productive pro (a point I made a few days ago) – finished third in voting for this award. In fact, 80% of voters thought Stackhouse was at least one of the top three reserves in the league. That’s right, 80%.
Why the focus on scoring?
The stat I most often see discussed in baseball is batting average. It is reported during every game and I would bet it is the one stat reporters most often turn to when discussing a hitter. Despite decades of sabermetricians noting the inadequacies of batting average, other stats – like OPS – just haven’t caught on with most writers.
And I think there is a good explanation for the persistence of batting average. Most casual fans know this measure, and the casual fan makes up the majority of any sports writer’s audience. In other words, sports writers are simply responding to what the majority of their audience knows.
My sense is that basketball fans are much the same as fans of baseball. The majority are casual fans whose attention is focused on the scorers. Consequently, despite the efforts of all the people who look at basketball statistics, the sports media will continue to focus primarily on points scored per game. That is what their audience focuses upon, and consequently we can expect in the future points per game and winning will still be the primary factors sports writers consider in making player evaluations.
– DJ
Wages of Wins Journal 
Matthew
July 25, 2007
I don’t believe sports writers use points per game in deciding who to vote for in various rewards. I read many of their columns during MVP selection and stats, of any kind, are rarely mentioned. Subconsciously I would guess it players a role, though.
Also, the other metrics you pointed out haven’t earned much respect. Why haven’t I seen a predictive model for PER yet? What use does a metric have if it can’t make predictions, only “evaluate” the past?
Also, it seems that too many statisticians spend, and in my opinion waste, their time simply trying to rank players. Virtually every one of your columns are about ranking players, and this is true of many others. This is why there aren’t nearly as many team metrics as player metrics.
Jason
July 25, 2007
I think the persistence of focus on scoring and scoring average in basketball is significantly more entrenched than batting average in baseball. Moneyball has drawn some reasonable attention to the value of some statistical measures over other.
Batting average is already an abstraction about performance that influences game results in a less than direct way–hits help score runs but not hits result in runs– rather than a direct measure of the actual objective (scoring more runs than the opposition). Replacing it with a better abstraction/proxy that correlates with winning more precisely isn’t necessarily easy, but moving from scoring average to another measure means partially abandoning that direct measure that matters in the final score–points scored–in favor of an abstraction. This abstraction includes more and factors in *how* the points were scored and how ths influences what other players on both teams also scored, but it’s an abstraction. It’s a greater leapto accept that a proxy for winning probabilities in basketball is more valuable than the currency that decides a contest on the scoreboard than it is to move from one efficiency measure to another.
The gut reaction for scoring average probably has historical roots, but points in a basketball matter. It’s just that *only* looking at average ignores that some ways of getting points take away from the possibility of others scoring and says nothing about how the other team scored.
I could add something to the Bryant/Lee debate here, because it seems relevant. As much as I want to subscribed to the notion that Lee could be more valuable, there’s something that just doesn’t sit right, though I can’t come up with much other than subjective responses, and suspect it’s because I still have some ‘scoring bias’ built in to what is visible when I watch a game. Points are recorded. We keep track of them because they decide games and there’s an instinct (right or wrong) that this must make the scorer more valuable.
But to Lee and Kobe specifically: Both were very valuable to their teams; the Knicks were outscored with every other player but Lee on the court last year. The Lakers weren’t quite so one sided, but they were also significantly better with Kobe on the floor. Lee’s Knicks had a poorer record in games he missed. Kobe’s games missed hardly constitute a statistically significant sample, but the 3-2 without him is *suggestive* that the scoring prowess alone *can* be replaced more easily than whatever Lee did for his team.
But as to swapping the players based on a number: I guess at some point, two questions come to mind. 1) is there a diminishing return in wp48 where we cannot really know if a .3something will necessarily remain better than the .2something if their environments were suddenly reversed. I’m not sure. It’s an empirical question difficult to control for.
The second question is tough as well: are particular aspects of the wins produced formula that are more or less influenced by teammates (or, conversely influence others more). To beat that dead horse, Troy Murphy was, in my opinion, immensely overvalued by the model while at GS. Despite his stats, the team fell behind when he was on the court and played as well or better in games he missed. He moved teams, his numbers dropped, he didn’t help Indy as much as he was predicted to and his loss was easily compensated for by the Warriors indicating to me that he was credited with things he wasn’t really solely responsible for and didn’t suffer penalties for errors he made. My impression of Murphy’s WP figure was based in his rebounds (something that required help from others to create the missed shots) and didn’t properly penalize him for porous defense.
Lee, I suspect wouldn’t be this sort of player and the +/- and record without supports this. Perhaps it’s that he does multiple ‘good’ things, hitting his shots (Murphy was a substandard shooter for a ‘4’) as well as rebounding and I cannot imagine he’s be the defensive problem that Murphy is–few players are. Consequently, I cannot see that he’d be fooling the model completely.
I can see him not being *quite* as ‘dominant’ as WP suggests he is. On a team with another front court rebounder (Eddy Curry–‘nuf said) I could easily see him losing one rebound a game in his minutes played. That doesn’t seem implausible at all. That’s still a very good total and still well, well, well above average per minute rate. And were he to have had 58 fewer rebounds (1 less a game) he’d *still* have a very, very good WP48. Everything else equal, he’d have to have had 3+ fewer rebounds a game (far less plausible, even if Curry suddenly became Dwight Howard) to drop down to a WP48 nearer to what Kobe had last year.
dberri
July 25, 2007
Jason,
Your comment is exactly the kind of thought process a team would have to go through in considering this trade. My sense is people look at WP48, or any measure, and think that is the final word on the subject of player evaluation. But as we said in the book, the numbers are where you start your evaluation. After you have the numbers, and you know what those numbers mean, you then have to think about what caused those numbers to be what they are. I agree that Lee is helped in New York by Curry, but not in the way people at Knickerblogger were suggesting. Because Curry is so unproductive, it allows Lee to be even better (which is what you said).
Mr. Parker
July 25, 2007
Every time someone mentions David Lee on
this website I am going to lobby for the Lakers
to trade for him. Isiah will be more than happy
to take Lamar Odom off the Lakers hand. Im sure
he would even throw in Balkman.
Jason
July 25, 2007
I was actually a bit surprised at how much a teammate would have to cut into Lee’s rebounds to take him down significantly. I actually expected to see a bigger dip if he had one fewer rebound a game (which I suspect is well within the “could be his teammates” zone).
I’ve tried really hard to find a reason why Lee isn’t elite level valuable, but it’s tough. The more you look at the numbers, the harder it is to find fault in him. Unless he’s a zero in many unreported areas, he’s really that good, though if such a case were true, you’d expect his +/- to show it. That this measure also indicates that he’s very, very good suggests to me that he’s very, very good.
As a tool, this is my present thinking on wins produced:
1) The stats have to be there. That seems pretty clear, else there wouldn’t be such a nice relationship between the stats and team record.
However, I’m not sure if we can say that a particular player produced X wins based on the stats, but that player’s *stats* were worth X wins. This may not seem like an important distinction, but I believe it is. Basketball is a team game and the other 9 guys on the court influence any particular players stats. *How much* they influence it though is a debatable point. [I’d say it’s *the* debatable point for anyone but the dedicated statistical nay-sayers, who do this more reflexively than with any actual reasoning. Those that say that basketball is just not a game that stats can tell you much about would have to conclude that far too much of the stats are dependent on others. I doubt very much that there are people who *really* don’t use stats. I’ve seen people who dismiss statistical analyses present someone’s ppg as reason why said player is good.] Empirically though, there are good players, good in multiple environments, with differing teammates, and they tend to have good stats suggesting that the player earns some share of his stats by what he does himself. This means:
2) some portion of the statistical contribution towards the wins is the result of what a particular player does.
Of course this portion can be anywhere from 100% down to almost nothing and it does not need to be a constant between players or across time for a particular player. This boils down to a problem similar to the ‘nature vs. nurture’ debate in biology. How much of an organism’s phenotype is a result of their inherent nature (usually meaning genetic) and how much is a result of environment. A calculation in quantitative genetics would be the ‘heritability’ of a trait. The heritability of WP48 would be how much we can, at minimum, expect to carry over onto a new team.
Unfortunately, this isn’t something that’s easy to do in biology and it’s not likely to be any easier in sports. It becomes something of an art.
dberri
July 25, 2007
Jason,
One thing lacking in the analysis of NBA numbers is perspective. I think it helps to think about what we see in football and baseball. In both of those sports, player performance is much less consistent across time. A quarterback in the NFL can see his performance change tremendously from season to season. A pitcher in baseball also tends to be quite inconsistent. Such inconsistency suggests to me that their stats are dependent on their teammates.
You see the same argument made in the NBA — performance is dependent on teammates — but we don’t see the same level of inconsistency. Jerry Stackhouse has been a below average player every year of his career. Shaq, Kevin Garnett, Jason Kidd, etc.. have always been above average players. These player play with different teammates. Yet their performance, in terms of “good” or “bad”, is basically the same.
My own research shows that performance is influenced by teammates. The law of diminishing returns does exist. But the effect seems rather small. Teammates are not enough to take a great player and make him bad. And they are not enough to make a bad player into someone great.
All that being said, a team does have to think about the numbers before making a decision in the NBA. It does seem to me, though, the amount of thinking in the NBA is much less than what has to happen in football or baseball.
mt
July 25, 2007
I’d like to see something about differences between scorers and role players. Is a scorer less likely to be above average? Are there more scorers or role players in the NBA? What is a good balance between the two for a team?
mt
July 25, 2007
This might have some bearing on Kobe vs david Lee
Jonathan Leaf
July 26, 2007
As poor a metric as batting average is, at least it is a metric of how well the act of hitting is performed. The number of hits a player has in a game or season in baseball is roughly analogous to points scored by a basketball player. But have you ever heard of a baseball fan saying a reserve was a good hitter becuase he had nine hits? Of course not. Yet basketball fans actually quote points scored as a meaningful measure of performance irrespective of shooting percentage, turnovers, etc.
So actually analysis of basketball is at a much more prmitive level relative to analysis of baseball than even you suggest.
Jonathan Leaf
July 26, 2007
As poor a metric as batting average is, at least it is a metric of how well the act of hitting is performed. The number of hits a player has in a game or season in baseball is roughly analogous to points scored by a basketball player. But have you ever heard of a baseball fan saying a reserve was a good hitter becuase he had nine hits? Of course not. Yet basketball fans actually quote points scored as a meaningful measure of performance irrespective of a player’s shooting percentage, his number of turnovers, etc.
So, in fact, analysis of basketball is at a much more primitive level relative to analysis of baseball than even you suggest.
Nate
July 26, 2007
One big reason that Lee was so ignored for the Sixth Man award was that lots of writers viewed Thomas’ bringing him off the bench–but giving him starter minutes–as a transparent gimmick to make Lee eligible for the award. I remember reading a couple columns mentioning that very thing, and thus dismissing Lee from consideration.
Vinny
July 31, 2007
WINS are the Way of the Wizzard DJ!
You are right, the media will not get it, nor will those whose dollars are invested in building championship teams — the owners and executives.
To this ongoing question of “NBA talent evaluation”, I have to (yes again — NOT by popular demand) reintroduce the “Position Pairing” model which suggest that the game of basketball is designed to favor wining with “guard/center” position pairs. And, win championship teams emerge without guard/center pairs topping the PAWS ranking, a truly exceptional player is on that team, because they literally “change the game.” My earlier posts on this “Position Pairing” model have counter Larry Bird, Michael Jordan, Stockton & Malone, Dr. J, and Oscar Robertson among these non guard/center PAWS leading players. To this list, we can also consider winning non-championship team players like Barkley, Rodman, Dominique Wilkins, Lenny Wilkins (not sure if he won a championship)), and Elgin Baylor.
I also, noted that the “hardwired pattern” of basketball favoring guard/center (outside/inside) “Position Pairing”
was useful for noting why “star” player cominations were not producing WINS (e.g., Carter & Kid; Iverson & Anthony, etc.).
My expectation of the wisdom of the “Position P
model led me to predict a Garnet trade to the “team formerly known as Lakers” — as the only center capable of productively pairing with Kobe. This would have been a trade to save the “team formerly known as Lakers” — and it did not get done (Jerry West would have gotten the deal done).
NOW we learn about the Garnet to the Celtics deal, and again the wisdom of “Position Pairings” reveals itself in the Boston braintrust, who know all too well about WINS, the pattern of the game, and the need to “pair” top guard Ray Allen with the “best” future center in the league — Garnett. Celtics know the way of the wizzard … and leprecon :) … and the “team formerly known as Lakers” do not :( [Kobe languishes]
BACK TO THE MEDIA AND NBA TALENT … the consideration of “game changing” players who WIN outside of
the “guard/center” pattern design is lacking, as well as the essential need to “pair” productive guards with centers to WIN within the “game design.”
So my (NOT back by popular demand) “Position Pairing” model will be tested by the Celtics this coming NBA season — and I predict success … by design!
Mr. Parker
August 3, 2007
Vinny,
Garnett a center?
I guess I can see where you are coming from.
After all you need a guy to rebound the misses
on defense and a guy to score.
Hence center/guard pairings are good.
Do you not consider Camby a good center?
Denver screwed up by bringing the tantalizing yet
unproductive Iverson in the door.