This week Malcolm Gladwell, author of the best selling books Blink and The Tipping Point, wrote a very positive review of The Wages of Wins in The New Yorker. Yesterday he offered a few extra comments, entitled Basketball By the Numbers, on his website.

Good journalists, and Gladwell is one of the best, are able to identify exactly the point people are making in an argument. In his comments, Gladwell takes what we say with respect to the NBA and identifies two specific questions that capture perfectly our basic story.

**“…for those aspects of basketball performance that are quantifiable (steals, turnovers, rebounds, shots made and missed, free throws etc) are the existing statistical measures we use to rate players any good? And if not, is there a better way to quantify the quantifiable?”**

In answering the first question Gladwell notes:

**“To the first question, “Wages of Wins” argues—convincingly—no. For instance, they show that the correlation between a team’s payroll and a team’s performance, in the NBA, is surprisingly weak. What that tells us is that the people charged with evaluating and rewarding ability and performance in the NBA do a lousy job. In particular, they argue, traditional talent evaluation over-rates the importance of points scored, and under-rates the importance of turnovers, rebounds and scoring percentage. Wages of Wins also obliterates the so-called NBA Efficiency rating, which is the official algorithm used by the league and many basketball experts to rank the statistical performance of players. The Efficiency rating, they argue, makes the same error. It dramatically over-rewards players who take lots and lots of shots.”**

The answer to the first question highlights the primary point our research into the NBA reveals. Whether you like our measure of performance or not, our most important finding does not depend upon how we measure productivity in the NBA. Our research into the relationship between payroll and wins, salary and player performance, and voting for the All-Rookie team, all tell the same story. If winning is the objective, then players are not being evaluated correctly by the NBA. I would add, if maximizing profit is the objective the story is the same.

How about the second question. Here is Gladwell’s answer.

**“Okay: part two. Is the Wages of Wins algorithm an improvement over the things like the NBA Efficiency system? To make the case for their system, the authors “fit” their algorithm to the real world. For the 2003-04 season, they add up the number of wins predicted by their algorithm for every player on every team, and compare that number to the team’s actual win total. Their average error? 1.67 wins. In other words, if you give them the statistics for every player on a given team, they can tell you how many wins that team got that season, with a margin of error under two wins. That’s pretty good.”**

The accuracy story is the same if we expand our sample. If we look at the past 10 seasons the average error is 2.3 wins. Again, our methods link wins to all the statistics the NBA tracks for its players. And these statistics do allow us to measure quite accurately the number of wins each player produces.

A few comments around the blogosphere have questioned our methods because a specific player is not ranked as high or as low as someone might like. This is of course not the appropriate method for evaluating any performance metric. And given the answer to the first question Gladwell poses, this is clearly not a good response with respect to the NBA. Our first point – and again, the most important point we make — is that how players are evaluated by the NBA is incorrect. Again, let’s review the evidence.

- Payroll only explains 12% of team wins.
- Player pay in the NBA is primarily determined by scoring totals.
- Various other decisions are also primarily determined by scoring totals.
- In sum, NBA decision-makers over-value scoring and this is why player pay has such a weak connection to team wins.

Given these results, is it really that surprising to learn that when one does estimate how many wins a player produces the results would contradict what is commonly believed? The common belief, which must be close to what the NBA believes, leads to payrolls that only explain 12% of wins. Or in other words, payrolls cannot explain 88% of team wins. So is everyone still sure that their favorite scorer is contributing that much to team wins?

— DJ

*Basketball Stories*

Blar

May 31, 2006

Hi DJ,

I’m here via Gladwell’s blog, and I was there via Matt Yglesias. I left a similar comment in both of those places, so if you’ve been reading those comments then you probably already know much what I’m going to say, but I’d like to hear your response.

It’s not clear to me that Wins Produced does a good job of disentangling teammates’ contributions to the team’s success, and your method of validating the measure, adding up the players’ Wins Produced to see if they’re close to the team’s actual win total, does not seem like an effective way to test that it does. On offense, the whole team is working together to score. If they succeed in breaking down the defense and getting high percentage shots then the credit should not necessarily go to the guy who scores (along with the guy who gets the assist), and if they end up settling for tough shots the blame does not necessarily belong solely with the person who ends up taking those low percentage shots. If your statistic apportions credit and blame in that way, as Wins Produced does, then of course you are giving responsibility to the right

team, so it’s no surprise that your statistic will closely approximate team wins, but that doesn’t mean that you’re crediting the rightplayers. You’re mostly using these statistics to talk about players – who’s overrated, who’s underrated, who should be MVP, who should be on the All-NBA teams, etc. – but it’s not clear how reliable Wins Produced is for evaluating players.This hypothetical might make the problem easier to understand. Suppose that you removed assists from your formula and then re-calculated everything. I bet that assists-free Wins Produced would be about as good as Wins Produced at predicting a team’s record, but if you used this new statistic to evaluate players then some players’ scores would be very different. There are lots of things that players do besides getting assists that help their teammates score efficiently (or make it harder for their teammates to score efficiently) that are not incorporated into your Wins Produced statistic (attracting double teams is an obvious one). Players that are good at those things are going to be underrated in Wins Produced, just as players like Kidd and Nash would be underrated if we took assists out of the picture, and players whose offensive statistics benefit from their teammates unmeasured abilities (or are harmed by them) would be overrated (or underrated).

(The reason that I picked assists for this hypothetical is that it’s one statistic that does not factor directly into a team’s offensive and defensive points per possession. If you started removing stats like turnovers that result in a change of possession or stats like field goals that result in points, then I assume that your altered Wins Produced statistic would become less reliable at predicting team wins.)

Fwd Credit

October 4, 2006

Nice and simple article to read, instead of reading

all that crap which is floating about on blogs.

Must admit it’s saved in my favorites……