Malcolm Gladwell Speaks Again

Posted on May 28, 2006 by


This week Malcolm Gladwell, author of the best selling books Blink and The Tipping Point, wrote a very positive review of The Wages of Wins in The New Yorker.  Yesterday he offered a few extra comments, entitled Basketball By the Numbers, on his website. 

Good journalists, and Gladwell is one of the best, are able to identify exactly the point people are making in an argument.  In his comments, Gladwell takes what we say with respect to the NBA and identifies two specific questions that capture perfectly our basic story.

“…for those aspects of basketball performance that are quantifiable (steals, turnovers, rebounds, shots made and missed, free throws etc) are the existing statistical measures we use to rate players any good? And if not, is there a better way to quantify the quantifiable?”

In answering the first question Gladwell notes:

“To the first question, “Wages of Wins” argues—convincingly—no. For instance, they show that the correlation between a team’s payroll and a team’s performance, in the NBA, is surprisingly weak. What that tells us is that the people charged with evaluating and rewarding ability and performance in the NBA do a lousy job. In particular, they argue, traditional talent evaluation over-rates the importance of points scored, and under-rates the importance of turnovers, rebounds and scoring percentage. Wages of Wins also obliterates the so-called NBA Efficiency rating, which is the official algorithm used by the league and many basketball experts to rank the statistical performance of players. The Efficiency rating, they argue, makes the same error. It dramatically over-rewards players who take lots and lots of shots.”

The answer to the first question highlights the primary point our research into the NBA reveals.  Whether you like our measure of performance or not, our most important finding does not depend upon how we measure productivity in the NBA.  Our research into the relationship between payroll and wins, salary and player performance, and voting for the All-Rookie team, all tell the same story.  If winning is the objective, then players are not being evaluated correctly by the NBA.  I would add, if maximizing profit is the objective the story is the same. 

How about the second question.  Here is Gladwell’s answer.

“Okay: part two. Is the Wages of Wins algorithm an improvement over the things like the NBA Efficiency system? To make the case for their system, the authors “fit” their algorithm to the real world. For the 2003-04 season, they add up the number of wins predicted by their algorithm for every player on every team, and compare that number to the team’s actual win total. Their average error? 1.67 wins. In other words, if you give them the statistics for every player on a given team, they can tell you how many wins that team got that season, with a margin of error under two wins. That’s pretty good.”

The accuracy story is the same if we expand our sample.  If we look at the past 10 seasons the average error is 2.3 wins.  Again, our methods link wins to all the statistics the NBA tracks for its players.  And these statistics do allow us to measure quite accurately the number of wins each player produces.

A few comments around the blogosphere have questioned our methods because a specific player is not ranked as high or as low as someone might like.  This is of course not the appropriate method for evaluating any performance metric.  And given the answer to the first question Gladwell poses, this is clearly not a good response with respect to the NBA.  Our first point – and again, the most important point we make — is that how players are evaluated by the NBA is incorrect.  Again, let’s review the evidence.

  • Payroll only explains 12% of team wins. 
  • Player pay in the NBA is primarily determined by scoring totals. 
  • Various other decisions are also primarily determined by scoring totals.
  • In sum, NBA decision-makers over-value scoring and this is why player pay has such a weak connection to team wins.

Given these results, is it really that surprising to learn that when one does estimate how many wins a player produces the results would contradict what is commonly believed?  The common belief, which must be close to what the NBA believes, leads to payrolls that only explain 12% of wins.  Or in other words, payrolls cannot explain 88% of team wins.  So is everyone still sure that their favorite scorer is contributing that much to team wins?

— DJ