It has been over five weeks since Malcolm Gladwell reviewed our work in The New Yorker. Like all great writing, Gladwell’s review has stood the test of time. As we look back on this great piece of writing, let me draw your attention to what has become my favorite line:
“The problem for basketball experts is that, in a situation with many variables, it’s difficult to know how much weight to assign to each variable. …. In the face of such complexity, people construct their own arbitrary algorithms—they assume that every factor is of equal importance, or randomly elevate one or two factors for the sake of simplifying matters — and we make mistakes because those arbitrary algorithms are, well, arbitrary.”
As I scan reaction to our work I seem much evidence of “arbitrary algorithms.”
Let me be clear. I think our work has generated two basic responses. From those who have actually read our work, the response has been generally quite positive. From those who have not read our work, but via Gladwell’s review or other media coverage have heard about our findings, the reaction has not always been so positive.
Why the “not so positive” responses? The findings that have been reported – Iverson is not as productive as people believe, Kevin Garnett is more productive than people believe, etc… – are not consistent with the arbitrary algorithms people have constructed to evaluate basketball.
This brings up a good question: How do we evaluate the validity of a model? As we tell our students, there are a few criteria one must consider.
1. Is the model theoretically sound? Our Wins Produced algorithm is based upon the fundamental idea that wins are explained by offensive and defensive efficiency. Not sure that one can argue with the theoretical validity of that position. It is important to note that this is not all we do. We have additional models to get at blocked shots, personal fouls, and assists. We would argue these additional models are also theoretically sound.
2. Does the model explain what it purports to explain? We have already noted that Wins Produced and actual team wins are highly correlated. As I noted in my New York Times op-ed: “For the past 10 seasons, the average difference between our projection and the actual number of games won by each of the league’s teams is only 2.3 wins.”
3. Does the model allow us to predict? As noted in our book: “… Lee and Berri (2004) examined the relationship between the number of wins a team achieved and the productivity of its players in the previous season. This work indicates that between 65% and 75% of current wins can be explained by what a team’s players did in the prior season. In other words, knowing past productivity allows one to predict better than one could if all you knew was the current salary players were scheduled to be paid.”
These are the three basic criteria. People looking at the Iverson story, without understanding our methods, want to insert a fourth criteria: Does the model coincide with what people previously believed? Unfortunately for the few critics of our work, that is not a valid criterion for evaluating any model.
Maybe at the end of the day, no approach based on numbers will ever work (I don’t believe that but I am trying to make a point here). Faced with the Wins Produced model – one that is theoretically sound, explains wins, and does a reasonable job at predicting the future — the Iverson supporters claim that the numbers cannot answer this question. This leaves me with a question. How else can we measure productivity in the NBA if we don’t consult the numbers?
One might say that we can’t, and then go watch figure skating. But this is not what the detractors claim. Not only do people argue that the number cannot tell us who is the most productive, but at the same time, these very same people without any numbers claim that they do know who is productive or not. And this appears to be one of the fallacies in the detractor’s argument. One the one hand these people are certain that basketball is too complicated for the numbers to tell us who is productive and who is not. On the other hand, these same people believe their arbitrary algorithms can see through all the complications and tell us that Iverson is truly a great player. Obviously you can’t have it both ways.
Now does our model answer every question one could have about NBA performance? As I have said before, Wins Produced tells us how productive a player has been, not why that player achieved the observed level of productivity.
In sum, Wins Produced does reveal what the statistics the NBA spends considerable resources to track mean in terms of wins. In other words, we can start with a player’s rebounds, steals, points, shooting efficiency, etc… and end with a measure of how many wins a player produces. We can do this because our algorithm is theoretically sound and explains what it purports to explain. And to our knowledge, the arbitrary algorithms that still claim Allen Iverson is a superstar can’t make either claim.
- DJ
11 responses so far ↓
Jeff Pan // July 6, 2006 at 6:53 am
You guys still haven’t answered my question about the 2001 76ers. If Allen Iverson, during his MVP year, had an average performance, how can you explain the Philadelphia 76er’s success? The other 4 starters were Tyrone Hill, Dikembe Mutumbo, George Lynch, and Eric Snow (none of whom had exceptionally good statistics), yet the team started the season 10-0 and made it all the way to the NBA Finals.
Tom Mandel // July 6, 2006 at 8:00 am
Here’s a thought — how about a link to your book on Amazon/BN or your favorite online bookstore (Seminary Books?), so we can buy it? Hey, that’s a really good idea!
Joël // July 8, 2006 at 4:48 am
Mister Mandel, if you surf to amazon.com or another big online book store and you put the title of the book in the search bar and press enter, you will get the info that you want in seconds and you can buy the book. Hey, that’s a really good idea.
Nikos // July 8, 2006 at 5:32 am
I also wanted to ask the same question. Iverson may not be terribly efficient, but it may be partially because the teams contructed around him aren’t particularly talented on offense. Therefore Iverson not seems to have bad selection, but he has to take most of the difficult shots on his team — which tend to drag down a players percentage/efficiency.
Also, how can Rodman be considered more productive then Jordan? Rodman is not a superstar. He is an excellent role player who rebounds and defenders extremely well, but his overall value is overratted in your algorithm IMO. I mean some reasons Rodman played as a Piston (before mid 90s) he didn’t exactly lead his team anywhere despite having some quality support (Dumars/Isiah). Rodman didn’t exactly make the Spurs that much better in 1994 alongside Robinson then when Robinson was the main big men (without Rodman).
How can Rodman be so valuable, yet not even be a competent offensive player (aside from a few assists)? He rebounds exceedingly well and avoids turnovers, which is what I think your algorithm overemphasizes.
dberri // July 8, 2006 at 10:09 am
Nikos,
Please read the post from a few days back on the Law of Diminishing Returns. Don’t skip the comments. That should answer your Iverson comment.
Wait, before you do that, maybe you should look up Iverson’s Olympic performance. He didn’t shoot well then, and he had much better teammates.
As for Rodman being better than Jordan, where are you getting this from? I did post on the value of Jordan some time ago also.
Dan Rosenbaum // July 8, 2006 at 2:29 pm
Nikos, I am sure you know as well as many that Iverson has not been a favorite of the APBRmetrics crowd. For example, he rated 23rd in net player wins in his MVP year – Mutumbo was 25th that year and George Lynch, Theo Ratliff, and Aaron Mckie were in the top 100.
The kinds of discussions that Wages of Wins will generate are very good for many front office personnel and much of the non-stats experts to hear. But this boil-down-player-value-to-one-number approach – while attractive – is something that folks have been moving away from. Much of this is thanks to Dean Oliver who has long argued that players play roles and a particular player’s value is highly dependent on the role he is placed in. This is one of the fundamental ways in which basketball differs from baseball.
On a very good defensive team that would otherwise struggle to generate decent shots, Allen Iverson can be a very valuable player – as he was in his MVP season. But Iverson likely would hurt a team with a highly efficient offense, because as David Berri points out there is very little evidence to suggest that he would be a lot more efficient if he shot less. The beauty of Allen Iverson is that he can be surrounded by a bunch of defensive specialists and by himself generate enough offense to make the team pretty damn good. But put him on a a mediocre defensive team, and he is not going to be particularly effective.
The more we divorce ourselves from the baseball-type thinking that assumes that the contributions of a given player are pretty much the same regardless of what team they play for, the closer we get to understanding how to translate basketball statistics into something useful for GMs and coaches. We are getting there and hopefully this book will bring more people into this discussion.
With me putting my work into practice for an actual team, I cannot always partake in these discussions, but I look forward to seeing how the authors of Wages of Wins respond to the APBRmetrics folks who have thought about these issues for a very long time. The authors and the APBRmetrics folks both could learn a lot from each other, and I look forward to seeing that play out.
Nikos // July 9, 2006 at 5:55 am
I appreciate the responses guys.
As far as the law of diminishing returns I would say that makes sense. But how much do you think Iverson’s efficiency would really DROP alongside Duncan over the course of an 82 game season where he practices with him day in and day out? Sure Iverson not score at a high volume, but he still has the ability to create his shot better than 95% of the league, and that is a pure skill that is more likely to be transferable then a role player who is in the right place at the right time who takes the ‘easy’ shots in a teams offense.
Iverson simply lead a bunch of average players to a very successful season (actually he did have a defensive anchor to help him a lot). Why not give credit or the benefit of the doubt to a player who carries a team’s offense and can create his own shot? I still cannot understand how guys like Mckie, Snow, Mutumbo in ANY system would be nearly as valuable as Iverson under any circumstance? Well maybe Mutumbo considering his defensive prowess in this prime, but the others? I mean what has McKie, Snow or any Philly role player really done on other teams? It is pretty clear Iverson was by far the most valuable player. By scoring at a high volume he allowed his teammates contributions to be more meaningful. If you replaced Iverson with just a mediocre SG who is average at scoring, you really think the team would sniff 40 wins without being extremely lucky? Iverson’s contributions allowed for his teammates to rack up those net wins as supporting players.
Dave, is their anyway you show that supporting players of Iverson were successful in other seasons? Is their anyway you can tell me how good Rodman’s Win Score was as a Piston or Spur?
Dan I understand the concept that the team is more important than the sum of the individual talents – but is their anyway you can prove Iverson’s support were valuable as role players in other years, maybe outside of Philly?
Dave, you mentioned in your book that Rodman was more productive than Pippen and PER minute more productive than Jordan. This is what I was referring too. How can this really be? Rodman when paired with David Robinson didn’t give them great success in the playoffs nor overwhelming regular seasons. Even as a Piston after their titles, the team wasn’t even close to elite despite having Rodman, Dumars, and Isiah in reasonable health. Rodman can’t be as valuable on other teams as his Chicago Bull Win Score indicates. But even though it is Chicago Bull specific, is Rodman really more valuable then Pippen 1996-98? Was Pippen simply overrated? Why aren’t players who play more and contribute more minutes to the team, while still being efficient (like Pippen) be considered more valuable than a guy like Rodman who goes one or two things real well?
The part about Kevin Garnett is interesting. I always felt he was underrated. But I think what the Wages of Wins tells us is that Iverson isn’t as good as his Accumulated and Per Game Stats might indicate. I just wish all these rating systems could adjust for teammates over several seasons, because it is really difficult to argue when things like +/- have been done for only a couple of seasons. This is why I feel it is important to factor things such as Usage, Usage/Efficiency, construction of teammates, team offensive and defensive efficiency etc….
Also what does the Olympics have to do with Iverson in particular? Even Tim Duncan did poorly in the Olympics. Iverson likely could at least be an excellent player if he played alongside Duncan for an 82 game season in the NBA setting.
Harold Almonte // July 10, 2006 at 4:54 pm
Nikos… Rodman once publicly said that Robinson was soft and avoided to defend the other teams best big men and do some kind of defensive plays in order to not look bad on the floor. Robinson was the most talented player ever to put a foot in a basketball floor and needed the hustler and smart Duncan to put a ring on his finger. Duncan won with and without him. Rodman won five rings only creating a lot of possesions for others.
Harold Almonte // July 11, 2006 at 5:40 am
Iverson could do that, because his teammates created a lot, lot of possesions for him to waste.
Harold Almonte // July 11, 2006 at 6:23 am
Ah! … Rodman won five rings like a starter, not from the bench, and could have won with Duncan, I think so.
crack // July 11, 2006 at 7:58 am
So, how do the predictions based on wins produced compare with predictions based on other metrics? How much better is it than just taking the wins from the previous year?