Back in September I was interviewed by Tom Sunnergren of Philadunkia. At the time, I noted that there was more questions from Tom to be answered. And I said at the time that I would get to those questions “this week”. Okay, I really meant “at the end of the Fall semester”.
Today the rest of the interview has finally been posted. These questions touch upon defense, position assignments, coaching, consulting in the NBA, how receptive the NBA are to statistical analysis, and which teams “get it”.
Before we get to my answers, first read Tom’s introduction to this interview; an introduction that I think captures exactly why many sports fans are hostile to statistical analysis.
There is a lot of resistance to stats amongst the general basketball consuming public. Well, not stats. New stats. We love the old stats. Points, assists, points…did I say points yet. We just don’t like people telling us what they mean. We like the power to control the narrative and we resent it deeply when nerds with quiet certainty wrest that power away from us. “Why don’t you watch the games?”, is the only counter we can muster in the face of their overwhelming facts.
People hate, absolutely loathe, being proved wrong. These same people have uniquely strong opinions about sports. And those opinions come from a position of knowledge.
Sports are probably the most complicated thing that the general public understands well, and for a long time, it was the subject area where the knowledge of the average follower most closely rivaled that of the experts, the practitioners. I’ve read a little about Afghanistan, but I obviously don’t have anything resembling the comprehension of the place that, say, David Petraeus has. I could though, I’m pretty sure, coach or GM a basketball team better than a lot of the guys who get paid millions to do it. A lot of people could.
So getting proved wrong about sports, a thing we understand nearly as well as we give our selves credit for, makes people crazy. One of the most enthusiastic perpetrators of this insanity is Dave Berri.
Berri –economist, professor, author, columnist, blogger– is the architect of win score, wins produced, wp48, and a host of other handy tools for understanding the why and how of basketball outcomes. A couple months ago he answered some of our questions on the Sixers (fyi, he saw this start coming) and now he’s provided some A’s for our Q’s on some of the nitty gritty of his methods.
Berri makes us crazy after the jump…
Wages of Wins Journal 
Gil Meriken
November 30, 2010
I’ll try to be as polite as I can here, you guys are for the most part civil.
But could it be that the resistance is not to statistical analysis in general, but to WP, PER , adjusted plus/minus, etc?
Could it be that the resistance is not due to some fear of data, but because they don’t think the system is correct in terms of evaluating winning performance and winning players?
A system can correlate well, be objective, be consistent … and still be wrong.
Before I get creamed here, I’d like to acknowledge, I could be wrong.
dberri
November 30, 2010
I think I would like you to explain how a system could correlate well (i.e. explain wins) and be consistent (allow us to say something about the future) and still be “wrong”. In what sense would it then be “wrong”? I think you need to define what you mean. Certainly a model can explain what we see, allow us to say something about the future, and not be consistent with what people generally believe. But when that happens, we tend to think that what peopel generally believe is “wrong”.
nerdnumbers
November 30, 2010
Gil,
I actually like your point. Any stats class teaches “correlation is not cause and effect.” Dean Oliver mentions the laugh test as a metric for failing a model. Now he’s wrong. But the important thing is to look at what the model tells you and apply it. So if I give you a model (e.g. Wins Produced) and say “This predicts wins well, says here’s what makes a player good and here’s what players it likes” then step two is to look into the model and decide if you should use it or not (not say it fails the laugh test and move on.)
The issue I know DJ has pointed out is even worse. If I said “This model doesn’t predict wins well, and it likes these players.” Your first step should be to throw the model out. While correlation does not imply cause and effect, no correlation pretty much rules out cause and effect. However, many people hold very tightly to these models “I know basketball, I watch every game.”, etc. If GMs really hand the scrutiny you hope for (rest assured I do too) the issue wouldn’t be a dislike for Wins Produced but also a change in general business (I’ve got to draft a shooting guard with potential!)
kevin
November 30, 2010
“I actually like your point. Any stats class teaches “correlation is not cause and effect.” Dean Oliver mentions the laugh test as a metric for failing a model.”
Well, I agree with Dave. If a model correlates well, then it has to be taken seriously. But one must take it a step further. The acid test of a model is it’s predictiveness. If a model is consistently predictive, and stays predictive as you tinker with the variables, then it probably isn’t “wrong”.
I think WP and WP48 are pretty good as far as predictiveness goes, once you factor out the injury and age variables. I think it does an espccially good job of reducing the high volume scorer types to their true value. It may overestimate low usage rebounder types a bit, since go-to guys take efficiency hits because it is they who have to assume responsibility of most of the difficult possessions but, overall, I think it works quite well.
PER, on the other hand, is awful. Hollinger is in a quandary because it’s turning out the Heat AREN’T a 66 wins team after all.
Italian Stallion
November 30, 2010
I think GMs would be more open to a model that did a good job of predicting how many games a team was going to win before the season started instead of explaining those wins after the fact.
Most people feel that team construction is an important part of success because they believe the productivity of an individual player is at least partially dependent on his teammates.
I realize this model suggests that player performance is fairly consistent from year to year and from team to team, but I think part of the explanation is that GMs and coaches are actually doing a pretty good job of constructing teams. They generally build well balanced teams with players possessing a variety of important skills that allow all the players to get at least close to their potential. When they make a mistake, they trade that player for what they need.
I think Arturo’s (and other) predictions before the season will be a pretty good test.
The model predicted several teams to be a lot better or worse than the general consensus either because of off season trades, highly productive players that were injured last year returning, etc…. It will be interesting to see how well they do. (of course everyone understands that injuries and other trades could happen this year also)
Leroy Smith
November 30, 2010
To test a model I only have to ask myself (and the Model’s creator two questions):
1) Does it rank Michael Jordan as the greatest player since the Merger?
2) Does it rank the 96 bulls and the 86 celtics as 1a and 1b as far as greatest teams in recent memory?
If a model is worth the cyber space it is posted on, it will answer these questions without equivocation.
ilikeflowers
November 30, 2010
You’re in luck Leroy, I just finished the BestModelEvar!
BestModelEvar{
long getPlayerRank(String name){
if(“Michael Jordan”.equals(name)){
result = 1;
}else{
result = 2 + Math.random()*Integer.MAX_VALUE;
}
return result;
}
long getTeamRank(String name, int year){
if(“Chicago Bulls”.equals(name) && year == 1996){
result = 1;
}else if(“Boston Celtics”.equals(name) && year == 1986){
result = 2;
}else{
result = 3 + Math.random()*Integer.MAX_VALUE;
}
return result;
}
}
nerdnumbers
November 30, 2010
Italian,
Dan Ariely did a great post on your point (http://danariely.com/2010/11/21/good-decisions-bad-outcomes/) In short predictions are all well on good but only when you consider how and why they were made. Portland and Miami look less promising than they did at the start of the season and some of these are for unexpected outcomes. As such I think the model that explains data you can control is better than “blind predictions”. Of course though as I’ve mentioned the problem is that the models being used by many people don’t even pass the “explain what happened” test :)
Leroy,
WP48 likes both of those teams but in what world is 15 years and 25 years ago “recent memory”?
Leroy Smith
December 1, 2010
Ilikeflowers, LMAO. I love your formula.
Nerd, I’m 30 years old so I guess 15 and 25 years ago are my “only memories”, but “recent memory” sounded better.
By the way, I refused to buy stock in the heat when all the “experts” were loving them. But I am now ready to start buying all the stock being dumped by conventional thinkers.
fricktho
December 1, 2010
A lot of these comments use the word ‘prediction’ and I’m not sure that is the best use of WP. It explains wins, or what players produce wins. It doesn’t necessarily predict wins. I know there are different articles explaining that WP isn’t drastically different across seasons, or with different teammates, or with different coaches, but there is some change. To use it as a predictor means to also be subjective to a degree when making those predictions, as in factoring age, the quality of teammates, etc. The idea that good players excel on bad teams, but their WP may decline when they play with better teammates. As we saw with Boston, and now with Miami. If you want to predict the upcoming season you have to factor in a lot of unknowns, and what you ‘think’ will be the result. Taking WP numbers straight from the previous season, or previous situation, and using those to predict the outcome of a change has some flaws. Not drastic flaws, and I think even without those things accounted for you can get a general idea of what teams will be good and what teams will be bad. But to actually predict the final season standings you need some type of great foresight whether you use advanced metrics or just intuition.
Italian Stallion
December 1, 2010
fricktho and nerdnumbers,
I think sophisticated users understand that things come up that change the results that a model predicted that have nothing to with the quality of the model.
The problem with only explaining things after the fact is that it doesn’t have much use or prove anything.
A GM wants to know that if a model says that player “X” will add 10 wins if he acquires him, that assuming no other issues, he will actually add close to 10 wins and not 5 wins because the model allocated team wins improperly on an individual basis.
Part of my point in the previous post was that I think they already consider issues of fitting players together properly fairly well and recognize when that’s limiting a player’s productivity.
Mike
February 24, 2011
How well does wins produced correlate with wins without the team defense adjustment?