Last week my sports economics class at Southern Utah University had a visit from Joe Price. Joe is an assistant professor of economics at BYU. More importantly – at least to readers in this forum – Joe is also one of the authors of the famous Price-Wolfers study. Readers might recall that this was a study of racial bias among referees in the NBA (a paper you can still see on-line HERE).
Back in May of 2007, this paper was the subject of a front-page story in the New York Times (written by Alan Schwarz). And it led to a substantial reaction from both the NBA and observers of the Association.
Two years ago this paper was presented at the Western Economic Association (in a session I helped organize and where I was officially the paper’s discussant). At a meeting, though, you only get to hear 15 to 20 minutes on a paper. Last week in class, though, Joe gave us a full hour. And the extra time made it even clearer why this is a very good piece of research.
Re-Telling the Story
Although I think most economists who have read the paper agree with my assessment, the reaction of non-economists was not always positive. In fact Joe began his presentation with the reaction of Charles Barkley (which you can still hear at the website of Justin Wolfers). After the students got to hear the assessment of Sir Charles, Joe commenced with the explanation of this research.
The aforementioned New York Times article highlighted the basic finding. Via an examination of more than ten years of data, Price and Wolfers found evidence that white referees called more fouls on black players. And furthermore, black referees called more fouls on white players.
To understand this result it’s important to have some sense of the work involved in this research. Here are some highlights.
- The data set consisted of over 250,000 player observations. The sheer size of the data set made estimating the model quite difficult. Joe indicated that one run of the model took three weeks to complete.
- The size of the data set was not the only complexity. Joe and Justin also controlled for literally thousands of independent variables. Again, this was a very large model.
- The basic finding – the race of the referees and players impacts the fouls called on a player – was consistently found despite the fact the authors employed a variety of different specifications. In sum, this result is quite robust.
All of these points can be seen in the article. Joe’s presentation, though, added in a few more details not found in the original paper.
The NBA’s Response
For example, at the time the study came out David Stern – the Commissioner of the NBA – was quite unhappy. He was so unhappy that he hired a consulting firm to refute the Price-Wolfers study. Unfortunately, the quality of work offered by the consulting firm was consistent with what you sometimes see in on-line studies. In other words, it wasn’t very good. In fact, much of it consisted of mistakes you would not expect an undergraduate in econometrics to make.
For instance, the Price-Wolfers study used dummy variables to capture race. A dummy variable takes on the value of zero or one to reflect a specific condition. In the Price-Wolfers study, black players were assigned a value of one while white players were assigned a value of zero.
The NBA study took the same approach. But they also decided to re-estimate their model with the dummy variable defined in the opposite fashion (white players are assigned one, black players are assigned zero). Such a step, though, is pointless. Re-defining the dummy variable just switches the sign on the coefficient. It doesn’t change anything else in the model and suggests that the person doing the work for the NBA didn’t understand dummy variables.
As Joe noted, the dummy variable issue was just the beginning of the problems with the NBA study. After detailing other problems Joe then observed that even the flawed NBA approach actually confirmed the story told by Price-Wolfers. Yes, the NBA study also presented evidence of racial bias.
A Trivial Implication and then the Big Story
After reviewing the NBA’s study, Joe then proceeded to talk about gambling. As Joe emphasized, he does not condone gambling (nor does this website). Nevertheless, Joe and Justin were able to find evidence that knowing the race of the referees and players in an NBA game can help a person who does gamble on NBA games. One should emphasize, though, that this result was based on past data. Joe was not certain it would be true today (so gamblers today may not profit from this information).
Although the gambling aspect is interesting, the point of the story is not how NBA fans can improve their gambling profits. The point of the story is not even how to improve basketball. No, the big story of this research is all about implicit bias.
Malcolm Gladwell discussed this issue in Blink. Much work has been done to eliminate explicit biases in society. Studies have shown, though that most everyone has implicit biases with respect to race, gender, sexual orientation…. In fact, you can take a test to determine how much bias you have. And implicit biases are harder to eliminate. That is, if you are not aware these exist.
Extending the Story
The good news is that there is evidence that if you are aware of your own biases – and you have an incentive to change your behavior — these biases can be mitigated. To see this, consider the latest from Joe Price. Joe – along with Lars Lefgren and Henry Tappen -is presenting a paper at the 2009 Western Economic Association meetings. The research presented in this paper focuses on evidence of racial bias among NBA players. The paper has not been published yet (and I don’t have an on-line link) but I can share the abstract:
Using data from the National Basketball Association (NBA), we examine whether patterns of workplace cooperation occur disproportionately among workers of the same race. We find that, holding constant the composition of teammates on the floor, basketball players are no more likely to complete an assist to a player of the same race than a player of a different race. Our confidence interval allows us to reject even small amounts of same race bias in passing patterns. Our findings suggest that high levels of interracial cooperation can occur in a setting in which workers are operating in a highly visible setting with strong incentives to behave efficiently.
This latest research from Lefgren, Price, and Tappen suggests that when people have an incentive to eliminate a bias, such biases don’t exist. Such research suggests that even implicit biases can be eliminated.
But the first step in that process is to recognize that the bias exists. In other words, David Stern’s first step – which consisted of denying that the issue existed — was very much in the wrong direction. Had Stern taken the time to understand Price-Wolfers the NBA probably could have already implemented steps to eliminate the bias (a bias that was uncovered in more recent years than the seasons originally investigated). In sum, positive steps can be taken when one understands the world as it is. Ignoring evidence, though, doesn’t generally help one solve a problem.
– DJ
The WoW Journal Comments Policy
Our research on the NBA was summarized HERE.
The Technical Notes at wagesofwins.com provides substantially more information on the published research behind Wins Produced and Win Score
Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:
Simple Models of Player Performance
What Wins Produced Says and What It Does Not Say
Introducing PAWSmin — and a Defense of Box Score Statistics
Finally, A Guide to Evaluating Models contains useful hints on how to interpret and evaluate statistical models.
Wages of Wins Journal 
TRad
April 15, 2009
The story about dummy variable is great :)
Adam
April 15, 2009
Thanks for your article and the interesting link to implicit bias tests! Though pointless, switching the dummy variable is not wrong. Unless it was used to draw a conclusion?!
dberri
April 15, 2009
Adam,
The different dummy variable definitions were used in “different” models. So the author would estimate Model 1 with one dummy and then Model 2 had the dummy re-defined. Model 1 and 2, though, were the same models.
Adam
April 16, 2009
Oh dear! Thank you, sir.
Horsecow
April 16, 2009
Thanks for the info, DB. I’ve always wondered if there might be a similar implicit bias due to crowd noise. Refs have to make split-second calls, and, as irrational as it may seem, I could see how they might hesitate just a little bit out of fear that the whistle wouldn’t be heard.
Michael
April 16, 2009
Interesting post prof, gives a little insight into the kind of work you guys do.
One thing I would like to say is that you seem quite harsh on David Stern here for his actions in response to the study. Whilst I understand your position, I think you need to realise that for Stern (and indeed the league itself) a study indicating racial bias amongst referees going back at least a decade would pose quite a significant political problem. Making an issue out of something like that which was only really made visible by a very large study based over 10 years of player observations wouldn’t be beneficial to the league in terms of public and player relations, marketing etc etc. Remember the hoopla about the dress codes? the new balls? Can you imagine the ruckus that would surround implementing systems to combat barely perceptible implicit racial bias amongst the leagues officials? How would the public respond? the press? the players? the officials themselves? Oscar Robertson would have a field day! I think it would be making a volcano out of a mole hill, the lava might build some bridges and fill some gaps, buts it’s more likely to start a fire!
JoeM
April 16, 2009
So it turns out I strongly prefer white people to black people. Not all that surprised.(guessed moderately pre-test)
/don’t consider myself racist at all really.
Tom Mandel
April 16, 2009
The dummy variable anecdote does tell us who the dummy is! :)
Tom Mandel
April 16, 2009
I agree about David Stern, however — there is no other position he could have taken but to challenge, and attempt to refute, the paper. Behind the scenes, however, the research may well have motivated, or may be motivating, some kind of program of which we are unaware?
stephanie
April 16, 2009
And if you’re Yao Ming, well, you may as well forget getting any calls.
Michael
April 16, 2009
Turns out on that test that my “data suggest[s] little to no automatic preference between African American and European American.”
Guess I could never be an NBA Ref :D
Shek
April 16, 2009
Horsecow,
The following study answers your question in the context of Italian soccer: http://people.su.se/~pepet/Socialpressure.pdf
Best,
Shek
Italian Stallion
April 17, 2009
Great Article.
Mark
April 18, 2009
Dave Berri, why as an economist are you against gambling?
Aren’t you supposed to like market completeness and/or have a libertarian impulse towards removal of unncessary government interference?
Also, Gladwell’s writing on implicit biases is politically correct drivel.
Evan
April 18, 2009
Mark —
Have you read Prof Berri’s blog long? I’m guessing not, as he’s very clearly not a libertarian, and pretty to the left on the political spectrum.
Evan
April 18, 2009
That last phrase was worded ineloquently.
Steve G
April 22, 2009
I think we should use Las Vegas bookies to keep industry research at bay. For example, as soon as you heard that David Stern had commissioned a research project to find out whether there is racial bias in the NBA officiating, what odds would take that the research would conclude there wasn’t?
crzyleg
April 22, 2009
What is the pin that yhe coaches in the NBA stand for CD.
RobertPL
April 24, 2009
It’s not a bad study, but the authors (and definitely the media) might be the overselling conclusions.
1. Economic significance vs. statistical significance. The claim of implicit bias comes from a racial dummy variable being significant. With so many observations in the sample even very small effects will appear statistically significant. How relevant is this implicit bias? Can it be translated to points per game? How often will it change the outcome of games if we replace an all black crew with an all white one?
2. I was impressed by the authors’ controlling for so many covariates, but they can’t possibly control for everything. Some relevant factors in making a call might be inherently unobservable.
Here is a story. Suppose referees are completely color-blind, but they hate players who bitch. The more a player complains, the more likely they are to call a foul on him. It might happen that black players complain more to white referees, because, say, they are less willing to accept their authority. This will cause white refs to call more fouls on black players, but not because of racial bias but rather dislike of bitching. So the conclusion is reversed: It might well be that it is the *players*, and not the refs, that have the implicit racial bias.
In sum, there might well be unobserved factors affecting call frequency that are also correlated with race or some of the other explanatory variables included. Hence, the study is not immune to omitted variable bias.