Providing the Statistical Analysis that Matthew Yglesias Needs

Posted on December 3, 2007 by


Matthew Yglesias – of the Atlantic – has asked for a specific study of the NBA.  In The Statistical Analysis We Need, Yglesias stated the following:

I was looking at this latest iteration of efforts to use adjusted +/- statistics to evaluate NBA players, and it served as a reminder of how frustrating I find it that such a large proportion of efforts to apply quantitative tools to the analysis of basketball are dedicated to these searches for magic formulae to assess player quality. There are other, more interesting and probably more fruitful, lines of inquiry where quantitative skills could shed some light.

For example, there’s a popular conception of a link between pace and defensive orientation — specifically the idea that teams that choose to play at a fast pace are sacrificing something in the defense department. On the most naive level, that’s simply because a high pace leads to more points being given up. But I think it’s generally assumed that it holds up in efficiency terms as well. The 2006-2007 Phoenix Suns, for example, were first in offensive efficiency, third in pace, and fourteenth in defense. But is this really true? If you look at the data season-by-season is there a correlation between pace and defense? When pace changes leaguewide, does scoring efficiency also change? Then there are lots of interesting team level issues to ask. Intuitively, some teams’ offenses are optimized for the fast-paced style and will function less efficiently during games that wind up being played at a slow pace. And vice versa also probably holds. But are there some teams who are making a mistake? Squads who score more efficiently when they play slower, but usually try to play fast?

I’m too lazy to actually conduct research into those questions, and I’m not even sure I know how to calculate a coefficient of correlation correctly these days, but I’d read someone who wanted to do it.

After I read this I decided to stop my work on my “magic formula” and decided to address the question posed by Yglesias.

The Link  Between Pace and Efficiency

I have data on possessions, offensive efficiency [points scored divided by possessions], and defensive efficiency [points allowed divided by possessions] from 1973-74 to 2006-07.  Now possessions have generally decreased across time.  So I calculated for each team their relative possessions, where relative possessions are a team’s possessions per game divided by the league average possessions per game in that particular season.

Given this data, is there a link between offensive efficiency and pace? In other words, if a team plays at a faster pace, does the team become more or less efficient?

The answer is no and no.  Specifically, the correlation coefficient between relative possessions and offensive efficiency tells us there is no relationship.  The correlation coefficient is only 0.03.  And when you regress offensive efficiency on possessions you fail to find a statistically significant relationship.

Turning to defense, we find a little more, although not much more.  The correlation coefficient between relative possessions and defensive efficiency is 0.17.  Regressing defensive efficiency on relative possession reveals that there is a statistically significant relationship.  The more possessions a team has per game – again, relative to the league average – the more points the team’s opponents will score per possession.  But relative possessions only explains 2.8% of defensive efficiency.  In sum, pace doesn’t tell us much about defensive efficiency.

And I suspect that a well-defined defensive efficiency model would tell us, as we found with respect to offensive efficiency, that pace doesn’t matter on the defensive side of the ball either.  In other words, the simple univariate model (or a model with only one independent variable) is mis-specified.  If I took the time to properly specify the model, by including all the independent variables that impact defensive efficiency, it may very well be the case that the statistical significance of pace will vanish.

Then again, it might not.  Anyway, it doesn’t appear that pace has much impact on defensive efficiency.

Hopefully that answers the question.  Now I need to get back to working on my “magical formula.”

On “Magical” Formulas

Okay, I’m just kidding.  I’m not working on any “magical formula.”  I am pretty happy with Wins Produced, Wins Score, and PAWSmin. Other than working on how I explain these models, I am not really working on these metrics.

I would note that there is nothing “magical” about the Wages of Wins models, or even adjusted plus-minus (or the models of Dean Oliver). Each of these models are just ways of looking at performance in basketball.  And each has their pluses and minuses (pardon the pun).

I sense, though, that people become frustrated with these metrics because they expect “magic.”  In other words, people want a number that answers all questions and reduce the cost of thinking to zero.  Models, though, help us explain the world we observe.  Models are not “magical”, nor do they remove the need to keep thinking.  And that is something to think about when you look at basketball measures, or any other models researchers offer to improve our understanding of our world.

– DJ