Different Answers — Same Conclusions

Posted on December 18, 2007 by

25


Who was the best player in 2006-07? The MVP was Dirk Nowitzki.  The scoring leader was Kobe Bryant.  And the leader in Player Efficiency Rating (PER) was Dwyane Wade.

Of course, voting by the sportswriters is problematic.  Sports writers tend to pick the leading scorer – or in the case of Steve Nash, leading contributor to offense – on one of the best teams.

Looking at scoring alone is also a problem, since it completely ignores all other facets of the game.  Plus, scoring alone is not highly correlated with team wins.

And then there is PER.  The problems with the PER were detailed in the following two posts:

Marvin Williams Makes a Hypothetical Deal (December 16, 2007)

A Comment on the Player Efficiency Rating (November 17, 2006)

Each of these posts highlighted how an inefficient scorer could dramatically increase his PER and Game Score (Game Score is John Hollinger’s simple version of PER) value by simply taking more inefficient shots.  Certainly inefficiency shooting is inconsistent with winning basketball games.

Well, if we are not going to rely on the sports writers, scoring, or PER, what else can we look at?  Surely I am not going to once again suggest Wins Produced?

The Best in Player Winning Percentage

Although I like Wins Produced, let’s look at something else.  Here are the top ten players in 2006-07 in Player Winning Percentage [PW%].

1. Dikembe Mutombo

2. Alan Henderson

3. Brent Barry

4. Chuck Hayes

5. Dirk Nowitzki

6. David Lee

7. Manu Ginobili

8. Ira Newble

9. Tim Duncan

10. Shawn Marion

Well, that’s interesting.  Alan Henderson, Chuck Hayes, and Ira Newble are all better than Tim Duncan.  Wow. 

Okay, what’s PW%?  Is this a new WoW Metric designed to torment sports bloggers?

Here is how PW% is described at Basketball Reference

Player winning percentage; the formula is ORtg14 / (ORtg14 + DRtg14).

What is ORtg and DRtg?  Again we turn to Basketball Reference.

ORtg:  Offensive rating (available since the 1977-78 season in NBA); for players it is points produced per 100 posessions, while for teams it is points scored per 100 possessions. This rating was developed by Dean Oliver, author of Basketball on Paper. I will point you to Dean’s book for complete details.

DRtg: Defensive rating (available since the 1977-78 season in NBA); for players and teams it is points allowed per 100 posessions. This rating was developed by Dean Oliver, author of Basketball on Paper. I will point you to Dean’s book for complete details.

Yes, PW% comes from the work of Dean Oliver and his wonderful book, Basketball on Paper. And the results of this work seem to defy conventional wisdom.

A Look at Adjusted Plus-Minus

Of course, Oliver’s work – like The Wages of Wins – is based on box score statistics. What if we turn to adjusted plus-minus?  Will this give us a measure that is consistent with common sense?

In the spirit of The Wages of Wins quiz posted by Ballhype, please take the following test.  Specifically, which of the following players is better?

Anthony Parker or Vince Carter

Dwight Howard or Brian Cook

Allen Iverson or Rajon Rondo

Dan Dickau or Steve Nash

Anderson Varejao or Chris Bosh

Hedo Turkoglu or Carmelo Anthony

Rudy Gay or Brandon Roy

Jason Richardson or Rafer Alston

Tracy McGrady or Bobby Jackson

Corliss Williamson or Carlos Boozer

Looking strictly at the Adjusted Plus-Minus per 40 minutes rankings of Steve Ilardi (posted at 82games.com), the answers are

Parker, Cook, Rondo, Dickau, Varejao, Turkoglu, Gay, Alston, Jackson, and Williamson.

Well, that hardly fits conventional wisdom.  Is Parker a more effective player than Carter, Iverson, or Melo? Is Dickau really a better point guard than Nash?  Should the Orlando Magic let Brian Cook play more than Dwight Howard? Was Rudy Gay a better rookie than Brandon Roy – the player named Rookie of the Year?

Learning How to Evaluate Models

The purpose of the above exercise was not to attack the work of Dean Oliver or the adjusted plus-minus approach created by Wayne Winston and Jeff Sagarin (yes, these are Ilardi’s numbers but this basic method was originated by Winston-Sagarin). 

Nor was this exercise designed to tell us that there is much to be learned in creating advanced statistical measures for basketball, and until this is learned, let’s stick with conventional wisdom.

No, the purpose of this exercise is to show that there is still much to be learned when it comes to determining the value of a particular model.

A few weeks ago I offered a column that was designed to help.  In A Guide to Evaluating Models, I identified several issues one should think about when looking at a model.  This list of issues included the model’s theoretical foundation, robustness, explanatory power, out of sample forecasting power, and simplicity.  I emphasized that this was but a partial list.  Furthermore, and this is perhaps most important, although my list was not all inclusive, a complete list would not argue that we should evaluate a model in terms of its ability to confirm what we already believed. 

Models are designed to test theories.  If you reject a model because your theory was not confirmed, then you simply don’t understand why you created the model in the first place.

Turning to basketball, this means we do not become unhappy when a model tells us that Andrew Bynum is more productive on a per-minute basis than Kobe Bryant.  Or as PW% indicates, Dikembe Mutombo is more effective than Yao Ming. Or as Adjusted plus-minus says, Anthony Parker is more effective than Carmelo Anthony.

Learning from these Models

When we look at Wins Produced, PW%, and Adjusted plus-minus, we see some similarities.  And we see some differences. Certainly, for reasons stated in The Wages of Wins, I prefer Wins Produced.  But regardless of my preference (or your preference) one result is clear from each of these advanced metrics.

The players who score the most are not always the players who contribute the most to wins.  This is the lesson that The Wages of Wins seeks to teach about basketball.  And when I look at these different advanced methods, it’s this lesson I think you learn.

One Final Thought

Let me close this column by repeating something I said a couple of weeks ago.

I would note that there is nothing “magical” about the Wages of Wins models, or even adjusted plus-minus (or the models of Dean Oliver). Each of these models are just ways of looking at performance in basketball.  And each has their pluses and minuses (pardon the pun)

I sense, though, that people become frustrated with these metrics because they expect “magic.”  In other words, people want a number that answers all questions and reduce the cost of thinking to zero.  Models, though, help us explain the world we observe.  Models are not “magical”, nor do they remove the need to keep thinking.  And that is something to think about when you look at basketball measures, or any other models researchers offer to improve our understanding of our world.

– DJ

Our research on the NBA was summarized HERE.

The Technical Notes at wagesofwins.com provides substantially more information on the published research behind Wins Produced and Win Score

Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:

Simple Models of Player Performance

Wins Produced vs. Win Score

What Wins Produced Says and What It Does Not Say

Introducing PAWSmin — and a Defense of Box Score Statistics

Finally, A Guide to Evaluating Models contains useful hints on how to interpret and evaluate statistical models.

Posted in: Basketball Stories