Dictionary.com tells us that the word “overrated” means: to rate or appraise too highly; overestimate
The word “high” is a relative term. To argue that something is “too high” or “too low” requires a reference point. Actually, to be precise, you need two points of reference.
For example, batting average in baseball will overestimate a player’s contribution if that player fails to draw walks and/or can’t hit for power. If we wish to argue that batting average overestimates a player’s contribution we must note where a player is ranked in terms of batting average and also where he is ranked in terms of a measure that considers plate discipline and power (i.e. OPS). In sum, we need two metrics to make the “overrated” argument.
Batting average has been calculated in baseball since the 19th century and it remains the most cited measure of a hitter’s effectiveness (one only needs to watch a baseball game on TV to see this point). When we turn to basketball, the “batting average statistic” (i.e. oldest and most cited) is points scored per game.
As we note in The Wages of Wins (and/or in this forum), this single metric does a remarkably good job of explaining the player evaluations of the coaches, general managers, and the sports media. And just like batting average, points scored per game is obviously not a perfect measure.
Focusing just on scoring ignores other factors that contribution to wins (such as rebounds, turnovers, steals, etc…). Scoring totals can also be inflated with more minutes and/or more shot attempts. In other words, lofty scoring totals may not coincide with scoring efficiency. Despite these shortcomings, scoring totals still dominate the discussion of a basketball player’s value.
Three Perspectives on the Overrated
The topic today is “overrated”, and as noted above, we need two reference points to tackle this subject. And given that this is The Wages of Wins Journal, the second reference point we will consider is Wins Produced. Specifically – as detailed in Table One — we are going to first look at how players rank in terms of both Wins Produced and points-per-game.
Table One: The Overrated Scorers
Table One reports the 15 players who are the most overrated by scoring. At the top of the list is Richard Jefferson. Of the 129 players who played at least 2,000 minutes in 2007-08, Jefferson ranks 9th in points per game. As detailed a few days ago, Jefferson – who used to be quite productive – doesn’t offer much beyond scoring these days. Consequently, with respect to Wins Produced he ranks quite low (106 out of 129).
Although points-per-game is frequently cited, there are of course more sophisticated metrics that try and take into account more of what a player does on the court. One of the oldest (and again, commonly cited) of these metrics is NBA Efficiency. This measure is calculated as follows:
NBA Efficiency = Points + Rebounds + Steals + Assists + Blocked Shots – All Missed Shots – Turnovers
Table Two looks at the most overrated players according to this measure.
Table Two: The Overrated in NBA Efficiency
Again our second point of reference is Wins Produced. And again we see Richard Jefferson topping the list. Joining Jefferson on both lists are Rudy Gay, Stephen Jackson, Rashard Lewis, Kevin Durant, LaMarcus Aldridge, Jamal Crawford, and Joe Johnson.
The similarities between the two evaluations are not surprising. The correlation coefficient between these two metrics for the 2007-08 season is 0.89 (NBA Efficiency per game and points scored per game has a 0.93 correlation).
Of course NBA Efficiency is not very sophisticated. This metric essentially adds together a player’s positive actions and subtracts off the negative. What happens if we turn to a measure that attempts to weight each player’s actions?
The most popular weighted measure is John Hollinger’s Player Efficiency Rating (PER). The simple version (okay, much simpler) of the PER metric – Game Score – indicates the basic weights employed.
Game Score = Points + 0.4*Made Field Goals – 0.7*Field Goal Attempts – 0.4*Free Throws Missed + 0.7*Offensive Rebounds + 0.3*Defensive Rebounds + Steals + 0.7*Assists + 0.7*Blocked Shots – 0.4* Personal Fouls – Turnovers
For 2007-08, Game Score and NBA Efficiency have a 0.99 correlation (this is basically what we find in other years as well). PER is a per-minute metric, and it has a 0.99 correlation with Game Score per-minute. Given the high correlation between these measures, we shouldn’t expect much difference in our list of overrated.
Table Three: The Overrated in PER
Before we get to the results, though, we need to note that since PER is a per-minute measure our second reference point has to be WP48 [Wins Produced per 48 minutes]. Despite this change, we still see some familiar names. Once again we see Richard Jefferson, Rudy Gay, LaMarcus Aldridge, Kevin Durant, Rashard Lewis, and Jamal Crawford. Jefferson, though, does not lead the list of overrated when we compare PER and WP48. Now LaMarcus Aldridge tops the list.
The Most Overrated
Although Aldridge tops the PER list, he is not the most overrated if we consider all three rankings. If we add together the difference reported in each table the most overrated player in the NBA for 2007-08 is….. yes, Richard Jefferson. Here are the 15 most overrated players if we consider all three perspectives.
1. Richard Jefferson
2. Kevin Durant
3. Rudy Gay
4. Stephen Jackson
5. Jamal Crawford
6. LaMarcus Aldridge
7. Michael Redd
8. David West
9. Rashard Lewis
10. Ron Artest
11. Allen Iverson
12. Tracy McGrady
13. Carmelo Anthony
14. Corey Maggette
15. Gerald Wallace
One should be clear that “overrated” does not necessarily mean “bad.” West, Iverson, McGrady, Anthony, Maggette, and Wallace were above average players [in terms of WP48] this past season. These players just weren’t as good as the scoring based metrics suggest.
In the next week (if I find the time) I will write a column on the most underrated. For those who want more on this topic, you can see a discussion of past seasons in Chapter 10 of The Wages of Wins. For more on NBA Efficiency, PER, and Game Score, please see the following posts:
NBA Efficiency: Do We Overvalue Rebounds? (November 9, 2006).
PER: A Comment on the Player Efficiency Rating (November 17, 2006)
Game Score: Marvin Williams Makes a Hypothetical Deal (December 16, 2007)
- DJ
Our research on the NBA was summarized HERE.
The Technical Notes at wagesofwins.com provides substantially more information on the published research behind Wins Produced and Win Score
Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:
Simple Models of Player Performance
What Wins Produced Says and What It Does Not Say
Introducing PAWSmin — and a Defense of Box Score Statistics
Finally, A Guide to Evaluating Models contains useful hints on how to interpret and evaluate statistical models.
23 responses so far ↓
John // May 15, 2008 at 6:26 pm
Can we get underrated? I’d like to see that a lot more.
Erich // May 15, 2008 at 8:00 pm
Hmmm.. today realgm.com had an article highlighting trade interest between Denver and New Jersey. Denver’s old GM is looking for some of his former draft picks in Carmelo & Camby and Jefferson may be part of the return. Denver could be trading their way back to the lottery…
At one point, I tried to calculate PER for NCAA players. I botched the math some how and never got any intelligible results. It is not a simple exercise.
mrparker // May 16, 2008 at 10:07 am
This article got me thinking about how this might affect a coaches mindset when deciding how to allocate their playoff minutes. I decided to take a look at how good of a job this year playoff coaches have been doing at rating their players correctly.
I’ve been “researching” playoff performance vs. regular season performance. I’ve been looking at the allocation of playoff minutes as compared to regular season performance(using my own rating system). I have my own data on how coaches have helped or hurt their teams with their lineup choices. What I’d love to see is an article here on which coaches have done the best/worst jobs according to wp48.
Its my opinino that Avery Johnson really botched the Hornets series by refusing to give any true center “starter minutes”. When Dampier is 7th in minutes played and their is no back up center something has been screwed up. He took his 4th best player who played the 4th most minutes in the regular season and gave him the 7th most minutes in the series. Meanwhile Jerry Stackhouse who was the worst player on the team played the 6th most minutes.
Another bad job has been turned in by Doc Rivers. Number 1, he is giving Ray Allen, the teams 3rd best player over the regular season more minutes than KG and Paul Pierce, the teams number 1 and number 2 best players over the regular season. Number 2, he is not giving Eddie House any minutes even though he has been effective at both pg and sg positions. Meanwhile, Sam Cassell has been playing 15 minutes a game as Rondo’s backup and as in the regular season has been the teams worst player in the regular rotation. It doesn’t take a rocket scientist to realize that Ray Allen is logging the most minutes while looking sluggish throughout the series. I propose that Tony Allen play more minutes as the team still had a great efficiency differential with Allen on the court.
The best job has been turned in by………Mike Brown. He is the only coach to give his 5 best players the most minutes. He has sat Pachulia on the bench and given Wally Szerbiak plenty of minutes. While the guy was pretty awful for the Cavs during the regular season he has been lights out in the playoffs.
Westy // May 16, 2008 at 11:31 am
Definitely interesting, and I agree to an extent. There is no doubt scoring is overvalued by the media (as batting avg. continues to be for baseball), and probably by some in positions of authority in the NBA. But I think many of those decision makers have a pretty good understanding of its correct position.
I guess this is possibly a good place to pull the whole discussion back out about what reliability WP has. Your assumption above is that the majority of team GMs are wrong and your metric is right. I guess I continue to find that unconvincing. I mean no disrespect, but do you ever wonder why everyone else doesn’t agree with your formula if it’s correct?
Does it matter that most of the other ‘experts’ have basically written this approach off at this point? The consensus seems to be, “…high defensive rebound weights correlate well with team wins only because its the simple way to incorporate team defense into the equation but it says nothing about how good individual players are.” Weightings that result in more similar scores at each position seems more accepted.
Further, at APBRmetrics, it is noted that, “As David Berri has shown, regressions won’t work at the player level, although they’re probably a good start. The best work on player value — certainly as the starting point for further work — is Dean Oliver’s stuff.” As I understand it, Dean Oliver also does not agree with the valuation of rebounds as presented in the WP formulation.
Maybe some of this will be addressed in the upcoming papers and book sequel; but I for one remain convinced that although PER may slightly overvalue scoring, WP substantially overvalues defensive rebounding, and so is similarly flawed. And right now that position seems to be supported by the majority of the advanced basketball statistics community.
GoldShammGold // May 16, 2008 at 11:45 am
Does anyone redo the rebounding stats to eliminate rebounded missed free throws?
Seems like that’s a useless indicator. Generally uncontested rebounds.
dustin // May 16, 2008 at 1:08 pm
Goldshamm,
One of the principles behind win score is the ability to calculate based on box score statistics and simple math (all you need to be able to do is add and subtract, you don’t even need to know division to divide something by two). There are likely a number of ways it could be improved, but it would be moving away from the simplicity principle.
merl // May 16, 2008 at 3:01 pm
Because the second reference point was always wins produced, isn’t this just a post about players that wins produced likes much less than the other metrics you are listing?
merl // May 16, 2008 at 3:03 pm
I mean, without referring to any statistics I could have suggested this heuristic:
Athletic scorers who are usually the focal point of their team’s offense, shoot a poor percentage and rebound below their position average.
Andrew G // May 16, 2008 at 8:56 pm
@Merl: Yeah, and those players aren’t valuable, especially compared to “efficient scorers who shoot a good percentage who rebound above their position average.”
mrparker // May 17, 2008 at 9:47 am
Westy,
I guess the olympic team will be a good indicator of wp48’s value this summer.
avg wp48 has been a pretty good indicator of efficiency differential.
I see that you mentioned Dean Oliver’s stuff. If you look at the formula’s for how he calculates ratings they seem to like the same players that wp48 likes. Now that basketball-reference has its win shares above average(wsaa) metric, which is completely derived from Dean Oliver’s off and def ratings, that is much easier to see.
Top 20 players by wsaa
——————————
1. Chris Paul-NOH 22.5
2. LeBron James-CLE 19.5
3. Amare Stoudemire-PHO 19.0
4. Kevin Garnett-BOS 18.0
5. Chauncey Billups-DET 16.5
6. Paul Pierce-BOS 15.5
Dwight Howard-ORL 15.5
Manu Ginobili-SAS 15.5
9. Kobe Bryant-LAL 15.0
10. Dirk Nowitzki-DAL 14.0
11. Tim Duncan-SAS 12.0
Tyson Chandler-NOH 12.0
13. Lamar Odom-LAL 10.5
14. Ray Allen-BOS 10.0
Shane Battier-HOU 10.0
Marcus Camby-DEN 10.0
17. Andris Biedrins-GSW 9.5
Carlos Boozer-UTA 9.5
Yao Ming-HOU 9.5
Steve Nash-PHO 9.5
top 20 players by wins produced
—————————————–
1. Chris Paul 25.4
2. Dwight Howard 24.6
3. jason kidd 21.3
4. Marcus Camby 21
5. Lebron James 20
6. Tim Duncan 18
7. Garnett 17.9
8. Steve Nash 16.7
9. Tyson Chandler 16.3
10. Amari Stoudamire 16.3
11. Kobe Bryant 16.4
12. Chauncy Billups 16
13.Ginoboli 15.5
14.Boozer 14.8
15.Nowitski 13
Pierce 13
Jefferson 13
18.Jamison 11
Iguadala 11
Baron Davis 11
What Jumps out at me is Jason Kidd being off of the list completely. I looked up his wsaa number and it is -.5. That is astounding that wow’s 3rd ranked player finished off of the list. However, if you look up just his numbers in Dallas he scored a 3.5 over a 29 game span. Prorated out to a fuller season he approximates closer to what Steve Nash’s numbers are.
This is a tangent. I set out to show that Dean Oliver’s numbers like the big time defensive rebounders as much as Berri’s wow stats. All of the wow favorites(underrated non-scorers like Tyson Chandler) are on the wsaa list as well.
Some the wsaa win totals match up really well with wow win totals. However, the big time rebounders are getting more credit from Berri then they are from Oliver. I hypothesize that this has mostly to do with Oliver discounting off rebounds while Berri does not.
ilikeflowers // May 17, 2008 at 12:18 pm
In terms of producing wins, isn’t a defensive rebound worth the same as an offensive rebound? After all a defensive rebound deprives your opponent of an offensive rebound and vice versa so they must have the same value. I wonder about this because the supposed over-valuation of defensive rebounds is often presented in the comments here as a weakness of W48. What am I missing? Can someone explain to me how just defensive rebounding can be overvalued without offensive rebounding being overvalued as well?
Very Rated « Working’s For Suckers // May 17, 2008 at 1:30 pm
[...] Posted in conversations I’ve had, sports by workingsforsuckers on May 17th, 2008 This post at the Wages of Wins brought back some fond memories of a debate between my good friends Dave and [...]
mrparker // May 17, 2008 at 2:52 pm
Ilikeflowers,
1 player can grab 3 offensive rebounds in 1 posession. In this respect not as much is gained from an offensive rebound as from a defensive rebound. A defensive rebound causes a change of posession.
ilikeflowers // May 17, 2008 at 6:54 pm
Each offensive rebound prevented a defensive rebound though.
mrparker // May 18, 2008 at 7:37 am
Ilikeflowers,
While thats true, an offensive rebound does not necessarily convert to an offensive success. A defensive rebound = a defensive success.
ilikeflowers // May 18, 2008 at 9:52 pm
Why would that matter regarding valuation of rebounds in terms of wins produced?
Westy // May 19, 2008 at 9:44 am
Thanks for posting those, mrparker! Very interesting. And indeed they do somewhat closely agree. (I wonder what the correlation between them would be? An interesting table would be the correlations between PER, NBA Efficiency, WSAA, and WP.) I for one would say that WoW does a pretty good job ranking the players, but I don’t think it’s perfect, and so I do wonder if it slightly overrates heavy rebounders who are efficient low volume scorers. In terms of rebounding, I would actually argue that defensive rebounding should be discounted more than offensive when looking at the individual player level. (does DO discount offensive and not defensive, mrparker?)
As merl pointed out, a portion of DR’s are from missed free throws. In addition, they are the end result of a defensive effort that seems to be teamwide rather than individual.
ilikeflowers // May 20, 2008 at 9:59 am
The missed free-throw defensive rebounds (MFTDR) would indeed seem to overvalue defensive rebounds when comparing those players who have the best position along the lane (C and PF). It doesn’t sound like it would impact defensive rebound valuation for the other positions nearly as much (and might even indicate that they are undervalued for those positions).
Once MFTDR’s are removed from the C and PF individual assignments are the remaining non-MFTDR’s properly valued? In other words do C and PF MFTDR’s account for the statistical results which seem to indicate that defensive rebounds are overvalued across all positions?
The Underrated in 2007-08 « The Wages of Wins Journal // May 24, 2008 at 8:54 pm
[...] 24, 2008 · No Comments Who is the most “underrated” player in the NBA? As I noted a few days ago, the answer to this question requires two metrics. The first metric should capture popular [...]
Nets Fan // May 26, 2008 at 8:21 pm
You’re bias against Jefferson is ridiculous…
At Least We Have the Bucks « The Wages of Wins Journal // July 31, 2008 at 10:16 pm
[...] May I identified Jefferson as the Most Overrated Player in the NBA. I have also noted that the decline of the New Jersey Nets can be directly tied to the [...]
Todd Kelly // February 24, 2009 at 5:22 am
It pisses me off that I can’t figure out what wins produced means.
Who was the Most Overrated in 2008-09? « The Wages of Wins Journal // July 26, 2009 at 11:06 am
[...] Joining Harrington on both lists are Andrea Bargnani, Jeff Green, O.J. Mayo, Richard Jefferson [most overrated in 2007-08], Rudy Gay, and Stephen [...]