A few days ago I was reading Marginal Revolution and I came across the following: Pinker reviews Gladwell.
A few months ago I saw Steven Pinker – a Harvard psychologist — in a rather lengthy interview on C-SPAN. After the interview I ordered Pinker’s latest book: The Stuff of Thought: Language as a Window into Human Nature. So I was familiar with Pinker’s work and was curious to see what he had to see about Gladwell.
The above link takes one to the New York Times review Pinker wrote of Gladwell’s latest book: What the Dog Saw. The review begins by praising Gladwell’s gifts as a writer. But it soon takes a turn as Pinker begins to argue that Gladwell – a journalist – is often not an “expert” on the topics he writes about. My reaction when I read this was that this seemed to be an unfair criticism. Journalists – as anyone whose work has been discussed by members of the media would know (and Pinker falls into this group) – do not claim to be “the expert.” In fact, this is why the call on people like Pinker. In other words, if journalists were the experts, they could skip the practice of talking to people like Pinker entirely.
Pinker, though, had apparently not considered this point and quickly moved on to an effort to illustrate Gladwell’s perceived shortcoming. What the Dog Saw is a collection of essays Gladwell originally wrote for the New Yorker. And part of this collection is the following article Gladwell published last year: Most Likely to Succeed: How do we hire when we can’t tell who’s right for the job? Readers of this article will discover that part of the story Gladwell tells focuses on how hard it is to draft a quarterback in the NFL. The “experts” Gladwell called upon to tell this story were two economists named David Berri and Rob Simmons.
In an article Rob and I published in the Journal of Productivity Analysis, we discuss how the statistical relationship between a quarterback’s draft position and his future performance is quite weak. Pinker, though, apparently disagrees. Nestled in his New York Times review is the following: It is simply not true that a quarterback’s rank in the draft is uncorrelated with his success in the pros….
Upon reading this I went from “Hey, I think Pinker is being unfair to Gladwell” to “Hey, I think Pinker just attacked my research.” So I decided to e-mail Malcolm and ask if he knew of Pinker’s source for his statement. The question was mostly a joke. My thinking was that Pinker’s reaction was simply a manifestation of the problem sports economists often face. Sports are a subject matter with an abundance of “experts”. Essentially, many people who have only watched sports often believe that they are an “expert” on the subject. Having conducted research on sports and economics since the mid-1990s, I am accustomed to people who have only watched sports disputing the findings of my co-authors and I. So I was thinking along the lines… maybe Pinker is a fan of the Jets, and he really hoped Mark Sanchez was going to live up to his lofty draft status.
Pinker’s response to Malcolm, though, revealed that Pinker was not just stating his belief as a sports fan. Pinker claimed he actually had sources. Specifically — as Malcolm detailed at his blog— Pinker found some stuff on the Internet that contradicted what Rob and I said in our article.
My sense is that Pinker never read our article. What he did find on the Internet is evidence that a quarterback’s aggregate performance (i.e. passing yards, seasons played, Pro Bowl appearances) is indeed related to draft position. And as Rob and I detailed in our article, this is true. Aggregate performance and draft position are statistically related. But as Rob and I argue, this is because in the NFL (like we see in the NBA) draft position is linked to playing time. And this link is independent of performance. In fact, Rob and I find that draft position – again, independent of performance – impacts a quarterback’s pay many years into a quarterback’s career.
To correct for this bias, we focused on per-play statistics. And here is a sample of what we found. After a quarterback has played five seasons in the NFL (minimum 500 career plays), here are the correlation coefficients between draft position and various career statistics:
Completion Percentage: -0.01
Passing Yards per Pass Attempt: -0.02
Touchdowns per Pass Attempt: -0.12
Interceptions per Pass Attempt: 0.00
QB Score per Play: -0.01
Net Points per Play: -0.02
Wins per Play: -0.02
QB Rating: -0.06
Our data set runs from 1970 to 2007 (adjustments were made for how performance changed over time). We also looked at career performance after 2, 3, 4, 6, 7, and 8 years. In addition, we also looked at what a player did in each year from 1 to 10. And with each data set our story looks essentially the same. The above stats are not really correlated with draft position.
But that is not all we did. Rob and I also looked at what factors determine where a quarterback is selected in the draft. We then looked at how the factors that determine draft position predict performance. This study revealed that the factors that get a quarterback drafted are not related to what he does in the NFL.
Again, I don’t think Pinker read our paper before commenting. And in this sense, he did the same thing he wants Gladwell to stop doing. Pinker commented on a subject where he was not well-informed. But the difference is this… Gladwell makes an effort to understand the subjects he discusses. Pinker didn’t make this same effort.
Beyond this point, let me also comment on the tone of Pinker’s critique (which is similar to the tone of many comments at Gladwell’s blog). Many people seem to want Gladwell to only write about “the truth.” In other words, people want Gladwell to focus on research that everyone agrees upon. But for research to be “interesting” it must say something new. And more often than not, the “new” research will contradict “old” beliefs. Consequently, people who liked the old stories will disagree. So when Gladwell talks about Allen Iverson not being a great player, or Kevin Garnett being underrated, or the inability of draft position to tell us much about a future quarterback’s performance, people who hold the “old” beliefs will disagree (and argue the “new” research is clearly bad). According to Gladwell’s critics, Gladwell should see this disagreement and find something else to write about.
But that is not a particularly reasonable approach. Again, “interesting” research will challenge people. Such challenges often provoke negative reactions. To only choose topics that will avoid such reactions reduces your range of topics to a number close to zero.
One last note… Martin Schmidt and I have a book coming out next March called Stumbling on Wins. This book will provide more details on quarterbacks and the NFL draft, and also discuss a host of other decisions made in sports. If you enjoy this story about the NFL draft, we hope you will enjoy our next book. Of course, if you didn’t like our draft story…. well, we are sure you will love all the other stories we tell :)
– DJ
The WoW Journal Comments Policy
Wages of Wins Journal 
simon
November 19, 2009
Thank you for the post. I see that you didn’t mention Steve Sailer at all. ;)
On the other hand, the Freakonomics people are getting heavily criticized for taking a contrarian view on the climate change issue in their new book. Then again it does seem they repeat a lot of old discussions (i.e. cooling theory, etc) that have already been done to death while ignoring the retorts readily available.
JAW
November 19, 2009
DB, do you mean that “We have no statistical evidence to claim that playing time is related to performance?”
Obviously Jamarcus Russell is a player who got more of a shot because of his draft status than other players would, and this is the case for a number of players.
At the same time, baseball is clearly a sport where players don’t get the same chances that football players do. They almost always have to show they can perform at lower levels before they get to perform in the major leagues. Baseball quite clearly exhibits a relationship between draft spot and performance at the major league level. Perhaps talent evaluation is just that much worse for quarterbacks, but I find that at least it colors my analysis. I’m perfectly willing to believe “the null hypothesis is not disproven” but I’m not completely willing to believe it means the opposite.
Kevin
November 19, 2009
Prof Berri, do you consider Sailer’s core argument specious? He claimed that because your admission criterion omitted quarterbacks drafted in later rounds who failed to obtain significant playing time, perhaps because of mediocrity, you were selling the true correlation short.
In other words, to meet your criteria, the QB already had to be pretty good OR highly invested in; effectively your sample was biased to include both good and bad high draft picks ( your claim that high draft picks who are bad still get to play IS true; teams have invested too much to let them go), but only good later round picks were included in your sample. So you don’t take account of the rather large amount of poor QBs taken in later rounds. Your selection criteria created the weak correlations.
Restated, the claim goes that what you really discovered is that team’s, for various reasons, teams force themselves to play bad high round draft picks, but are better about letting lower round draft picks go, feeling no aversion to losing their services. Bud Adams spent all that money so Vince Young had better play! That’s what your data really say. Your finding might be better framed as a sort of loss aversion, or an interaction between talent and degree of investment which creates an appearance of ignorance or at least irrationality.
It’s great stuff either way, so please don’t get defensive. Seriously, I just want to know what you think about the possibility. Is the alternative hypothesis true, not true, or possible?
Kevin
November 19, 2009
Sorry, 3rd paragraph, ‘high’ and ‘low’ is confusing:
What you really discovered is that teams force themselves to play bad early round picks and are better about letting later round picks go…
Italian Stallion
November 19, 2009
D.Berri,
“Essentially, many people who have only watched sports often believe that they are an “expert” on the subject. Having conducted research on sports and economics since the mid-1990s, I am accustomed to people who have only watched sports disputing the findings of my co-authors and I. ”
IMO, this is the kind of comment that invites the critics and contrary opinions.
There is no question that the analysis of statistics done properly and in an unbiased fashion (by someone like you) is a powerful way of evaluating players.
However, I think even you will concede that you can see some things that as of now aren’t reflected in the “available stats”.
Of course, seeing those things and understanding them is also a skill. Just as there are some lousy statisticians, there are scouts, fans, former players etc., that watch games that aren’t too good at that. However, some people are highly skilled at it.
What you said sounded almost the same as a former player showing disdain for everyone that uses stats because they have never actually stepped onto a professional court or field.
So how could they understand everything that’s going on in the game?
Well, we know that they can understand a real lot.
M Stein
November 19, 2009
Steven Pinker replies:
What Malcolm Gladwell calls a “lonely ice floe” is what psychologists call “the mainstream.” In a 1997 editorial in the journal Intelligence, 52 signatories wrote, “I.Q. is strongly related, probably more so than any other single measurable human trait, to many important educational, occupational, economic and social outcomes.” Similar conclusions were affirmed in a unanimous blue-ribbon report by the American Psychological Association, and in recent studies (some focusing on outliers) by Dean Simonton, David Lubinski and others.
Gladwell is right, of course, to privilege peer-reviewed articles over blogs. But sports is a topic in which any academic must answer to an army of statistics-savvy amateurs, and in this instance, I judged, the bloggers were correct. They noted, among other things, that Berri and Simmons weakened their “weak correlation” (Gladwell described it in the New Yorker essay reprinted in “What the Dog Saw” as “no connection”) by omitting the lower-drafted quarterbacks who, unsurprisingly, turned out not to merit many plays. In any case, the relevance to teacher selection (the focus of the essay) remains tenuous.
khandor
November 19, 2009
David,
If I understand what’s written here correctly …
The question I have related to the performance of NFL quarterbacks and their draft position is a simple one:
What’s the reason you [or anyone else] would conduct academic research on that subject and choose not to include the data for each and every QB drafted from 1970 to 2007?
Thanks, in advance.
Aon
November 20, 2009
Gladwell is a bullsh*t artist. Anyone who is a fan of his ‘work’ is just showing how dumb of a person they are. Gladwell’s next book should be called “Stretch”, b/c that perfectly describes most of his arguments and ideas.
Steve Sailer
November 20, 2009
What Berri is doing, in effect, by using his “per-play” measure is comparing quarterbacks taken at the top of the draft (most of whom get a lot of plays in the NFL) to those taken lower in the draft who turned out to be surprisingly better than expected, and thus get a lot of plays. He’s essentially leaving out of his analysis all those lower drafted quarterbacks who turned out to be as mediocre as expected and thus didn’t get many plays. In other words, his methodology is pre-rigged to produce the conclusion that Malcolm likes.
Through 2008, among quarterbacks drafted from 1980-1999, top ten draftees averaged 2,975 pass attempts in their careers. Quarterbacks drafted 11th to 100th averaged 1,470 attempts, a little less than half as much. And quarterbacks drafted 101st or higher averaged only 387 attempts.
So, Berri is more or less throwing away the lousier half of the sample of quarterbacks drafted 11th-100th (and totally ignoring all the quarterbacks drafted after 100) and comparing them to all the quarterbacks drafted in the top ten.
When you actually count everybody drafted, you get the following figures for career yardage (through 2008):
Drafted
Mean Yards Median Yards
Top 10 20,296 18,148
11-100 10,099 3,881
101+ 2,614 0
The differences between the mean and the median (50th percentile) point out that the higher drafted players tend to be safer bets. The quarterback at the 50th percentile among the top ten draftees of his year goes on to have a fairly impressive NFL career, throwing for 18,148 yards. (The median top ten quarterback of 1980-1999 in career yardage was Jim McMahon, who led the Chicago Bears to the 1985 Super Bowl title.)
In contrast, the 50th percentile of the 11th to 100th picks of his year only accumulates 21% as much career yardage. The median quarterbacks of the 11-100 group are Mark Herrmann and Chuck Long.
And the 50th percentile of 101st plus picks never completes a pass in the NFL).
So, the top ten quarterbacks drafted in the eighties and nineties tended to be safer bets, which has its value. (General managers in this decade, however, might have gotten overconfident from a pretty decent run of luck with high draft pick quarterbacks in the two previous decades.)
On the other hand, there are lots of diamonds in the semi-rough of the 11-100 group, such as Brett Favre, Dan Marino, and Boomer Esiason. And in the 101+ group, there are diamonds in the real rough like Mark Brunell, Trent Green, and Matt Hasselbeck. (And that’s not to mention the undrafteds, like Kurt Warner.)
http://isteve.blogspot.com/2009/11/gladwell-strikes-back.html
reservoirgod
November 20, 2009
Prof:
I’m dying for a football post on QB Score & RB Score. There’s so many interesting stories – Peyton vs. Brady (20 HoF QBs chose Peyton), Favre, Brees, Chris Johnson vs. Adrian Peterson. I’ve been reading Football Outsiders, but I would like to hear your opinion (or read your numbers) about this NFL season.
montestewart
November 21, 2009
Steve Sailer seems to be saying that the analysis in Gladwell’s article is rigged to exclude lower drafted QBs that performed poorly, thus assuring the desired outcome. If this is true, it calls the results into question, and it calls for an answer.
Maybe I’m reading it wrong, but doesn’t Sailer’s response have its own question marks? It looks like he’s ignoring the per play approach and continuing to apply the career aggregate approach that the article was attempting to discard in an attempt to remove playtime bias from the analysis.
Why not simply apply per play analysis to all QBs, drafted or undrafted (if they ever signed and played)? Why is everyone hiding the ball?
As too journalists not being “experts,” “experts” are experts in their field, and unless it’s writing or some other form of communication, they are not “experts” at communicating their expertise. It’s a very hierarchical world. Let’s not rock it.
Alex
November 21, 2009
Not as fancy as what Dr. Berri did, but: I got the career numbers and draft position for every quarterback drafted between 2000 and 2006. This did not include sacks or fumbles. The data set has 95 quarterbacks. If you look at every quarterback, there is a strong negative correlation between draft position and completion percentage, passing yards per attempt, passing TD per attempt, interceptions per attempt, and a rough estimate of QB score per play. But, it’s likely not significant for interceptions (just looking at R squared). If you take out the 24 quarterbacks who never threw a pass, the correlations become weaker and less significant. If you further limit it to the quarterbacks who attempted more than 100 passes, the only significant correlations are passing yards per attempt and pseudo QB score, both of which are about -.37. The R squareds are only around .14. So to the extent that this would hold up with a larger sample size, it seems that quarterbacks drafted in later rounds perform worse across the course of their careers.
montestewart
November 21, 2009
An inverse correlation between QB draft position and performance is intuitive.
I’ve heard arguments that many QBs need real game PT to develop. I’ve also heard arguments that a high draft position commits a team to a greater degree as far as money, roster space, time, public image, etc. So the QBs with few or no pass attempts may be a mixture of lack of ability and lack of commitment to allow them to develop. After these are removed from the equation, the negative correlation is not as great as I would have expected.
Steve Sailer
November 21, 2009
By eliminating all the drafted quarterbacks who never showed themselves good enough in practice to get 500 pass attempts in the NFL, Dr. Berri is imposing severe restriction of range on his analysis.
Steve Sailer
November 21, 2009
Also, high draft choice quarterbacks are much more likely to be thrown into starting jobs before they are mature. And they are likely to be forced to start for bad teams, since bad teams get most of the top ten draft choices.
For example, Peyton Manning was drafted by 3-13 Colts, who immediately made him their starter at age 22. His rookie year, he threw 28 interceptions and the Colts went 3-13 again. So, that year, which almost nobody except a very high draft choice would have had to endure, lowered Manning’s average per play statistics.
In contrast, Matt Cassel sat around on the bench, until, at his physical and mental peak age, and in good health because he hadn’t been absorbing punishment in games since high school, he was suddenly plugged in for Tom Brady into a superb offense. And his per play numbers were very good.
So, after one season of starting 7th round draft pick Matt Cassel’s per play numbers were a lot better than Peyton Manning’s.
There are a lot more examples of how being drafted early in the first round can have a negative effect on a quarterback’s per play statistics.
montestewart
November 21, 2009
The Manning-Cassell comparison illustrates how hard it is to come up with a perfect formula, and it does not even account for the team investment cost in drafting a late rounder and keeping him around for development purposes, or that Cassell came into a much better team.
A hard-to-prove (impossible-to-prove?) premise is that the late rounder may enter the lineup as a more physically and mentally polished player, or he may never be given a chance to show his skills. As a late round pick, there is much less risk in cutting him loose. (Of course, he could be signed elsewhere and explode, and wouldn’t that make the the old team look great?)
It would be interesting to compare early picks (ones that were presumptively starting or fighting for the position) against later picks that were forced into starting early in their careers, through injuries or other circumstances.
In any field, there’s frequently a suspicion (I’m sure it’s frequently true) that some near the top don’t merit such a lofty position, and some closer to the bottom deserve a higher position. It’s not always so easy to show the size of these two groups through numbers.
Regardless of exceptions, it seems to me that most of the truly great QBs were recognized (based upon draft position) as having that potential when they came into the league.
Ty from Bucks Diary
November 22, 2009
DB,
You’re on a roll… great post.
I had read that book review as well. Do you have any insight into Pinker’s other claim that Gladwell’s major contention in “Outliers”, namely that IQ does not correlate with professional success, is bogus? That was one of the major points I took away from “Outliers”. I think I’ve been repeating it too. I didn’t read Gladwell’s endnotes, I guess I just assumed he wouldn’t pull something like that out of his ass… he’s generally quite reliable and always interesting.
Do you have any thoughts on that?
Rumblebuckets
November 22, 2009
1) Not understanding what he writes about is a fair criticism of Gladwell. For example, he writes about Cezanne, but he obviously has no eye for art, nor does he care to, he does like the Bougie stories that have been created around these men to sell their paintings though, and consequently he equates what a masterpiece is to its value on the market. But what makes a masterpiece is that it touches upon beauty, is that it gets you to the point in reality where you actually are using your eyes, where you feel the painting as it moves in you, where you are really seeing, which as Valery states, “is to look without naming.”
2) He almost purposely makes misleading statements. Picasso’s works in his mid-60s aren’t valued that highly, he says. But when was Guernica painted? 1937. Picasso was 56, not exactly a spring chicken. The Charnel house was painted eight years later. So while Picasso did paint a lot of great works when he was young, he continued to make great work into his 60s. Now, of course he did slow down, but of course he’s cherry picked Picasso. Look at Anselm Kiefer or Nic Carone if you want contemporary artists, or Peter Agostini if you want a forgotten contemporary artist, or any number of old master painters like Rembrandt (who was dead before Picasso stopped making great paintings), or Michelangelo or Titian, who famously on his death bed claimed, “But I was just beginning to learn how to paint.” He was 90, and what’s more one can see that what he says is true. So making great paintings or sculptures when you are old is not so rare for an artist. Of course, these guys also made great paintings and sculptures when they were young.
3) That Cezanne didn’t make great paintings when he was young is also just simply not true. He didn’t make as many of them, as he liked the Bohemian lifestyle, but what he made was still great, if by great we mean that they convey emotion, that they are alive. Of course, he didn’t yet paint in that Cezanne style that he came to later, which is why they don’t cost as much. But you can find a number of abandoned works from his later years that don’t show that much. Hell, the Barnes collection has quite a few of them (As well as a fake Renoir.) Of course, there are those of which we can hardly say a word as well. So the entire conceit of the article is wrong. Cezanne did make great paintings when he was old, yes, but he also made bad paintings when he was old, and great paintings when he was young.
4) His experts often aren’t experts (and this is still talking about the Cezanne article.)
“Galenson argues, rarely engage in that kind of open-ended exploration. They tend to be “conceptual,” Galenson says, in the sense that they start with a clear idea of where they want to go, and then they execute it. “I can hardly understand the importance given to the word ‘research,’ ” Picasso once said in an interview with the artist Marius de Zayas. “In my opinion, to search means nothing in painting. To find is the thing.” He continued, “The several manners I have used in my art must not be considered as an evolution or as steps toward an unknown ideal of painting. . . . I have never made trials or experiments.”
Picasso may have said that, but what does the first quote mean? It is a Picasso-ism, and the second is incredibly misleading because Picasso was not a stupid man, and he knows the public’s fascination with inspiration. But if that were true, how can you explain all the different versions of Guernica? Or all the different analytical cubist paintings in which he is slowly growing into something else, slowly changing, searching. Of course he abandons analytical cubism, either because he gets bored or because he knows it’s a better avenue for fame (or because he is searching.) I mean Picasso is art’s ultimate fame whore. And perhaps that is the reason he has the lack of talent to go on. Cezanne is not that. He’s the guy who went into the world every day to make a painting. Perhaps that’s the difference between the two and not some hackneyed difference between “conceptual” and “Experimental.”
5) Anecdotes just aren’t proof, even when they are approaching the truth of an event. They are illustrative and misleading, especially if you know don’t view them as an honest researcher, but someone who is just illustrating a point he wants to illustrate. This leads to any number of hilarious statements in this article, “The young Cezanne couldn’t draw.” Look at his academy drawings, and tell me he couldn’t draw. Of course he could draw. Now, this myth comes primarily from the fact that he got kicked out of the academy, but he got kicked out because he drew too well, with too much feeling (as did Van Gogh), he didn’t make the same cookie cutter, paint by numbers drawings as did everyone else.
6) And then his conclusions, I mean what do they mean: “This is the final lesson of the late bloomer: his or her success is highly contingent on the efforts of others. ”
That’s not true specifically about Cezanne, that’s true of everyone. Unless you are a subsistence farmer or hunting alone in the woods, your success is highly contingent on the efforts of others. Merely to eat a meal, even a crappy one. But also there is a serious conflation here (and Gladwell often does this), in that he changes what success means, from making masterpieces, to becoming recognized as a great painter. (Of course the conflation has a not so tenuous link to the way he defined masterpiece in the first place which allows this conflation to almost seemlessly happen, without the reader’s noticing.) But Cezanne would have been successful, if he never became famous, and it’s not like that sort of success helped him at all. He died before it happened.
Which of course begs the question, how could Cezanne be considered successful if the definition of painterly success is painting masterpieces, and a masterpiece is defined by how much it is worth, whatever the cut off, and his paintings were worth less in his lifetime the material cost to paint them?
And of course this is not an isolated case. But you know, he writes for the New Yorker. I think the problems with my encounter were perhaps my expectations.
7) Of course, his problems are some of your own as well. Of the statistics that are generally kept, you’ve certainly determined which ones are the most important, but you often make it seem as if they tell the complete story, when they only tell a large part of it. And your understanding of the why is generally lacking, because you come to them by way of reverse engineering, not by insight or intuition, but by looking at the correlation to end result (wins.) Of course, there is no inherent problem in that, it can even (And often does) enlighten you to things you would have never seen, or that people have learn how not to see (a bigger problem), but I think it does contribute to your lack of understanding in the way basketball works on some levels, like the fact that some players productivity does indeed hinge on other players. Or your implication earlier that basketball stats are better than baseball stats, your reasoning being that basketball players are more consistent and that you could better use them for predicative purposes, but baseball stats tell you so much more, they are so much closer to quantifying that game than the game of basketball, and so you can use them to much better analyze what actually happened, and understand the game by way of numbers, which is really what I think you are trying to do. You’ve just conflated understanding of the occurrence with the ability to predict outcomes, and while understanding of the what has happened does go into the ability to predict what will happen, they are not the same thing.
Steve Sailer
November 22, 2009
Dear Rumblebuckets:
Thanks.
khandor
November 22, 2009
Rumblebuckets,
1. 5) Anecdotes just aren’t proof, even when they are approaching the truth of an event. They are illustrative and misleading, especially if you know don’t view them as an honest researcher, but someone who is just illustrating a point he wants to illustrate. This leads to any number of hilarious statements in this article, “The young Cezanne couldn’t draw.” Look at his academy drawings, and tell me he couldn’t draw. Of course he could draw. Now, this myth comes primarily from the fact that he got kicked out of the academy, but he got kicked out because he drew too well, with too much feeling (as did Van Gogh), he didn’t make the same cookie cutter, paint by numbers drawings as did everyone else.”
Is YOUR use of this anecdote:
A. Proof
B. Illustrative and misleading
C. The Truth
D. Something else
E. All of the above.
[In my book, it’s E. … and, filled with tremendous irony. :-) 1. In general, “Qualitative Research” can make fine use of collated anecdotal evidence. 2. Bill Walsh said: “Look for and find playmakers. Then, the Art & Science of good coaching comes into play; transforming what may, otherwise, have been just an isolated event into a repetitive act.” 3. Those who are hung up by the small sample size associated with the truth contained in an isolated event, are wasting their time attempting to analyse Sporting Excellence with a Paint-By-The-Numbers approach.]
2. For the most part … I’m in agreement with your take in #7.
3. But what makes a masterpiece is that it touches upon beauty, is that it gets you to the point in reality where you actually are using your eyes, where you feel the painting as it moves in you, where you are really seeing, which as Valery states, “is to look without naming.”
re: a masterpiece
If you are truly touch with the subject … it IS something which you can FEEL and RECOGNIZE, right away … like the riff of this drum … without needing to crunch numbers.
Cheers :-)
Patrick Raulerson
November 24, 2009
I say Sailer/Pinker probably win this round (pending Berri’s rebuttal, so far conspicuously absent).
And Pinker is right about Gladwell, who often tries to pass off enthusiasm and cute connections as deep thought. Berri defends him by inventing straw men who protest that Gladwell should stick with the conventional wisdom and stop challenging everyone with his radical revelations about the New and Upsetting Truth, but that’s not what Pinker is complaining about at all – he’s saying (or at least suggesting) that Gladwell does a disservice to sincere, honest, competent intellectual inquiry by cherry-picking data and shaping stories to support a preconceived narrative that he thinks (quite correctly) will resonate with his readership. Gladwell, like most popular writers, is much more interested in selling books than divining the truth. Which is not to say Gladwell is always wrong – just that it’s dangerous to assume he’s correct, since being correct isn’t his chief motivation.
JAW
November 24, 2009
http://www.pro-football-reference.com/blog/?p=4740#more-4740
Draft position is not independent of performance.
Jim Glass
November 25, 2009
Essentially, many people who have only watched sports often believe that they are an “expert” on the subject.
As to this, my favorite words in the entire Belichick “4th & 2” broughaha were from ESPN’s Bill Simmons in the course of his extended tantrum blasting Belichick’s call, invoking his own superior authority…
“… and this is coming from someone who watches 12 hours of football every Sunday dating back to elementary school”!
;-)
That said, and as big a fan of this site and your analysis as I am in general, I just do not see the logic, when seeking the correlation between QB success and draft position, of excluding from the sample all the QB’s who do not succeed. Which is what is done by not considering QBs who don’t play five years (and who don’t get to play at all). How can that possibly not result in huge selection bias in the data?
I mean, if one looks just at All Pro QBs their stats would all be pretty close together, because they are all All Pro. But if one looks at how many came out of a top round draft pick and how many came out of the 6th round (Tom Brady), I would bet confidently without looking up the data that there are a lot more of the former (Manning, Palmer, Brees, Rivers, Ben R., ) per pick. Do we really conclude from this data that there is a lack of correlation between draft position and likelihood of success as an All Pro QB because Brady’s numbers are as good as the others?
I must be missing something here.
I must be missing more if the argument is that owners decide what QB will play by the amount of money invested in them at the draft. I mean, I understand that a QB with a lot invested in him is guaranteed a long look and some extra chances, compared to low-cost picks.
But the implied obvious “money” argument doesn’t seem to make sense in light of the fact that owners make *more* money from QBs who fill the stands with fans by flinging the ball around best — and more money *yet* from cheap low picks who do it. Tom Brady was maybe the the greatest money bargain in NFL history!
If 6th round picks like him really are as likely to succeed as 1st rd picks, why wouldn’t the owners be drafting all QBs in the low rounds and using top picks for where draft position does matter, like RB?
This seems to indicate truly massive irrationality on the part of owners, at great cost to them, with no evident explanation. Not at all like the comparably tiny “coaches punt too often on 4th down” inefficiency which is easily explained.
But I don’t doubt that I am misunderstanding something, and I haven’t read your paper any more than Pinker has, so I’m not arguing anything — just interested and curious.
Is your paper online anywhere?
dberri
November 26, 2009
Jim:
Not sure why people missed this paragraph (you are not the only one), but apparently they have (please look at the article again, the following sentences were always there):
“We also looked at career performance after 2, 3, 4, 6, 7, and 8 years. In addition, we also looked at what a player did in each year from 1 to 10. And with each data set our story looks essentially the same. The above stats are not really correlated with draft position.”
I will post again on this topic. Perhaps next week (after Thanksgiving).
Keane Wheeler
November 30, 2009
This is a fantastic discussion and has wider implications for talent identification programs.
Steve Sailer
November 30, 2009
Prof. Berri:
“Not sure why people missed this paragraph (you are not the only one), but apparently they have (please look at the article again, the following sentences were always there):”
I’m not sure why you are assuming your critics don’t understand your argument. They understand it; they just aren’t impressed by it for reasons stated in the comments above.
Steve Sailer
November 30, 2009
Jason Lisk on the blog of Pro-Football-Reference.com sums up his analysis of the Gladwell-Pinker battle royale:
“When we look at the top 20% of late round picks (those who are judged good enough to play or forced into action because of injuries) and they are roughly similar to the top 80% of high draft picks, that does not mean that late round picks are equal to early picks, and the NFL has a quarterback problem where nothing that happens before can predict what will happen in the future. Per play stats matter, and it’s important to look at quarterbacks from that perspective, otherwise we reward compilers who get opportunities without merit. More opportunities matter too, though, and a quarterback who plays well over a larger sample size is likely better than a quarterback who plays well over a small one, particularly when qb stats are more volatile due to outside factors such as teammates.
“This is a distinction that, as far as I can tell, Gladwell fails to grasp.”
http://www.pro-football-reference.com/blog/?p=4740#more-4740