Roger Clemens, Steroids, and Bringing People Together

Did Roger Clemens take steroids?

The following paragraph appeared in the New York Times this Sunday:

Last week, Roger Clemens made the rounds on Capitol Hill to rebut charges by Brian McNamee, his former trainer, that he used steroids and human growth hormone late in his career. In addition, Clemens’s agents from Hendricks Sports Management have provided a report loaded with numbers – 45 pages, 18,000 words and 38 charts – to support his position. You can find the report at the Web site http://www.rogerclemensreport.com/.

This was the opening paragraph in the following Keeping Score column authored by Eric Bradlow, Shane Jensen, Justin Wolfers and Adi Wyner.

Report Backing Clemens Chooses Its Facts Carefully

These four professors from the Wharton School argued that the stats – contrary to the assertions of Hendricks Sports Management — do not show Clemens is innocent.  And this point was noted at ESPN.com in the following articles.

Report: Penn professors’ findings contradict Clemens’ analysis of career stats

Professor: Stats show something unusual happened in Clemens’ career

While Wolfers and company were making their argument, JC Bradbury at Sabernomics appeared to be telling a different story.  Bradbury argued in the following columns that the evidence did not indicate that Clemens was guilty of taking steroids.

How Did Clemens Age Relative to Other Pitchers?

Official Clemens Response to the NY Times Article

A Critique of the Clemens Report

So here we have two of my friends appearing to have a very public disagreement.  And this led me to think of my role in life as a uniter (yes, I have always thought of myself as a uniter, not a divider). :) So last night I sent the following e-mail to both Bradbury and Wolfers.

Would each of you agree with the following statements?

Justin and company are arguing that the statistics do not show Clemens is innocent.

JC is arguing that the statistics do not show that Clemens is guilty.

Both Bradbury and Wolfers graciously responded to my inquiry.  And each also agreed to let me post their responses.  In alphabetical order, here is Bradbury and Wolfers.

JC Bradbury’s Response:

I agree that the statistics cannot exonerate Roger Clemens nor any other baseball player accused of using steroids. I also think they cannot convict. Alan Schwarz had a nice piece on this in Sunday’s New York Times. What we can do is look at certain metrics to see if the evidence leads us one way or the other. In Clemens’s case, especially considering the specificity of Brian McNamee’s allegations, I don’t think swings in the data support the current allegations.

Clemens is one of the best pitchers in baseball history and he has aged well. But, I think the aging pattern is more consistent with someone who changed his style to compensate for the effects of aging. When I look at pitchers, I like to break down their stats into three components—strikeouts, walks, and home runs—that are not polluted by the contributions of fielders. Also, I would expect steroids to impact these areas performance measures differently. The effects of anabolic steroids ought to improve strength, speed recovery, and increase aggression; therefore, I expect the largest impact of steroid use would be visible in strikeouts.  This effect also might manifest itself in walks if the drugs allow the pitcher to pitch within the zone more often.  Home runs ought to improve, but given the variance of home runs, changes in this area are the most difficult to interpret. And in Clemens’s case, this metric doesn’t seem to yield much information, so I will ignore it here.

The path of Clemens’s strikeout rate relative to the league average was in decline for the latter part of his career. There are two distinct jumps in this area: 1996–1998 and 2002 (maybe 2004, too). The only one of these jumps that fits with McNamee’s story is 1998. However, Clemens was already having a good strikeout season before McNamee alleges that Clemens became interested in using steroids. In fact, his strikeout performance looks quite similar to his excellent 1998 and 1997. This is an odd time for Clemens to decide that he wants to use steroids, but I can only guess at what might motivate him. Clemens did have a good second half of the season, but the change in performance is explainable from natural swings in performance based on his prior performance.  The spike in 2002 occurs while McNamee is still Clemens’s trainer, yet he testified that Clemens was no longer using. In terms of ERA, 2002 was not a good year for Clemens because his walks and homers allowed were up.

When we get to walks is when the story gets interesting. His aging pattern is in reverse. He starts out good, gets worse into his mid-30s, and then improves. Much of his success late in his career is because he stopped walking batters. And given that his strikeout rate was not improving, this leads me to believe that this reflects a change in the style of his pitching. It could the result of his work on improving his split-finger fastball, which is one of the explanations for his success put forth in the Clemens report, or possibly he was using his veteran knowledge to better exploit the strike zone.

We also have the issue of the sample. I don’t think there is any denying that Clemens performance is atypical. Very few pitchers are effective into their 40s. But, it is not so atypical as to be unexpected by natural means. Including the comparable pitchers listed in the Clemens report, you could add John Smoltz and Jamie Moyer, who have excelled in the latter halves of their careers. I don’t think it is wrong to compare Clemens to these pitchers. All the Clemens team needs to show is that his performance is not without precedent.

Finally, the good ERA years of 2004–2006, which are the post-40 performance spikes that initially made everyone so suspicious of Clemens, occurred while McNamee is still his trainer (and claims Clemens was clean), and after MLB had instituted random drug testing.

So, to put it in Justin’s terms, I think the evidence supports the defense. Of course, given that my opinion is based heavily on the testimony of Brian McNamee, I may change my opinion as new facts come to light.

Justin Wolfers Response:

I think that you have characterized our views correctly: We are arguing that Clemens’ career statistics do not show him to be innocent.  My colleagues and I wrote our piece because we were thought the attempt by Clemens crisis management firm to spin the data were dishonest.  This is not a case where the data can speak clearly enough to prove guilt or innocence, and this is not a point that the earlier sympathetic press coverage spoke to.  As economists and statisticians, we thought this was a nice real world example we could use to help the public understand just what can and can’t be shown with statistics.

So I think you are right to say that JC and I don’t disagree (or don’t disagree much).  Beyond what the data don’t “prove” (both guilt and innocence), there is a tougher intermediate question: Are Clemens’ career statistics better thought of as evidence for the prosecution, or evidence for the defense?  We see enough unusual patterns in his career trajectory that we think of them as being more persuasive for the prosecution than for the defense.  Different approaches yield slightly different conclusions, but enough of them look somewhat odd that it is hard to see an honest presentation of the data helping Clemens’ case.  In fact, I was a bit surprised that Clemens’ folks opened this can of worms. 

Of course, it is an old advocacy trick to simply try to bury an issue under a mountain of statistics, hoping that journalists are not well enough trained to dig deeper.  There is so little in the Clemens Report that is actually analytically useful (read it and see!), that I’m pretty sure this is what was going on.  (The fact that his agents are working with a PR crisis management firm on this issue largely confirms this.)

Summarizing Bradbury and Wolfers

So there you have it.  Two academics looking at the same issue, and although at first glance it may not appear to be the case, each is reaching similar conclusions.  The reason for the difference is that Bradbury and Wolfers, et. al, are addressing different audiences.  Bradbury is concerned about people in the media and the general public who have already convicted Clemens.  To these people Bradbury says “not so fast, the statistical evidence does not convict.” 

Wolfers et. al. are addressing paid consultants, who wish to say the statistical evidence clears their client.  To these people, Wolfers, et. al. says “not so fast, the statistical evidence doesn’t clear Clemens.”

Painting a Broader Picture

I would like to emphasize something that Wolfers stated. As economists and statisticians, we thought this was a nice real world example we could use to help the public understand just what can and can’t be shown with statistics.

Let me generalize beyond the work of the Hendricks Sport Management report. Consultants (and some journalists) frequently offer black and white answers.   With respect to consultants, this is understandable.  People do not like to give money to consultants who say “well, the answer depends on your perspective.”  Definitive answers, even if the stats don’t support your level of certainty, keep the money flowing. 

So when you hear phrases like

“We KNOW this to be true” or

“We KNOW this is not true” or

“Our work is on the cutting edge and cannot be refuted.”

you should be a bit suspicious. 

Statistics are not quite this definitive.  Academics understand that statistics only “suggest” interpretations, and the interpretations depend upon the question being addressed.  Furthermore, there are often different points of view so it’s always possible that what you say could be refuted.  And that the refutations can also be refuted.

Does this mean that you can use statistics to say anything you want?  No, but it does say that you have to be careful when you see consultants offer “definitive” answers.  Often when you delve a bit deeper, you see that the consultant is just spinning the data.  And if your not careful, all that spinning can make you just a bit dizzy.

– DJ

Posted in: Baseball Stories