The Federal Reserve and the Value of Scouting

Today’s column is truly about sports.  Specifically the role of “staring” in the evaluation of talent.  But I am going to begin in an unusual place and work my way back to the subject of talent evaluation in sports. And along the way, I will connect the Federal Reserve and the formulation of monetary policy to an argument made in The Wages of Wins.  

The Federal Reserve Comes to Bakersfield

This quarter at California State-Bakersfield I taught two courses, Intermediate Macroeconomics and the Economics of Religion.  In my macroeconomics course we used a simulation offered by the Federal Reserve of San Francisco.  For much of the quarter the SF Fed e-mailed my students links to information relevant to the setting of monetary policy.  And then this past week, representatives of the SF Fed came to Bakersfield.

Part of their visit was about running the simulation of the Federal Open Market Committee (FOMC) in my class.  Earlier in the quarter students had been divided into teams, with each team representing a member of the FOMC board.  On Thursday night, each team gave a five minute presentation describing the state of the economy and what monetary policy should be (specifically what should be done with the Fed Funds Rate).  The simulation, which is only offered by the SF Fed (not the other 11 Federal Reserve banks) was a smashing success.  Although I think I have some skills as a teacher, there is no way I could have hammered home the details and significance of monetary policy without this simulation.

Although helping me teach was one reason the Fed came to Bakersfield, the other reason was more important.  On Wednesday night I attended a small dinner party with many of the top business leaders in Bakersfield.  In essence, that dinner served as a scouting trip by the Federal Reserve.  And apparently, these scouting trips happen all the time.  Although the Fed has access to an abundance of macroeconomic data, and extremely sophisticated economic models (yes, even more sophisticated than the stuff we see in sports), the Fed still takes time to gather anecdotal data from business leaders.  This is necessary because there is a lag in data collection and analysis.  The Fed can tell you with great precision what happened last quarter or last year.  But what’s happening at this very instant is harder to pin down because the data hasn’t been collected yet.  Hence the need for scouting trips.

Staring in Sports

Okay, what does this have to do with sports?  The answer can be seen in two quotes from The Wages of Wins.  The first is from Chapter 10, where we discuss the merits of scoring, data evaluation, and “staring” (this excerpt was posted at wagesofwins.com in April of 2006).

We have shown that scoring by itself does not create wins. Yet many scorers, despite not offering many wins, are paid the most by their respective teams and are considered the key to whatever wins the team actually achieves. Sometimes this is not a problem. When the Spurs think their leading scorer, Tim Duncan, is their best player, they are right. When the Lakers thought that their two leading scorers, Shaq and Kobe, were their best players, they were right. When the 76ers, though, think their leading scorer, Allen Iverson, is their best player, the data tell a different story. Unfortunately, every year a team wins a title thinking their leading scorer is their best player; therefore, this particular mistake, without some statistical analysis, is quite difficult to see.

We would note that the focus on scoring may just be a way of simplifying the complex information talent evaluators face. The same abilities that allow a person to score in the NBA would also allow a player to rebound, generate steals, and create assists. Players like Jordan, Garnett, and Kobe are not just great scorers, but have athletic abilities that allow these players to accumulate rebounds and assists. So when people see an athlete who can score, a leap of faith might be made. If the player can score, he probably can do all the other things a team needs to win, and consequently significant scoring ability is seen as evidence that the player helps a team accumulate many wins.

Unfortunately there are clear exceptions to this simplification. When scorers do not actually contribute significantly to wins, teams often end up losing. Consequently, teams have turned to other explanations for why losses accumulate. Teams talk about the importance of coaching and team chemistry. And they add and subtract players, hoping to find the combination that works. Every year, one team hits the jackpot, while others keep searching. Unfortunately, without the proper statistical tools, many teams are left in the dark.

And this is a point we wish to emphasize. Without statistical analysis, one cannot see how the actions the players take on the court translate into wins. One can play basketball. One can watch basketball. One can both play and watch basketball for a thousand years. If you do not systematically track what the players do, and then uncover the statistical relationship between these actions and wins, you will never know why teams win and why they lose. Staring at these players play is not a method that will ever yield the answers that the proper analysis of statistics will yield. And this is true if you stare for one day, or as we said, if you stare for a thousand years.

The Value of Scouting

My sense is that some people look at the above quote and conclude that we think traditional scouting, which involves watching the players (or as we put it “staring”) has no value.  But if you look back in Chapter Seven of the book you see the following quote.

Before moving on, let’s observe that the Kobe and Shaq story highlights a key aspect of any statistical measure of productivity.  Kobe’s performance appeared to initially be negatively impacted by the loss of Shaq.  Meanwhile Shaq appears to be suffering at the hands of Father Time. These stories demonstrate that one cannot end the analysis when one has measured the value of player performance. Knowing the value of each player is only the starting point of analysis. The next step is determining why the player is productive or unproductive. In our view, this is where coaching should begin. We think we can offer a reasonable measure of a player’s productivity. Although we have offered some insights into why players are productive, ultimately this question can only be answered by additional scrutiny into the age and injury status of the player, the construction of a team, and the roles the player plays on the floor.

The data collected by the NBA – and yes, I mean the box score data – can give you a very good picture of “how” good a player has been.  As noted many times in this forum, if we know shooting efficiency, rebounds, turnovers, etc… we can estimate how many wins a player has produced.  But the data cannot tell us why a player achieved a specific level of shooting efficiency, or grabbed so many rebounds, or committed so many turnovers, etc… To answer the “why” question, you have to go look at the player.  In other words, this is where scouting and coaching come into the equation.

Of course, coaches and scouts also believe they know the “how” question.  In other words, some coaches and scouts think the data analysis is unnecessary.  And although I get in trouble for saying this, I think that’s a bit unrealistic.  Let me put it this way.  A person watching a player can pay attention to only so many things.  And when the person is watching a specific player, he can’t also be paying close attention to all the other players on the court.  In the end, the “one person watching” method leaves out relevant information.

Another way to look at this issue is to consider the box score data as “many people staring”.  The people tabulating the data record shooting efficiency, rebounds, turnovers, etc… for every player on the court throughout the game.  And this is done for every player in every game.  We can show that the various factors watched have a clear impact on wins.  In essence, all this staring does indeed give us a very good picture of how productive every player in the NBA has been in the past.

Stats or Scouting?

Okay, let’s tie this argument back to what I saw the Federal Reserve do this past week. Just like in sports, the macroeconomic data tells us more than any business leader – or small collection in Bakersfield – is going to know.  Again, data is simply more comprehensive.  Still, we wouldn’t want the Fed to only look at stats.  Such an approach would cause the decision-makers to miss what is happening right now and that would be a problem. 

What if the Fed took the approach of some decision-makers in the NBA?  What if the Fed decided that all the macroeconomic data was useless and that all that mattered was the anecdotes it heard from business leaders? Obviously that would also be a huge problem.  You certainly wouldn’t want the Fed to try and change interest rates solely based on what a few business leaders told them about the state of the economy. 

And hence we come to the argument offered in The Wages of Wins.  We are not suggesting that “staring” isn’t useful.  What we are suggesting is that “staring” alone can’t get the job done.  You need to look at – and understand – what the numbers are telling you.  And when you do that, better decisions will be made.  At least, obvious mistakes – like thinking Eddy Curry can save your team – will be avoided.

– DJ

Our research on the NBA was summarized HERE.

The Technical Notes at wagesofwins.com provides more information on the published research behind Wins Produced and Win Score

Wins Produced, Win Score, and PAWSmin are also discussed in the following posts:

Simple Models of Player Performance

Wins Produced vs. Win Score

What Wins Produced Says and What It Does Not Say

Introducing PAWSmin — and a Defense of Box Score Statistics

Advertisement

Posted in: Baseball Stories