Baseball, regression to the mean, and avoiding potential clinical trial biases

This post originally appeared on The Timmerman Report. You should check out the TR.

It’s baseball season. Which means it’s fantasy baseball season. Which means I have to keep reminding myself that, even though it’s already been a month and a half, that’s still a pretty short time in the long rhythm of the season and every performance has to be viewed with skepticism. Ryan Zimmerman sporting a 0.293 On Base Percentage (OBP)? He’s not likely to end up there. On the other hand, Jake Odorizzi with an Earned Run Average (ERA) less than 2.10? He’s good, but not that good. I try to avoid making trades in the first few months (although with several players on my team on the Disabled List, I may have to break my own rule) because I know that in small samples, big fluctuations in statistical performance in the end  are not really telling us much about actual player talent.

One of the big lessons I’ve learned from following baseball and the revolution in sports analytics is that one of the most powerful forces in player performance is regression to the mean. This is the tendency for most outliers, over the course of repeated measurements, to move toward the mean of both individual and population-wide performance levels. There’s nothing magical, just simple statistical truth.

And as I lift my head up from ESPN sports and look around, I’ve started to wonder if regression to the mean might be affecting another interest of mine, and not for the better. I wonder if a lack of understanding of regression to the mean might be a problem in our search for ways to reach better health.
Continue reading

Baseball analytics, arthritis, and the search for better health forecasts

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

It’s Fourth of July weekend in Seattle as I write this. Which means it’s overcast. This was predictable, just as it’s predictable that for the two months after July 4th the Pacific Northwest will be beautiful, sunny and warm. Mostly.

Too bad forecasting so many other things–baseball, earthquakes, health outcomes–isn’t nearly as easy. But that doesn’t mean people have given up. There’s a lot to be gained from better forecasting, even if the improvement is just by a little bit.

And so I was eager to see the results from a recent research competition in health forecasting. The challenge, which was organized as a crowdsourcing competition, was to find a classifier for whether and how rheumatoid arthritis (RA) patients will respond to a specific drug treatment. The winning methods are able to predict drug response to a degree significantly better than chance, which is a nice advance over previous research.

And imagine my surprise when I saw that the winning entries also have an algorithmic relationship to tools that have been used for forecasting baseball performance for years.

The best predictor was a first cousin of PECOTA. Continue reading

Can studies of bosses help us figure out how good sports managers are?

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

The world would be a simpler place, although maybe a much more boring and predictable one, if every aspect of performance could be measured directly. My completely unoriginal thought here is that one of the reasons sports appeal to so many people is because they provide clarity. In a confusing, complex world where the NSA is sucking up our information like a Dyson vacuum sucks feathers in a henhouse, and we’re told this is for our own good, clarity can be refreshing.

The simple view of an athlete’s performance is that all the accolades (or jeers), all the milestones (or flops), all the accumulated statistical totals (or lack thereof) are because of that athlete’s ability: his or her drive, passion, training, and natural ability. And that performance is measured via the statistics each sport collects and chooses to honor and promote. Performance is right there, what more do you need? What more could you want? Continue reading

Transparency and the invisible hand in hospital and healthcare costs

All opinions are my own and do not necessarily reflect those of Novo Nordisk

One of the things that sometimes seems to get lost when people talk about the power of the market to create efficiency is that a free market requires that information be shared and freely available and understandable by everyone.  When information is withheld by one side or the other of a transaction, or when different customers for a service or product are unable to compare prices, the metaphor of the invisible hand breaks down.

You can see, this, interestingly enough, in sports as it relates to both the trading of players under contract and the signing of free agents.  Since I’m a baseball fan, let me link here to a discussion of research that’s been done looking at Major League Baseball.  The studies looked at players traded or signed by a different team as a free agent and how those players performed in subsequent years versus players whose original team re-signed them.  It turns out that players who switched teams did, indeed, perform more poorly relative to projections than players who stayed.  This suggests that the original teams have proprietary information that allows them to make better decisions about which players to retain.  Thus the market for baseball players isn’t quite free and efficient because of information asymmetry.

And unfortunately, information asymmetry is also rampant in other industries such as healthcare. Continue reading

Major League Baseball should be all over the quantified self movement

All opinions are my own and do not necessarily reflect those of Novo Nordisk

Baseball players break down.  Their performances fluctuate.  As a group there are some interesting generalities with respect to how pitching, hitting and fielding change with age.  But the error bars are huge.  There are many things we still don’t know about baseball players, about why one prospect hits the ground running and another flames out.  And we also don’t know if there is any way to know, since the task of putting together the skills needed to play major league baseball may be one of the most complex of the major sports, and understanding complexity is hard.

But it seems worthwhile to give it a try.

The Mystery of the Missing Ligament

Let’s talk about R.A. Dickey for a minute.  Not because he’s a highly interesting human being, although he is.  And not because he’s a knuckleballer, which is fun and interesting due to rarity and the entertaining sight of six foot athletes flailing at baseballs traveling with the flight path of a drunken small-nosed bat.  But rather because he was drafted in 1996 in the 1st round by the Texas Rangers, and only during his physical workup was it discovered that he was missing a key ligament in his arm.  The Ulnar Collateral Ligament (UCL), to be exact.  Without which, it is assumed, a pitcher cannot pitch. Continue reading