Baseball, regression to the mean, and avoiding potential clinical trial biases

This post originally appeared on The Timmerman Report. You should check out the TR.

It’s baseball season. Which means it’s fantasy baseball season. Which means I have to keep reminding myself that, even though it’s already been a month and a half, that’s still a pretty short time in the long rhythm of the season and every performance has to be viewed with skepticism. Ryan Zimmerman sporting a 0.293 On Base Percentage (OBP)? He’s not likely to end up there. On the other hand, Jake Odorizzi with an Earned Run Average (ERA) less than 2.10? He’s good, but not that good. I try to avoid making trades in the first few months (although with several players on my team on the Disabled List, I may have to break my own rule) because I know that in small samples, big fluctuations in statistical performance in the end  are not really telling us much about actual player talent.

One of the big lessons I’ve learned from following baseball and the revolution in sports analytics is that one of the most powerful forces in player performance is regression to the mean. This is the tendency for most outliers, over the course of repeated measurements, to move toward the mean of both individual and population-wide performance levels. There’s nothing magical, just simple statistical truth.

And as I lift my head up from ESPN sports and look around, I’ve started to wonder if regression to the mean might be affecting another interest of mine, and not for the better. I wonder if a lack of understanding of regression to the mean might be a problem in our search for ways to reach better health.
Continue reading

Advertisements

Baseball analytics, arthritis, and the search for better health forecasts

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

It’s Fourth of July weekend in Seattle as I write this. Which means it’s overcast. This was predictable, just as it’s predictable that for the two months after July 4th the Pacific Northwest will be beautiful, sunny and warm. Mostly.

Too bad forecasting so many other things–baseball, earthquakes, health outcomes–isn’t nearly as easy. But that doesn’t mean people have given up. There’s a lot to be gained from better forecasting, even if the improvement is just by a little bit.

And so I was eager to see the results from a recent research competition in health forecasting. The challenge, which was organized as a crowdsourcing competition, was to find a classifier for whether and how rheumatoid arthritis (RA) patients will respond to a specific drug treatment. The winning methods are able to predict drug response to a degree significantly better than chance, which is a nice advance over previous research.

And imagine my surprise when I saw that the winning entries also have an algorithmic relationship to tools that have been used for forecasting baseball performance for years.

The best predictor was a first cousin of PECOTA. Continue reading

Biopharma should choose targets using a baseball-style draft

All opinions my own and do not necessarily reflect those of Novo Nordisk

I was sitting around last evening checking out how the end of my fantasy baseball season is working out (for the record, first out of ten in one league and fourth in the league I wrote about here) and I starting thinking again about the parallels between baseball and drug development (which I previously wrote about here and here for example, and also Stewart Lyman has a nice piece on a similar theme here). And it hit me that there’s another way in which biopharma could take a  page from baseball: fantasy and Major League Baseball both.

Biopharma could institute a draft for drug targets.  And to explore this I’m going to employ the time-honored, not to mention trite and artificial, format of a series of questions and answers.

Continue reading

Undervalued assets in biopharma hiring: Adaptability

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

A night of fantasy baseball goes horribly awry

This season I had a spectacularly poor fantasy baseball auction draft.  It was my own fault.  For those of you unfamiliar with fantasy sports, a group of friends create teams by selecting players from a real sports league and track their performance over the season.  The better your players perform, the better you do in your league.  Many leagues, like ours, select players by means of an auction draft.  Everyone gets a certain amount of virtual money to bid on different players, and you use that finite amount of money to fill out your roster.

On the night of our draft, because I had made plans to go out, I set up the auction software with a bunch of default values for different players.  Basically, amounts that I was willing to bid up to for each.  This is called robo-drafting.   I thought I’d set my boundaries well.

I was wrong. Continue reading