This piece was originally posted on Xconomy on March 1, 2013.  The views in it are my own and do not necessarily reflect the views of Novo Nordisk.

Consider a candidate.  Selecting that candidate takes thousands of hours of time and research–checking background, verifying data, assessing probabilities, projecting futures.  Once selected, more years of development follow, during which time the odds of success are less than 10%.  And if that candidate finally does make it, there’s just a small window of exclusivity before protection expires and that candidates goes out to the broader market.

I’m talking, of course, about baseball players.

So I’m a Mariners fan.  Have been since about 1999 (I moved to Seattle in 1996, so missed the big comeback year and it took me a little time to catch up).  And like all fans, I watch and hope, year after year, looking for signs of improvement, direction, some indication that there’s a plan. I keep looking towards some point in the not too distant future–let’s call it “next year” or even “year after next”–when I’ll once again be able to root for a winning team.

But while the Mariners may still be in what feels like eternal rebuilding, I’ve been able to find a silver lining in my fandom:  I’ve realized drug development seems to be learning from baseball.

Statistics, but the right ones.

There are a lot of parallels between the businesses of baseball and drug development.  Both involve long periods of development followed by limited periods of exclusivity for the product (drugs, players) being developed.  Resources (targets, talent) are rare.  Assets get traded or bought or sold.  There are the juggernauts and the mid-market and the small-market players.  And there’s the always-present need to keep doing more and finding better ways of winning, preferably with less.

One of the more fascinating developments in baseball has been the rise of a new statistical framework around the game.  Baseball has always been the most statistically conscious of sports, but it’s also been the most heavily invested in it’s own history and mythology.  Ken Burns is not making a 18 ½ hour documentary on Arena Football anytime soon.  That reverence for history means there has been a lot of resistance to new ideas.  For almost the entire modern era of baseball, certain statistics (ERA, W-L records, batting average, RBIs) have been the gold standard for performance.  Even though, when it comes down to it, they’re not really the best things to measure if you want to create a winning baseball team.

As Dave Cameron from Fangraphs has discussed on several occasions (like this one), statistics in baseball are how we figure out the the answers to questions.  We might be asking who’s the Most Valuable Player (*cough*Trout*cough*) or what kind of pitcher or hitter a given team should be trying to get through free agency, or whether a player can be expected to sustain his level of performance.  Some statistics like RBIs, venerated for years, are actually not that useful since they partially reflect circumstances outside a hitter’s control but are treated as a direct proxy for ability.  Albert Pujols would have trouble cracking 70 RBIs a year if he were batting 9th.

But okay, drug development.  Better statistics are making their way into drug development, exemplified by the increasing emphasis on Big Data.  Collaborations are getting larger and drug companies are trying hard to capture as much data as possible, whether it’s clinical, metabolic, transcriptional, genomic, proteomic or any other flavor that becomes possible.  However, the key will be figuring out which statistics, which measurements, are really relevant to the main questions drug companies want to answer:  why do people get sick and how can we figure out what a drug will actually do once it gets into the human body?  Are drug companies figuring this out?  And for the moment I’m leaving out biotech startups, since the current bar to taking advantage of Big Data is still beyond the reach of most small companies, at least right now.

In my view the answer is maybe.  The move towards biomarkers throughout the drug development pipeline is reassuring as it shows a realization that we need to measure outcomes more clearly and quickly.  There is also a greater recognition that bioinformatics is a key element of a drug development pipeline.  More importantly, there needs to be a recognition that specific outcomes (the game-winning RBI, the successful PhIII trial) aren’t necessarily justifications for the decision-making that came before.

 

Giving Richie Sexson a multiyear contract to join the Mariners in 2005 was a bad decision.  Advanced metrics had pegged his skillset as a poor fit for the Mariner’s home stadium, and his likelihood of sustaining his performance at that point in his career was low.  As it happened, he did fade away after a couple of years, but the key point is that even if he had performed reasonably throughout his contract, it would still have been a bad decision based on what we knew given our best tools at the time.  Pharma needs to develop those tools to not just gather more data, but figure out how to ask the right questions and trust what the data is saying..  However, it’s not clear that Pharma has reached its Moneyball moment.

Undervalued Assets

Which leads to another lesson from baseball: the under-appreciated asset.  Contrary to what some commentators have suggested, Moneyball wasn’t ultimately about Billy Beane deciding to draft only fat slow guys who could take a walk and get on base.  The real story was the concept of finding the market inefficiencies in Major League Baseball to get an edge.  Oakland plays in a lousy stadium with a putrid revenue stream and a snooty neighbor across the Bay who refuses to let Oakland move to San Jose.  In order to compete, their front office recognized it was necessary to find players and skill sets that were less valued by their competitor even though those skillsets were just as important to winning baseball games as more conventional talents.

During the year chronicled in Moneyball,  on base percentage (OBP) was batting average’s country mouse cousin.  Oakland’s insight was that batting average is just a proxy for not making outs, and in that respect OBP is a lot more important.  A player with a batting average of .300 who never walks makes an out 70% of the time.  But a player with a batting average of .250 and who walks 15% of the time only makes an out in 60% of his plate appearances. In baseball the single most important commodity is outs, of which you have but 27.  Oakland exploited this market inefficiency to get inexpensive guys who may not have always hit the ball, but still got on base.  Here’s the thing:  as market inefficiencies have shifted, so has Oakland’s strategy in player acquisition.

In drug development we can see some pharma searching for that kind of edge.  GSK and others are making a push into orphan diseases, turning an under-appreciated approach into what may become a glutted market to the latecomers.  How else might drug development search for an edge?  The logical answer is: every way it can.  As Jonah Keri described in his excellent book The Extra 2%,  Tampa Bay, another small market team in a lousy stadium deal has nevertheless managed to create a thriving, successful baseball club by taking advantage of every possible way it can compete, whether by taking on undervalued and risky assets or employing probably the most experimental and forward thinking manager in major league baseball.  If there is an advantage to be gained, Tampa Bay is exploring it.

Just recently, it’s been reported that Tampa Bay will face a reduction to their draft pool  budget in 2013 because they spent too much money on International Free Agents.  This might seem like a problem for a team that needs to build via their farm system because of limited revenue.  But it may actually be a calculated risk that they can get more for their money by overstepping MLB rules rather than committing too heavily into what’s thought to be an overall poor US draft cohort this year.  Drug development companies would be well advised to take this kind of approach and encourage a broad exploration of every way in which efficiencies might be gained, whether it’s in discovery, manufacturing, patient recruitment, therapeutic areas or technology.  And most importantly, companies need to set up mechanisms to broadly communicate the results, good and bad, and support and laud all of them, not just the ones that succeed.

Randomness and the unlucky bounce

Which brings me to a last insight from baseball.  Baseball is a probabilistic game.  The best players in the world get a hit three times out of ten.  Random factors, or at least factors so uncontrollable as to essentially behave like random factors, can influence the outcome of a game.  Just ask the Chicago Cubs.  In Moneyball, Billy Beane is described as saying, “My s*** doesn’t work in the playoffs,” by which he meant that the team he built was designed to perform above average, on average, over the course of a 162 game baseball season.  But the playoffs are fickle and any team can beat any other team in a best of 5 or best of 7 series.

We don’t appreciate how much randomness affects everything.  In The Drunkard’s Walk Leonard Mlodinow provides ample evidence about how little we really control everything around us, even though we might think we do.  He also shows the poor grasp people have on probabilities.  In baseball the probabilities are made manifest in the statistics we track, and maybe that’s helped drive the adoption, finally, of better statistical tools.

Baseball is full of random happenings.  Adam Dunn hit 40 home runs a year (more or less)  like a metronome for seven years, and then in 2011 was completely lost at the plate.  And then in 2012 he hit 41.  Drug development is full of randomness too.

Drug development sometimes seems to show a much more deterministic mindset.  I blame the successes of the 80s, when a whole raft of wonderful drugs entered the market, and lulled people into a sense that this kind of productivity could go on forever.  Pipelines came to be viewed by companies and analysts alike as though they were treadmills steadily pushing new drugs forward, as though making drugs was like manufacturing widgets.  And yet, even companies that create widgets (albeit very large and complex widgets) have problems meeting their deadlines and come against unexpected issues.  How much more uncertain is drug development, which deals with trying to figure out how biology works?

What lesson can drug development companies take?  Here there is one important difference:  even a failing baseball team often makes money, whereas a failing pharma company faces being bought or imploding.  On the other hand, poorly performing franchises in the Major Leagues have been threatened with being shut down, or at least moved, so perhaps there are still some parallels there.  One key learning is the value of stability.  Over-reaction to poor results can be deadly to the long-term health of a ballclub, or a company, as it can lead to the loss of talent due to mis-assignment of blame.  Another key point is diversification of revenue streams.  Some of the best positioned ballclubs are there because they have worked hard to increase revenue beyond box-office sales and the occasional t-shirt purchase.  Similarly, some of the best positioned Pharma companies are diversified players like Roche and Johnson and Johnson.

Maybe the most important lesson is to realize in a random world there is no way to guarantee success in drug development, and therefore, the goal is to set up the best processes, with clear measurements and benchmarks; to evaluate constantly but to intervene rarely; to work on increasing the probability of success.  The 2001 Mariners won 116 games and still didn’t even make the World Series.  And yet, few question that they were the best team that year by far.  The goal for the Mariners after that season was to evaluate how they got there, try to separate luck from skill, and attempt to replicate those elements that were under the control of the players and the front office.  That could be the approach taken in drug development as well.

Baseball, or the Movie Industry, or Oil exploration, or…

I’d love to delve into other concepts, like Value Over Replacement Player (VORP) and how we might apply that to drugs and scientists, but that could be a thought for another day.  As the drug development industry continues its struggle with how to carve out its future (because, you know, eventually there won’t be any more companies left to buy), it seems potentially fruitful to try and learn from other industries that have been faced with similar challenges.