Sequencing in polio, baseball pitching and cancer: sometimes the order of events matters

This piece originally appeared in the Timmerman Report.

What do the polio virus, baseball pitch choice and cancer have in common?

The answer, of course, is sequencing. But not in the “figure out the DNA” way (although that’s involved). Instead in the “what comes first” way. Confused? Read on!

A big perk of Seattle is proximity to great institutions of biomedical research like the University of Washington and the Fred Hutchinson Cancer Research Center. Ever since my graduate student days in genetics at UC-Berkeley I’ve enjoyed going to seminars–especially seminars that are outside my field of study. Very little beats a good seminar for giving you a quick, condensed view of the state of a field of research. A bad seminar…well…we all could use more sleep, right?

In early October, Raul Andino of UCSF came to the Hutch to talk about his work on viral evolution. His team has been examining a clever real-world system to track the evolution of viruses. The near-eradication of polio (one of the great public heath victories of the past century) has led to the curious problem that as of the middle of this year most new cases of polio arose as a result of vaccination efforts. The live, attenuated vaccine that’s used in the developing world can, in very rare cases, mutate in just the wrong ways in its host, leading to the creation of a virulent strain that can infect others. In the US we use an inactivated polio vaccine which requires several injections; in much of the developing world the oral polio virus is preferred due to its ease of administration, lower cost, and immunization profile. The Andino lab realized that by studying these isolated outbreaks, which all originated with the same, genetically identical progenitor, they could test a hypothesis about the adaptive landscape of virulence evolution. Continue reading

Advertisements

Baseball, regression to the mean, and avoiding potential clinical trial biases

This post originally appeared on The Timmerman Report. You should check out the TR.

It’s baseball season. Which means it’s fantasy baseball season. Which means I have to keep reminding myself that, even though it’s already been a month and a half, that’s still a pretty short time in the long rhythm of the season and every performance has to be viewed with skepticism. Ryan Zimmerman sporting a 0.293 On Base Percentage (OBP)? He’s not likely to end up there. On the other hand, Jake Odorizzi with an Earned Run Average (ERA) less than 2.10? He’s good, but not that good. I try to avoid making trades in the first few months (although with several players on my team on the Disabled List, I may have to break my own rule) because I know that in small samples, big fluctuations in statistical performance in the end  are not really telling us much about actual player talent.

One of the big lessons I’ve learned from following baseball and the revolution in sports analytics is that one of the most powerful forces in player performance is regression to the mean. This is the tendency for most outliers, over the course of repeated measurements, to move toward the mean of both individual and population-wide performance levels. There’s nothing magical, just simple statistical truth.

And as I lift my head up from ESPN sports and look around, I’ve started to wonder if regression to the mean might be affecting another interest of mine, and not for the better. I wonder if a lack of understanding of regression to the mean might be a problem in our search for ways to reach better health.
Continue reading

Baseball analytics, arthritis, and the search for better health forecasts

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

It’s Fourth of July weekend in Seattle as I write this. Which means it’s overcast. This was predictable, just as it’s predictable that for the two months after July 4th the Pacific Northwest will be beautiful, sunny and warm. Mostly.

Too bad forecasting so many other things–baseball, earthquakes, health outcomes–isn’t nearly as easy. But that doesn’t mean people have given up. There’s a lot to be gained from better forecasting, even if the improvement is just by a little bit.

And so I was eager to see the results from a recent research competition in health forecasting. The challenge, which was organized as a crowdsourcing competition, was to find a classifier for whether and how rheumatoid arthritis (RA) patients will respond to a specific drug treatment. The winning methods are able to predict drug response to a degree significantly better than chance, which is a nice advance over previous research.

And imagine my surprise when I saw that the winning entries also have an algorithmic relationship to tools that have been used for forecasting baseball performance for years.

The best predictor was a first cousin of PECOTA. Continue reading

Big data and baseball efficiency: the traveling salesman had nothing on a baseball scout

All opinions are my own and do not necessarily reflect those of Novo Nordisk

The MLB draft is coming up and with any luck I’ll get this posted by Thursday and take advantage of web traffic. I can hope! Anyway, Tuesday in Fangraphs I read a fascinating portrayal of the draft process, laying out the nuts and bolts of how organizations scout for the draft. The piece, written by Tony Blengino (whose essays are rapidly becoming one of my favorite parts of this overall terrific baseball site), describes all the behind the scenes work that happens to prepare a major league organization for the Rule 4 draft. Blengino described the dedication scouts show in following up on all kinds of prospects at the college and high school levels, what they do, how much they need to travel, and especially how much ground they often need to cover to try and lay eyes on every kid in their area.

One neat insight for me was Blengino’s one-word description of most scouts as entrepreneurs. You could think of them almost as founders of a startup, with the kids they scout as the product the scouts are trying to sell to upper layers of management in the organization. As such, everything they can do to get a better handle on a kid’s potential can feed into the pitch to the scouting director.

I respect and envy scouts’ drive to keep looking for the next big thing, the next Jason Heyward or Mike Trout. As Blengino puts it, scouts play “one of the most vital, underrated, and underpaid roles in the game.” While one might make the argument that in MLB, unlike the NFL or NBA, draft picks typically are years away from making a contribution and therefore how important can draft picks be?, numerous studies have shown that the draft presents an incredible opportunity for teams in building and sustaining success. In fact, given that so much of an organization’s success hinges on figuring out which raw kids will be able to translate tools and potential into talent, one could (and others have)  made the argument that scouting is a huge potential market inefficiency for teams to exploit. Although I’ll have a caveat later. But in any case, for a minor league system every team wants to optimize their incoming quality because, like we say in genomic data analysis, “garbage in, garbage out.”

As I was reading this piece, I started thinking about ways to try and create more efficiencies. And I started thinking about Big Data.  Continue reading

Biopharma should choose targets using a baseball-style draft

All opinions my own and do not necessarily reflect those of Novo Nordisk

I was sitting around last evening checking out how the end of my fantasy baseball season is working out (for the record, first out of ten in one league and fourth in the league I wrote about here) and I starting thinking again about the parallels between baseball and drug development (which I previously wrote about here and here for example, and also Stewart Lyman has a nice piece on a similar theme here). And it hit me that there’s another way in which biopharma could take a  page from baseball: fantasy and Major League Baseball both.

Biopharma could institute a draft for drug targets.  And to explore this I’m going to employ the time-honored, not to mention trite and artificial, format of a series of questions and answers.

Continue reading

Major League Baseball should be all over the quantified self movement

All opinions are my own and do not necessarily reflect those of Novo Nordisk

Baseball players break down.  Their performances fluctuate.  As a group there are some interesting generalities with respect to how pitching, hitting and fielding change with age.  But the error bars are huge.  There are many things we still don’t know about baseball players, about why one prospect hits the ground running and another flames out.  And we also don’t know if there is any way to know, since the task of putting together the skills needed to play major league baseball may be one of the most complex of the major sports, and understanding complexity is hard.

But it seems worthwhile to give it a try.

The Mystery of the Missing Ligament

Let’s talk about R.A. Dickey for a minute.  Not because he’s a highly interesting human being, although he is.  And not because he’s a knuckleballer, which is fun and interesting due to rarity and the entertaining sight of six foot athletes flailing at baseballs traveling with the flight path of a drunken small-nosed bat.  But rather because he was drafted in 1996 in the 1st round by the Texas Rangers, and only during his physical workup was it discovered that he was missing a key ligament in his arm.  The Ulnar Collateral Ligament (UCL), to be exact.  Without which, it is assumed, a pitcher cannot pitch. Continue reading

Fielding percentage for UK surgeons

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

Last week I posted on how our measurements of defense in baseball have become a lot more sophisticated, and how that gave me hope for the evaluation of innovation.  If baseball, one of the most tradition-bound of US sports can adopt to new metrics, surely business can too.

I was reminded of this with the publication of a recent article about the National Health Service (NHS) in the United Kingdom and their plan to publicize the surgical success rates of clinicians across their country.  Surgeons in eight different specialities will have their mortality rates for specific procedures, length of hospital stays post surgery, and other elements published in tables for anyone to access.  The first group to have this information released is vascular surgeons.

A fascinating aspect of how this is being done is that publication of one’s rates is voluntary, but if a surgeon chooses not to have his or her rates published, that surgeon will be named.  It’s not quite putting people into stocks in the public square, but it is definitely a form of public shaming meant to increase participation.

Nevertheless, six surgeons have opted out and been named.  Game theory might predict these are surgeons on the low end of the measured metrics, who are taking a calculated risk that the negatives associated with not publishing their rates are less than the negatives that would come with disclosure of their rates.  But that’s not the case.  The NHS has stated that none of these surgeons lie outside the normal range for the reported metrics.

Instead, these doctors are protesting that the metrics are not measuring the right things.   They suggest the metrics don’t take into account the subtleties involved in surgical cases, how procedure names alone don’t properly capture how difficult or easy a procedure might be for a given patient.  Are there comorbidities?  Is a patient in generally poor health?  Is a surgeon one who specializes in tricky, difficult cases which would therefore lead to a lower success rate even though the surgeon him or herself might be highly skilled and effective?  Could these metrics scare new surgeons away from performing more difficult procedures?

This echoes the debate about defense in baseball, and whether standard metrics such as fielding percentage are the best for measuring defensive ability, or if more elaborate measures better reflect reality.

Still, while I agree with the viewpoint that we should always try to improve metrics, I also think the NHS is doing the right thing.  I think in this case the proper analogy might be baseball defense back at the time before the invention of fielding percentage.  In the practice of medicine world-wide there is a surprising lack of information about measures like success rates and efficacy.  As Sir Bruce Keogh said to the BBC: “This has been done nowhere else in the world, and I think it represents a very significant step.”  To take another quote from the article, Professor Ben Bridgewater commented, “We’ve been collecting data on cardiac surgery since 1996 and we’ve been publishing it at individual surgeon level since 2005, and what we’ve seen associated with that is big improvements in quality: the mortality rates in cardiac surgery today are about a third of what they were ten years ago.” That which we don’t measure, we can’t improve.

In the US, that idea is becoming more prominent.  Recent articles in Time and the New York Times have highlighted how transparency is lacking in the United States healthcare system, and the Obama Administration’s emphasis on comparative effectiveness is another thrust in that direction.  What the NHS is doing is a great model and a great start, and I hope they continue to both make these aspects of healthcare more transparent and work to refine their metrics so that they accurately reflect the difficulty of practicing good medicine.