One way to improve clinical trial reporting: a Yelp-style rating system

This piece originally appeared in the Timmerman Report.

STAT recently published an in-depth report about the many research centers that don’t bother to publicly disclose the results of their clinical trials, even though they are required to do so. This follows on a New England Journal of Medicine article back in March that had a similar analysis of the lack of reporting and publication of clinical trial data to

Most observers of biomedical research would agree that getting clinical trial data out about what happened in a trial is pretty important, whether the trial succeeded or failed. After all, biomedical translational research is most meaningful when done on human subjects and negative information can be quite informative and useful. Animal models are nice, but translation of results from animals to humans is a spotty proposition at best. We need to know what’s working, and what’s not, to know how to best allocate our research resources and how to treat patients.

The lack of reporting is an embarrassment for research. It’s also understandable, because so far the FDA hasn’t used its authority to punish anyone for delayed reporting. Nobody appears to have lost any research funding because they failed to post trial results in a timely manner. Universities told STAT their researchers were “too busy,” given other constraints on their time, to report their results. So what really seems to be going on is that reporting is prioritized below most other activities in clinical research.

It was interesting and eye-opening that industry fared better than academia in both the STAT story and the NEJM article with respect to how many studies have been reported. Having seen the industry process first-hand, I’d speculate that (at least for positive trials) there’s a much stronger incentive to get data out in public. Successful trial results can create buzz among clinicians and patients, revving up trial enrollment which can then help get a new drug on the market faster, and convince people to use it when it’s available. It may be that in academia the effort of getting trial results in the required format for is perceived as too much work, relative to the rewards. Academics are naturally going to spend more energy on directly rewarded activities like writing grant proposals and writing peer-reviewed scientific publications that help them win even more grants, promotions, and other accolades. Well okay. If this is the case, then figuring out new incentives may be key.

So what would work? Anyone who participates in a clinical trial is providing time, may be subject to risks and often is asked to provide samples that are biobanked to support future exploratory and translational research. It’s like when people donate to food banks. I’m pretty sure they mean that food to be eaten and not to sit on a shelf. These participants in clinical trials deserve to have their volunteerism rewarded.

This got me thinking about how to empower patients to get more of what they want. Patient-centered research is a buzzword these days, and for good reason. Patients have at times been an afterthought in the biomedical research enterprise. I thought of services like Yelp and Uber and Angie’s List and other peer-to-peer systems that allow users to get information, provide feedback and give ratings to specific providers. And I wondered: could this be a way to apply pressure to clinical trial researchers to improve their reporting? Continue reading


Baseball analytics, arthritis, and the search for better health forecasts

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

It’s Fourth of July weekend in Seattle as I write this. Which means it’s overcast. This was predictable, just as it’s predictable that for the two months after July 4th the Pacific Northwest will be beautiful, sunny and warm. Mostly.

Too bad forecasting so many other things–baseball, earthquakes, health outcomes–isn’t nearly as easy. But that doesn’t mean people have given up. There’s a lot to be gained from better forecasting, even if the improvement is just by a little bit.

And so I was eager to see the results from a recent research competition in health forecasting. The challenge, which was organized as a crowdsourcing competition, was to find a classifier for whether and how rheumatoid arthritis (RA) patients will respond to a specific drug treatment. The winning methods are able to predict drug response to a degree significantly better than chance, which is a nice advance over previous research.

And imagine my surprise when I saw that the winning entries also have an algorithmic relationship to tools that have been used for forecasting baseball performance for years.

The best predictor was a first cousin of PECOTA. Continue reading

The fall and rise of the LEGO Kingdom: A review of “Brick by Brick”

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

When people ask me what I did growing up, they expect me to say “surf.” I know this because when I tell them what I did for fun their next question is always, “What, you didn’t surf?” I didn’t. Still haven’t learned. Instead I did a lot of the things boys all over the US did. I watched TV. I hung out at the mall and at fast food restaurants. And I played with LEGO.

The brick fundamentally hasn’t changed since I was a kid. My son has a bunch and the basic essence is still snapping things together with that satisfying “click,” and the gradual accretion of form and function from individual, generic elements. Kind of like how life evolves, you know? And yet at the same time LEGO has undergone great changes in packaging, themes, toy categories, and target audiences. Today it’s one of the most respected and recognized toy brands in the world. But something I hadn’t realized until reading “Brick by Brick” by David Robertson and Bill Breen is how close LEGO actually came to crashing and burning in the 90s and early aughts, before recovering to once again become a commercial powerhouse.

Continue reading

The innovators dilemma in biopharma part 3. What would disruption look like?

All opinions are my own and do not necessarily reflect those of Novo Nordisk.

h/t to @Frank_S_David, @scientre, and the LinkedIn Group Big Ideas in Pharma Innovation and R&D Productivity for links and ideas

Part 1 is here.

Part 2 is here.

In the previous parts to this series I’ve covered both why the biopharma industry is ripe for disruption, and what the markets might be that could support a nascent, potentially disruptive technology until it matures enough to allow it to supplant the current dominant industry players.  In this final part I’d like to ask what disruption would look like and provide some examples of directions and companies that exemplify what are, to my mind, these sorts of disruptive technologies and approaches. With, I might add, the complete and utter knowledge that I’m wrong about who and what specifically will be disruptive! But in any case, before we can identify disruption, it’s worthwhile to ask what are the key elements of biopharma drug development that serve as real bottlenecks to affecting  human health, since these are the elements most likely to provide an avenue for disruption. Continue reading

Are market cap and present cash flows the best way to measure innovation?

All opinions are my own and do not necessarily reflect those of Novo Nordisk

Forbes, with the help of the folks from The Innovator’s DNA recently published their coverage and rankings of the 100 most innovative companies.  I’m particularly interested in their ranking method, as it contains elements that are near and dear to my heart–namely, metrics and crowdsourcing.  In a nutshell, they describe how they use a company’s current market capitalization, along with it’s current net present value based on cashflows, to extrapolate how much the market feels the company has in potential.  The method nicely incorporates crowdsourcing in that the market cap measures how much investors as a whole think a company is really worth, now and in the future, and if that’s higher than expected based on cashflow, that suggests investors are factoring in a bonus to value based on future expectations.  Higher future expectations are interpreted as investors seeing a particular company as innovative and having the potential for great leaps forward in offerings and/or income.

I really like using the crowd in this way, and would love to see an analysis that retrospectively looks at these kinds of values over, say, 1970-1990, and combines that with a mature assessment of which companies have been adjudged by business historians to truly have been innovative standouts, which is not the same as business successes.  We say now that Bell Labs was one of the most innovative places on the planet in the 1900s.  Would the same have been said at the time?

At the same time, I can’t help musing if this process couldn’t be made even better.  Recognizing innovation when it’s happening has obvious advantages for anyone looking to get into the next amazing thing, whether as a participant, an investor, or a policy maker.  So let’s start by examining where there might be shortfalls to the Innovator’s DNA method. Continue reading

Finding an Alka Seltzer for the oceans

All opinions are my own and do not necessarily reflect those of Novo Nordisk
Some time ago I wrote an article for Real Change (reposted here) about research being done at the University of Washington to understand the effects of ocean acidification due to rising atmospheric carbon dioxide levels.  The rising pH of the oceans is another one in my list of things we don’t worry enough about with climate change but really should.  Like the bees.  It’s such a seemingly tiny, subtle thing.  The measured decrease in pH of maybe 0.1 units is due to ocean waters absorbing atmospheric CO2 and the resulting conversion of some of that to carbonic acid.  Seems small but it’s really a big deal.
Scientists have documented apparent effects of ocean acidity on coral reefs and oysters, among other organisms (abstracts from links; articles behind paywalls), and while oyster farmers can try to add antacids to their spawning beds, the oceans as a whole are a bit large for a local solution.  Which is why I was excited to see the Paul Allen Family Foundation post the current submissions to their Ocean Challenge (HT @deirdrelockwood).
Let me provide a disclaimer that I have not read most of these proposals in depth.  However, scanning through the titles and sampling a few in greater detail, it’s clear that the Ocean Challenge has prompted a number of groups to come up with ideas about how to try and monitor, test, and mitigate the effects of ocean acidification, at least at the local level and in some cases on a grander scale.  The proposals are available online for public comment, and finalists selected in September.
There are a couple of things to really like about this.   All the proposal summaries are devoid of names and affiliations, which may lead to more unbiased evaluations by public commentators.  This is something that’s been debated for years with respect to other granting agencies like the NIH.  Another great thing is that this is open–anyone can apply and everyone’s ideas are out there for others to learn from, debate, and expand upon.  I’m a fan of open source science, and transparency, and this feels like it’s in that vein.  And last, this is really a big problem.  Not to say government agencies aren’t funding and studying this, but as we’ve seen with other private non-profit foundations like the Gates Foundation, there is a third way beyond government and private industry to try and effect policy and make changes.  I hope for success from this effort.  Because I would really miss oysters.