One way to improve clinical trial reporting: a Yelp-style rating system

This piece originally appeared in the Timmerman Report.

STAT recently published an in-depth report about the many research centers that don’t bother to publicly disclose the results of their clinical trials, even though they are required to do so. This follows on a New England Journal of Medicine article back in March that had a similar analysis of the lack of reporting and publication of clinical trial data to clinicaltrials.gov.

Most observers of biomedical research would agree that getting clinical trial data out about what happened in a trial is pretty important, whether the trial succeeded or failed. After all, biomedical translational research is most meaningful when done on human subjects and negative information can be quite informative and useful. Animal models are nice, but translation of results from animals to humans is a spotty proposition at best. We need to know what’s working, and what’s not, to know how to best allocate our research resources and how to treat patients.

The lack of reporting is an embarrassment for research. It’s also understandable, because so far the FDA hasn’t used its authority to punish anyone for delayed reporting. Nobody appears to have lost any research funding because they failed to post trial results in a timely manner. Universities told STAT their researchers were “too busy,” given other constraints on their time, to report their results. So what really seems to be going on is that reporting is prioritized below most other activities in clinical research.

It was interesting and eye-opening that industry fared better than academia in both the STAT story and the NEJM article with respect to how many studies have been reported. Having seen the industry process first-hand, I’d speculate that (at least for positive trials) there’s a much stronger incentive to get data out in public. Successful trial results can create buzz among clinicians and patients, revving up trial enrollment which can then help get a new drug on the market faster, and convince people to use it when it’s available. It may be that in academia the effort of getting trial results in the required format for clinicaltrials.gov is perceived as too much work, relative to the rewards. Academics are naturally going to spend more energy on directly rewarded activities like writing grant proposals and writing peer-reviewed scientific publications that help them win even more grants, promotions, and other accolades. Well okay. If this is the case, then figuring out new incentives may be key.

So what would work? Anyone who participates in a clinical trial is providing time, may be subject to risks and often is asked to provide samples that are biobanked to support future exploratory and translational research. It’s like when people donate to food banks. I’m pretty sure they mean that food to be eaten and not to sit on a shelf. These participants in clinical trials deserve to have their volunteerism rewarded.

This got me thinking about how to empower patients to get more of what they want. Patient-centered research is a buzzword these days, and for good reason. Patients have at times been an afterthought in the biomedical research enterprise. I thought of services like Yelp and Uber and Angie’s List and other peer-to-peer systems that allow users to get information, provide feedback and give ratings to specific providers. And I wondered: could this be a way to apply pressure to clinical trial researchers to improve their reporting? Continue reading

No, CRISPR-Cas won’t save the day for ag biotech

You want to know how to drive a scientist crazy? Insist that you believe something that’s not supported by current scientific evidence. Tell her vaccines cause autism, or creationism is just as valid a theory as evolution, or that climate change isn’t really happening, I mean, after all, a monster blizzard hit Washington DC this January! Global warming, pssh…

There’s an old episode of Friends that did a good job of showing how this kind of conversation goes. Phoebe professes not to believe in evolution and Ross, a paleontologist, keeps trying to convince her that evolution is real using scientific evidence and logic. He grows increasingly frustrated and insistent as she continues to deny the basis of his life’s work, finally losing it when she goads him into admitting (like a good scientist) that even theories like evolution are not immune from questioning and testing.

We train scientists to carefully generate, weigh and use evidence. To no one’s surprise, this leads many scientists to generalize and think that in all matters having to do with the physical world we all should and of course will follow the evidence. Yes, sometimes that leads to unpopular ideas, and sometimes the ideas change as the weight of evidence changes. This training can make scientists kind of boring at cocktail parties. Still, the overall scientific process keeps moving forward and it’s because of this reliance on evidence.

But many people (including, at times, even some scientists) don’t always think the same way about things in the physical world. And that’s why I’m pessimistic that CRISPR-Cas technology will peacefully resolve the Genetically Modified Organism (GMO) debate. Continue reading

Baseball, regression to the mean, and avoiding potential clinical trial biases

This post originally appeared on The Timmerman Report. You should check out the TR.

It’s baseball season. Which means it’s fantasy baseball season. Which means I have to keep reminding myself that, even though it’s already been a month and a half, that’s still a pretty short time in the long rhythm of the season and every performance has to be viewed with skepticism. Ryan Zimmerman sporting a 0.293 On Base Percentage (OBP)? He’s not likely to end up there. On the other hand, Jake Odorizzi with an Earned Run Average (ERA) less than 2.10? He’s good, but not that good. I try to avoid making trades in the first few months (although with several players on my team on the Disabled List, I may have to break my own rule) because I know that in small samples, big fluctuations in statistical performance in the end  are not really telling us much about actual player talent.

One of the big lessons I’ve learned from following baseball and the revolution in sports analytics is that one of the most powerful forces in player performance is regression to the mean. This is the tendency for most outliers, over the course of repeated measurements, to move toward the mean of both individual and population-wide performance levels. There’s nothing magical, just simple statistical truth.

And as I lift my head up from ESPN sports and look around, I’ve started to wonder if regression to the mean might be affecting another interest of mine, and not for the better. I wonder if a lack of understanding of regression to the mean might be a problem in our search for ways to reach better health.
Continue reading

An Open Standard for APIs Could Lead us to Better Health

There’s a parable about the elephant and the rider that’s been used by Chip and Dan Heath, and that originated with Jonathan Haidt, to describe how humans make decisions. A person’s mind can be thought of as consisting of a rider, representing the rational part of human thinking, and the elephant she’s riding, representing emotion. Both of these play a role in how a person decides things, and many of us believe the rider–the rational part–is in charge. The rider taps the elephant with her guide stick, and the elephant obediently moves in that general direction or does a specific task, like carrying lumber from place to place.

Except that’s not how a lot of decisions actually get made. Instead, the elephant sees a bunch of bananas, or a herd of other elephants, or a nice cool river to bathe in, and goes that way instead. And the rider…well, the rider can’t do much about it except, after the fact, rationalize how she always wanted to go in that direction to begin with. Yeah, it was time for a bath, sure

This framing has stuck in my mind for years and it’s a really helpful way of looking at many of the odd things that people do or say, ranging from climate change denial, to believing genetically modified organisms are inherently evil, to smoking despite everything we know about the harms that result, to even saying that Paul Blart, Mall Cop II is really, you know, not that bad–really. And it also speaks to one of the more vexing problems we have in human health. Why do people keep doing things they really probably shouldn’t, and know they shouldn’t, if they want to stay healthy?

I’ve touched before on how the power of digital tools can help make it easier for us to make good decisions. OPower is doing this for power consumption and conservation, and with the advent of tools like Apple’s Healthkit and the proliferation of activity trackers, the time is right to do this for health. Continue reading

Making Change

And now for something completely different! Short fiction in honor of the recent unveiling of the Apple iWatch and Healthkit.

“I wouldn’t eat that if I were you.”

Sylvia paused, bacon cheeseburger halfway to her mouth, and peered at the neon green band wrapped around her wrist. The wraparound touchscreen was currently showing a cat emoji. It had a frowny face, expression halfway between puzzlement and alarm.

“What did you say?”

“I’m just saying,” said her Best Buddy wristband, “that when we met a few weeks ago, you mentioned wanting to keep your weight in a specific range.” The emoji shrugged. “Little friendly reminder. You know?”

Sylvia carefully put the burger back down and resisted the urge to lick grease off her fingers. She fumbled for her napkin, her fingers leaving translucent streaks on the thin, white paper.

“I–well, yeah. But, I mean, you’ve never said anything like this before like when–” She broke off, remembering the milkshake, the onion rings, the King-size Choconut bar…

“Well it’s not the first thing you do, is it? When you meet someone and you’re just getting to know them?” The cat had morphed into a light pink, animated mouse, standing on its hind legs, bashfully kicking one leg. “But now, we’re friends!” Continue reading

Could pro sports lead us to wellness?

Comment From Bill
St. Louis is being hindered in the stretch drive by some kind of GI bug passing through (so to speak) the team. Reports have as many as 15 guys down with it at once. That seems a lot, but given the way a baseball clubhouse works, my question is why don’t we see more of that? Answering that baseball players are fanatically interested in sanitation and hygiene ain’t gonna cut it, I don’t think…

12:10
Dave Cameron: They have access to a lot of drugs.

–comment from a chat at Fangraphs, September 24, 2014

So this comment caught my eye. Ever since I began following sites like BaseballProspectus.com and Fangraphs.com, and reading things like Moneyball, I’ve found myself thinking about efficiency and unappreciated or unexplored resources in different situations.

I realize this was a throwaway line in a baseball chat. But it piqued my interest because it seems to point out something that’s maybe underappreciated and understudied about how sports teams go about their business–specifically, the kinds of things they do to keep their athletes healthy.

My question is, does this represent a potential source of “Found Research” data that could help the rest of us reach wellness? Continue reading