Lack of replication no surprise when we’re studying really complex problems

All opinions are my own and do not necessarily reflect those of Novo Nordisk

For another nice take on this topic see Paul Knoepfler’s blog post here.

One of the sacred (can I say sacred in reference to something scientific?) tenets of the scientific method is reproducibility.  If something is real and measurable, if it’s a fact of the material world, then the expectation is that the result should be reproducible by another experimenter using the same methods as described in the original report.  One of the most well known (among physicists anyway) examples of irreproducible data is the Valentine’s Day Magnetic Monopole detected by Blas Cabrera back in 1982.  Great experimental data.  Never repeated, and therefore viewed as insufficient proof for the existence of a magnetic monopole.

So it’s troubling that in the past few years there have been numerous stories about the lack of reproducibility for different scientific experiments.  In biomedical science the number of  reports on the difficulty of reproducing results has gotten so great that the NIH has begun thinking about how to confirm and require reproducibility of some kinds of experimental results.  Just a few days ago another field, that of psychological priming, saw the publication of an article that the effects of “high-performance priming,” could not be reproduced.  This is another field undergoing serious questioning about whether/why results don’t reproduce, with commentary from such luminaries as Daniel Kahneman. Continue reading