The Limitations of Science, Especially Epidemiology

Recently, Dr. John Ioannidis, who has been referred to as a "meta-researcher," has been getting quite a bit of press, most recently (as far as I know) in Newsweek. Keep it coming.

The Atlantic notes: 

Ioannidis may be one of the most influential scientists alive. Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.

It's funny how the media cycle operates. Ioannidis has popped up in a couple of magazine articles (both great reads) in the last few months:

Lies, Damned Lies, and Medical Science (The Atlantic)

The Decline Effect and the Scientific Method (The New Yorker)

Here is the original essay that was published by Ioannidis in PLoS that has created a stir: Why Most Published Research Findings Are False (2005)

From The Atlantic: 

Studies have gone back and forth on the cancer-preventing powers of vitamins A, D, and E; on the heart-health benefits of eating fat and carbs; and even on the question of whether being overweight is more likely to extend or shorten your life. How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all. 

Also, in the New York Times magazine, Gary Taubes noted at the end of a stellar article on epidemiology in 2007:

So how should we respond the next time we’re asked to believe that an association implies a cause and effect, that some medication or some facet of our diet or lifestyle is either killing us or making us healthier? We can fall back on several guiding principles, these skeptical epidemiologists say. One is to assume that the first report of an association is incorrect or meaningless, no matter how big that association might be. After all, it’s the first claim in any scientific endeavor that is most likely to be wrong. Only after that report is made public will the authors have the opportunity to be informed by their peers of all the many ways that they might have simply misinterpreted what they saw. The regrettable reality, of course, is that it’s this first report that is most newsworthy. So be skeptical. . . .

All of this suggests that the best advice is to keep in mind the law of unintended consequences. The reason clinicians test drugs with randomized trials is to establish whether the hoped-for benefits are real and, if so, whether there are unforeseen side effects that may outweigh the benefits. If the implication of an epidemiologist’s study is that some drug or diet will bring us improved prosperity and health, then wonder about the unforeseen consequences. In these cases, it’s never a bad idea to remain skeptical until somebody spends the time and the money to do a randomized trial and, contrary to much of the history of the endeavor to date, fails to refute it"