My conversation with my former student exactly mirrored discussions I have had with other scientists in pharma and biotech in this regard: a high proportion of the output from academic labs can't be reproduced and is thus worse than useless. "Worse" because it sends companies down dead-end avenues, wasting money and scientific talent that otherwise could be pursuing potential disease therapies with a chance of working. An article in the Wall Street Journal, quoting Bruce Alberts (a prominent scientist and former editor of a prestigious research journal), puts it this way:
Two years ago, a group of Boston researchers published a study describing how they had destroyed cancer tumors by targeting a protein called STK33. Scientists at biotechnology firm Amgen Inc. quickly pounced on the idea and assigned two dozen researchers to try to repeat the experiment with a goal of turning the findings into a drug.
It proved to be a waste of time and money. After six months of intensive lab work, Amgen found it couldn't replicate the results and scrapped the project.
I was disappointed but not surprised," says Glenn Begley, vice president of research at Amgen of Thousand Oaks, Calif. "More often than not, we are unable to reproduce findings" published by researchers in journals.
This is one of medicine's dirty secrets: Most results, including those that appear in top-flight peer-reviewed journals, can't be reproduced. "It's a very serious and disturbing issue because it obviously misleads people" who implicitly trust findings published in a respected peer-reviewed journal, says Bruce Alberts, editor of Science. . . .
Drug manufacturers rely heavily on early-stage academic research and can waste millions of dollars on products if the original results are later shown to be unreliable. Patients may enroll in clinical trials based on conflicting data, and sometimes see no benefits or suffer harmful side effects.The WSJ article cites a study by the pharmaceutical company Bayer that found that company scientists had to halt 64% of early drug projects based claims in the literature due to inability to replicate published results:
A lot has been written about scientific fraud and how (unsurprisingly) it is probably more prevalent in the highest-prestige journals, about how the scientific literature is biased towards positive results and about the pressures of "publish or perish." I don't have anything new to say about these pressures on academic scientists: pressures that underlie the problem of there being so much garbage in the literature. I will only say this: the "soft money" system that makes scientists' livelihood dependent on winning grants plus the current brutal funding environment is a perfect formula for ensuring increased levels of scientific cheating and the resulting waste and damage to efforts to cure diseases.