A Prescient Thought by C.S. Lewis

The 50th anniversary of the death of C.S. Lewis was three days ago (the same day as the 50th anniversary of the death of JFK). I saw the passage from his writings that I have pasted below in a number of different articles and blog posts noting the occasion, including one by Steven Hayward (currently a scholar at the University of Colorado). As Hayward notes, it was prescient:

• Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron’s cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience.

"Hooey" Part 2: Harold Varmus Weighs In

How hard is it going to be to change the culture of scientists judging their peers by whether they make the effort to get their work in the journals Cell, Science etc.?  I don't know. Based on the discussions of grants and investigators that took place at the most recent NIH study section (meeting of scientists to evaluate NIH grant applications) I attended, the tendancy of many towards judgments based on controversial measurements of journal prestige is well-ingrained.  This bias was discussed in my last post.

I don't want to belabor the point, but I will anyway. This is from a recent article in Science on the prominent biomedical scientist Harold Varmus.  He has held a number of prominent positions, and is now head of the National Cancer Institute:

• Varmus also wants to address what he calls the "flawed values system" that the competitive atmosphere has spawned. He laments that researchers feel they will win funding only if they publish in top journals such as Science, Nature, and Cell. At NCI, he is pilot-testing a way to change this part of the scientific culture: by revising the "biosketch," the summary of a researcher's record that accompanies a grant proposal. Varmus wants to replace a section that now lists major publications with a narrative describing the investigator's five major accomplishments.This approach, already used by the Howard Hughes Medical Institute (HHMI), should not only discourage reviewers from assessing their colleagues based only on where they're published; it will also help researchers on large teams whose names are buried in a list of authors receive the credit they deserve, Varmus says. And he thinks that easing the rush to publish in high-impact journals might help address a much-discussed problem: that many NIH-funded studies have proved difficult to reproduce.  (emphasis mine)

Incidentally, I didn't realize until reading the article that Varmus was an important player in bringing about the doubling of the NIH budget over 5 years beginning about 1999. Paradoxically, this doubling contributed greatly to the subsequent funding woes, now worse than ever. Indeed, the article candidly states that the doubling helped bring about today's funding problems. Varmus claims that the doubling wasn't a mistake: he knew that the rapid increase in the NIH budget could cause problems but thought these could be avoided by facilitating a "soft landing." This "soft landing" was to be afforded by continuing budget increases that "kept pace with rising costs . . . ." This was plainly naive. How could anyone who observes how the government operates expect this type of  long-term rational budgeting? Of course there was no soft landing. The biomedical work force and the number of new grants funded was greatly expanded, but these good times were, sure enough, followed my a massive hangover when it turned out there wasn't enough money to keep the party going.

A follow-up to the previous post regarding "hooey" in the scientific literature

I was talking to a prominent, accomplished biochemist at my institution about a possible collaboration. I mentioned a key paper supporting the scientific premise, and she asked me where it was published. When I gave her the name of the solid but non-prestige journal where it appeared, she said something along the lines of: "Oh good, it wasn't in Cell. That means its not a bunch of hooey." My paraphrasing is accurate, but the only word I distinctly remember is "hooey."  Hence, the title of this follow-up to the immediately preceding post.

A surprisingly huge amount of BS ends up in the scientific literature. Many of the reasons for this are well-known and fairly obvious: for example, intense pressures to publish, difficulties publishing negative results, and academic and grant review committees that simply count numbers of papers published rather than assess content and contribution. Given the competition and politics involved in publishing in the very highest prestige journals (such as Cell), is it any wonder that misrepresentation and outright fraud is even more prevalent in these periodicals? Leaders in the field are urging us to stop paying attention to measurements of journal prestige and focus more on the content of what is published: basically, to stop the games surrounding publishing in Cell, Science, etc.  This is highly commendable. But it is going to be hard to change this attribute of biomedical scientific culture.

In any case, yet more articles have come out accurately reporting on what is becoming less our dirty little secret and more our public scandal.  Yet another article.  And a video discussion.

Cheating, tweaking the data and the view of academic biomedical output from industry

I talk occasionally with an exceptionally bright former Ph.D. student of mine who works now in pharma. I wanted my next post to be positive--there is no lack of good things to say about the nation's biomedical research enterprise--but a conversation I just had with him prompted this post instead.

My conversation with my former student exactly mirrored discussions I have had with other scientists in pharma and biotech in this regard: a high proportion of the output from academic labs can't be reproduced and is thus worse than useless.  "Worse" because it sends companies down dead-end avenues, wasting money and scientific talent that otherwise could be pursuing potential disease therapies with a chance of working.  An article in the Wall Street Journal, quoting Bruce Alberts (a  prominent scientist and former editor of a prestigious research journal), puts it this way:

Two years ago, a group of Boston researchers published a study describing how they had destroyed cancer tumors by targeting a protein called STK33. Scientists at biotechnology firm Amgen Inc. quickly pounced on the idea and assigned two dozen researchers to try to repeat the experiment with a goal of turning the findings into a drug. 

It proved to be a waste of time and money. After six months of intensive lab work, Amgen found it couldn't replicate the results and scrapped the project.

I was disappointed but not surprised," says Glenn Begley, vice president of research at Amgen of Thousand Oaks, Calif. "More often than not, we are unable to reproduce findings" published by researchers in journals.

This is one of medicine's dirty secrets: Most results, including those that appear in top-flight peer-reviewed journals, can't be reproduced. "It's a very serious and disturbing issue because it obviously misleads people" who implicitly trust findings published in a respected peer-reviewed journal, says Bruce Alberts, editor of Science. . . .

Drug manufacturers rely heavily on early-stage academic research and can waste millions of dollars on products if the original results are later shown to be unreliable. Patients may enroll in clinical trials based on conflicting data, and sometimes see no benefits or suffer harmful side effects. 

The WSJ article cites a study by the pharmaceutical company Bayer that found that company scientists had to halt 64% of  early drug projects based claims in the literature due to inability to replicate published results:

A lot has been written about scientific fraud and how (unsurprisingly) it is probably more prevalent in the highest-prestige journals, about how the scientific literature is biased towards positive results and about the pressures of  "publish or perish." I don't have anything new to say about these pressures on academic scientists: pressures that underlie the problem of there being so much garbage in the literature. I will only say this: the "soft money" system that makes scientists' livelihood dependent on winning grants plus the current brutal funding environment is a perfect formula for ensuring increased levels of scientific cheating and the resulting waste and damage to efforts to cure diseases.

Learning from Spitzer and Weiner

When I'm in the hospital being a doctor, I'm treated respectfully by staff, patients, students, nurses, other docs, etc. My job entails interacting with others in a professional manner and working with them towards common goals. There is a hierarchy on the hospital wards and in clinic, and I am generally at or near the top. Being constantly critiqued by colleagues is not part of clinical work.

On the other hand, when I'm wearing my academic researcher hat, I'm constantly being critiqued, evaluated, scored, criticised, etc. In theory, it's my academic and scholarly work that's being scrutinized, but it's hard not to take it personally. I am subject to scrutiny when I submit manuscripts for publication, when I'm considered for academic promotion and when I submit grant applications. Part of being a molecular biologist is being told your grant is not good enough to be funded, your manuscript needs more work or is outright rejected, and receiving other various sorts of criticism--usually anonymous--on a frequent basis. Grants, of course, do get funded, but rejection is a more common outcome. Manuscripts do get published, but again rejection is frequent. When papers are not rejected, there is often harsh criticism of the submitted draft: ok, not necessarily harsh, but to those without thick skin, even gentle criticism can often sound harsh. It helps greatly to have thick skin in this business*.

In my quest for ways to thicken my skin, I've found Anthony Weiner and Eliot Sptizer to be inspirational. If I had done what they had and had been subject to the same public humiliation and condemnation, I am not sure how I would ever have overcome the embarrassment. I can imagine hiding in a closet and wanting to disappear. If I were out in public, I would constantly be thinking about what other people were likely thinking about me. In short, it would be a disaster for my self-esteem. Weiner and Spitzer, however, are back to work in politics, out on the campaign trail and seemingly with their high regard for themselves intact, with nary a hint of shame. This is the pinnacle of thick skin, of unshakable self-esteem. And guess what: both have very strong chances of winning. Spitzer and Weiner should be an inspiration for every bioscientist who has ever moped for more than a couple of hours after a grant or manuscript rejection (which means, I think, a great many of us).


*DrugMonkey put it more succinctly:

I think our career / academics sets an environment in which you are constantly told you aren't doing quite enough, aren't quite excellent/brilliant enough and gee, we could probably get along without you, chumpie.

Some doubts arising about the Medical Scientist Training Program? Part 2

An NIH Physician-Scientist Workforce Working Group has been formed and charged with gathering data and making recommendations about how to best sustain the physician-scientist biomedical workforce. The Physician-Scientist Working Group has not met yet, but there was a discussion regarding its goals at the June meeting of the Advisory Committee to the NIH Director (ACD). You can view a video of the ACD meeting here; you may want to skip right to the part about the Physician-Scientist Workforce Working Group. Most of the preceding discussion focuses on gloomy budgetary news and the resulting career difficulties facing biomedical researchers. The relevant segment begins at about the hour thirty-one minute (1:31) mark and runs about 10 minutes.

In Part 1 of this post, I noted that questions about the future of the MSTP program are being raised at the highest levels of the NIH. By "highest levels" I meant by the NIH Director himself. The Director, Francis Collins, asked the first question about the working group, which was whether there would be a focus on the "whole question about MD/Ph.D. training programs as a major part of [the working group's] deliberations." This is because "there are folks who are really beginning to wonder if we did the right thing by making this [MD/PhD training programs] such a centerpiece of our effort to train physician-researchers." Interesting.

The Advisory Committee and Dr. Collins should be lauded for instigating this analysis of physician-scientist training. As I alluded to in Part 1, I think the data may show that a surprisingly large fraction of MD/Ph.D. graduates end up in positions in which the benefit derived from the Ph.D. training, if any, is far outweighed by the cost in time and in taxpayer funds of that training. Indeed, the representative of the working group noted that combined clinical degree/PhD programs training dentist-researchers and veterinarian-researchers seem to be more successful than MD/PhD programs in turning out biomedical clinician-researchers. In any case, it will be interesting to see the data when it becomes available.

Maybe some doubts arising about the Medical Scientist Training Program? Part 1

I recently completed a several-year stint on my institution's Medical Scientist Training Program (MSTP) admissions committee. The MSTP is an NIH-funded program that pays for combined M.D./Ph.D. training at various research-oriented medical schools. The applications I saw--from the subset of candidates who were offered interviews--were highly (scarily) impressive: they sometimes (OK, often) made me wonder whether I would have been admitted had I applied in recent years.

The applicants I interviewed uniformly stated they envisioned a career in academic medicine with something like 20% of time devoted to clinical work and the rest being devoted to research (with teaching and administrative work also thrown into the mix). This distribution of effort conforms to the classic model of an academic physician-scientist. Out of curiosity (not as part of the formal interview), I asked a few of the applicants if they knew where their salary would typically come from if they pursued such a career. I was surprised that only one applicant (of the four I asked) was familiar with "soft money" (this applicant's parent was an academic physician).

I stumbled upon an online student forum with many participants applying to MSTP programs or at various stages of MD/Ph.D. training. It's not surprising to see that after they learn about the soft money system and become aware of its downsides, many of the trainees begin to reevaluate things, thinking much more concretely about how they want to structure their careers and about trade-offs involving risk, stress, job security, etc. We should probably ensure that students contemplating applying for MSTP positions are made aware of the practical realities of a career as a clinician-scientist up front.

My impression--based not on any formal analysis but rather on my first-hand knowledge of the career paths of other MD/Ph.D grads and my perusal of residency match and alumni information on the MSTP websites of various medical schools--is that a large fraction of MD/Ph.D. graduates go on to careers in which the ~4 years of intensive research training that goes into the Ph.D. is unnecessary and a waste of public funds. This could be the subject of an entire post or two, and I'm not going to go into detail here; just one observation along these lines. The nature of my graduate school work would have made me an ideal candidate for a dermatology residency. My research mentor (himself a physician-scientist) made it clear to me that he hoped I did not take this route. Why? Because, I would encounter sirens, the lifestyle and pay of dermatology private practice, that would lure my research career to its doom. As others have noted, it seems now that an increasing number of MD/Ph.D. candidates leverage the caché of their Ph.D. degrees to gain admittance to competitive residency programs in clinical specialties--such as dermatology, ophthalmology and radiology-- known for good lifestyle and lucrative salaries. Over 40% of MD/PhD graduates choosing dermatology, ophthalmology or radiology, for example, end up in private practice.

I recently learned that there are people paying attention attention to all this and that questions about the future of the MSTP program are being raised at the highest levels of the NIH. I will provide specifics in my next post.*

*Update: part 2 now posted.