Gaffes and Leaks Invade Journals' Ivory Tower | ||||||
| ||||||
BOSTON, April 11 -- The integrity of the medical publication process has been called into question by a series of embarrassing incidents over the past two years.
For example, just last month, it was revealed that authors of a high-profile 2006 New England Journal of Medicine paper on CT screening for early-stage lung cancer had failed to disclose patents they held on related technology as well as the fact that a tobacco company had provided much of their funding.
The NEJM is not the only top journal to be embarrassed.
In the space of two months in 2006, authors of three separate papers in the Journal of the American Medical Association -- on arthritis, cancer, and migraine treatment -- belatedly acknowledged funding and consulting relationships with firms selling arthritis, cancer, and migraine drugs, which they had not disclosed to the journal's editors.
The journal system has traditionally been based on trust -- that authors will report their results fairly and honestly and that editors and reviewers will live up to their responsibilities.
"Medical journals are an extraordinary public institution," said Harold Sox, M.D., of Dartmouth, editor of the Annals of Internal Medicine. "People look to us to decide whether research is true or not."
Pointed out Steven Nissen, M.D., of the Cleveland Clinic, "Our colleagues and patients can only trust in evidence-based medicine if they trust in the evidence. To the extent that problems undermine trust, they undermine the entire scientific process."
According to Jerome P. Kassirer, M.D., NEJM's editor-in-chief from 1991 to 1999, "The situation [with researchers' conflicts of interest] if anything has gotten worse since I was editor of the journal."
In this special investigation, MedPage Today took a close look at some of these problems and potential solutions.
Disclosure: Falling Short of the Goal
Virtually all the major and many lesser journals have adopted some type of disclosure policy in the past 20 years. The NEJM's approach is typical.
"For a journal to be of value, it must publish authoritative, up-to-date information that is free of commercial influence. ... [We] have set policies to ensure that the financial associations of authors are disclosed and that published articles are not influenced by those financial associations," according to the NEJM policy. "These important editorial policies have raised awareness about conflict of interest in medical research and have helped prevent bias -- and the appearance of bias -- stemming from authors' financial interests."
Because financial relationships carry the most obvious potential for bias and are easy to characterize, the NEJM's disclosure form, like those of most journals, focuses on them.
It asks all manuscript authors to declare such relationships in six specific categories, as well as any other type of arrangement that "could compromise the interpretation of the article" if not disclosed.
"The required documentation from the journals is really excellent," said Dr. Nissen. The problem, he and others said, is that authors don't always follow the spirit and the letter of those standards.
The case of Claudia I. Henschke, M.D., Ph.D., of Weill Cornell Medical College in New York, is an example. She was at the center of furor last month when the New York Times revealed that her research on CT scanning to detect early-stage lung cancer was funded by tobacco-industry money (See: Early Lung Cancer Study Found Funded by Tobacco Money).
It also turned out that she and a Cornell colleague, David F. Yankelevitz, M.D., held royalty-paying patents on CT-based tumor detection technology.
The Cornell researchers did not disclose the tobacco funding to the NEJM or other journals where they published aspects of their research.
Dr. Henschke said the patents had been disclosed to NEJM, but editors there decided not to include them in the published disclosures. Other journals only found out about the patents from news reports after the articles were published.
Cornell's administration would not allow the researchers to speak with MedPage Today, and a NEJM spokeswoman declined to discuss the matter, citing its confidentiality policy.
But according to Bruce A. Chabner, M.D., of Boston, the editor of The Oncologist, which published a paper from the Cornell group last January, the researchers initially did not believe they were required to disclose their patents.
Dr. Chabner said he learned about the patents from press reports just as the January issue was going to press. He got in touch with the Cornell researchers, who told him, he said, that because the technologies covered by the patents were not explicitly mentioned in their article, they were not obliged to disclose them.
The Oncologist's disclosure form specifies that researchers should report financial relationships involving products or patents "that are the subject of the matter under consideration."
Likewise, the NEJM directs authors to report relationships to "products studied or discussed in the article and companies that make related products."
According to press reports, the NEJM's editors initially agreed with their position, apparently switching signals only after they learned about the tobacco-company funding.
Dr. Chabner's response was to publish a letter from the Cornell researchers acknowledging the patents, along with an editorial suggesting the investigators should have known better.
"While there is no explicit reference to the software patents or biopsy needles, it is equally clear that the widespread adoption of CT screening as the standard of care for the early detection of lung cancer would increase the commercial value of these patents and products," Dr. Chabner wrote.
"It seems to us that the readership of a scholarly journal deserves to know when an author could reasonably be expected to derive personal benefit from the results of his/her paper."
Similar disagreements were at the root of the 2006 controversies over JAMA study disclosures. In each case, the authors indicated that they believed their relationships were under the journal's radar.
Catherine DeAngelis, M.D., JAMA's editor, declined to be interviewed for this article. She told the Associated Press in 2006 that authors should "always err on the side of full disclosure," and she would decide what is relevant.
JAMA's written policy calls for authors to report "all conflicts of interests, including specific financial interests and relationships relevant to the subject matter or materials discussed in the manuscript."
Adriane Fugh-Berman, M.D., a complementary medicine researcher at Georgetown University in Washington, who has also published on physician ethics and drug marketing, says such phrasing is a mistake.
"There should never be wording about 'relevant' conflicts," she said, because a researcher's judgment on relevance may not jibe with editors or readers.
Disclosure, however, "fixes the wrong problem," according to Dr. Kassirer, the former editor of the NEJM.
The real issue, he said, is not the extent of disclosure of conflicts of interest -- it's the conflicts themselves. He pointed to research indicating that company-sponsored studies are more likely to find in favor of the company's products.
Asked if what he reads in the major journals is less trustworthy than 10 or 15 years ago, he responded, "Yes. I trust them less."
Disclosure isn't enough to deal with this problem, he said. Journal editors need to root out bias by alerting reviewers to authors' industry ties so they can be watchful for bias.
Others maintain that disclosure is the critical element.
Eric Campbell, Ph.D., a health policy researcher at Harvard Medical School, acknowledged that the disclosure system sometimes fails, but he pointed out that it is part of a larger system that checks the influence of pharmaceutical companies and others seeking to influence the journal process.
"It's a whole bunch of regulatory mechanisms that make this work," he said.
He also cautioned against excessively high expectations of medical journals. "What's written in journals are claims of truth," he said, and readers ultimately must judge for themselves whether they may be biased or incomplete.
Another Harvard researcher, Christopher Cannon, M.D., said a possible solution to questions of conflicts of interest would be an online disclosure registry along the lines of www.clinicaltrials.gov. Researchers would file and update a single statement listing all their relationships. Journals could access the statement when a researcher submits an article, and publish those relationships it deems relevant. Such a system would be easier for everyone, he said.
George Lundberg, M.D., editor-in-chief at Medscape Journal of Medicine and a former editor of JAMA, agreed that, all in all, disclosure policies are generally serving their main goals.
"The whole journal enterprise consists of requiring trust between the authors and the editors and the peer reviewers," he said. "I think most of the time it actually works."
Still, Dr. Lundberg added, "there are enough instances where it clearly doesn't, that you have to be pretty cautious about it."
Sanctity of Peer Review
One of the foundations of the peer review system is that reviewers should be anonymous and that they don't discuss the unpublished data to which they are privy.
Twice in the past year, though, the sanctity of peer review has been violated.
In March 2007, Martin B. Leon, M.D., of Columbia University, gave a series of public talks in which he criticized the COURAGE study, which found that coronary artery stents were essentially co-equal with medical therapy for stable angina.
Trouble was, the study had not yet been published. He had reviewed the manuscript for the NEJM, and said so in one of his pre-publication talks.
In May, another NEJM reviewer, Steven Haffner, M.D., of the University of Texas Health Sciences Center in San Antonio, sent his review copy of a paper on rosiglitazone (Avandia) to the drug's manufacturer (See: COURAGE Embargo Break: Slip of the Tongue or Sabotage? and Leaked NEJM Manuscript on Rosiglitazone (Avandia) Puts GSK Under Scrutiny).
Dr. Nissen said he worries about whether peer review can continue on its traditional trust basis. As well he might, because it was his paper on rosiglitazone that Dr. Haffner leaked.
He said he was astonished when he heard about the incident, which was revealed publicly earlier this year. "I shred [review papers he receives] so that they're not laying around where someone might read them."
Dr. Lundberg said he was not surprised that a peer reviewer would break confidentiality. "What's surprising is that so many don't break it," he said. "It's against human nature."
He pointed out that peer reviewers are supposed to receive potentially world-changing information, which they are to keep absolutely secret, along with their role in the review process. If the paper is rejected, they have to pretend they never saw it. "This is an astonishing thing to ask," he said.
"The fact that it works so well, most of the time, is to me a wonderful commentary on the professionalism of our academic community in the peer review process," he said.
Others interviewed by MedPage Today declined to suggest that the two episodes amounted to a breakdown of the peer-review process generally. "That would be reading the tarot cards too closely," said Dr. Kassirer, the former NEJM editor.
Crime and Punishment
Opinions differ on what could be done to encourage uniform adherence to the rules protecting journals' integrity.
Dr. Kassirer said journals can ask tough questions about the research design and statistical analysis. He said some of the major journals have started to do so, requesting details of protocols and commissioning independent statistical reviews.
In 2006, the Associated Press quoted Dr. DeAngelis as saying, "I'm not God and I'm not the FBI," when asked whether JAMA should investigate author disclosures.
Others interviewed by MedPage Today also found such policing out of bounds or too difficult.
Dr. Nissen and Dr. Lundberg agreed it would be impractical for journals to spend resources on policing disclosures, except in cases where there is a specific reason for suspicion. Dr. Lundberg pointed out that many smaller journals don't have full-time staffs.
"The answer is not to have a police force involved at the journals, Dr. Nissen said. "They just can't operate that way." Instead, Dr. Nissen said, violators should be punished, harshly and publicly.
Although NEJM and other journals say they have a sanctions policies for dealing with authors who don't follow disclosure rules, the NEJM, for one, keeps such penalties confidential. Sanctions the journal may have imposed on Dr. Leon, Dr. Haffner, or the Cornell researchers were not disclosed.
"The editorial process is strictly confidential," said a NEJM spokeswoman. "If a breach of ethics occurs, the editors handle it privately."
That's part of the problem, according to Dr. Nissen.
He believes public punishment is necessary to get researchers to take the policies seriously. "We have to make [violating policies] absolutely so unacceptable, and the price for doing that sufficiently high, that physicians just won't do it."
He suggested that authors' own institutions should impose penalties and say so.
"If the publicity around that is sufficiently painful, then I think people will not [violate policies]," Dr. Nissen said.
"I do think we need a code of conduct here," he added.
Dr. Campbell, the Harvard health policy researcher, was skeptical that much could be done to increase adherence to disclosure policies. "The ultimate burden lies with the scientists," he said.
Officials at the University of Texas Health Sciences Center would not comment on whether Dr. Haffner had been formally reprimanded or otherwise sanctioned, citing confidentiality of personnel issues.
Similarly, Cornell officials had no comment on whether Drs. Henschke and Yankelevitz might be penalized. In fact, the university issued a statement rebutting suggestions that the researchers had done anything wrong.
MedPage Today was able to learn independently that NEJM's editors informed Dr. Leon that he would no longer be asked to serve as a peer reviewer or invited to submit commentaries, perspectives, or editorials. The editors also requested Columbia to consider academic sanctions (See: COURAGE Embargo Break: Slip of the Tongue or Sabotage?).
Columbia declined to comment on whether it had imposed sanctions on Dr. Leon.
Dr. Sox, the Annals of Internal Medicine editor, said that more than formal policies are needed to maintain the integrity of research reporting.
"The biggest tool we have is the passion of the editor to get it straight, to find all the slanted writing and to make sure that all conflicts are reported," he said.
He added that the group-based editorial policy at the Annals helps to keep standards high. "People have to convince other people that an article should be published," he said. "This group process, getting input from lots of people, one of whom may see something no one else sees that turns out to be really important, is a key element in maintaining the integrity of the process." | ||||||
| Related Article(s):
|
April 14, 2008
The medical journals are now being scrutinized
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment