David O. Warner, M.D.

hen new drugs or devices are introduced into clinical practice, industry sponsors most of the animal experiments and many of the clinical trials that support the regulatory approval processes. Funding from industry may now be more important than ever to academic medical centers, as their financial situation has deteriorated over recent years — although academic centers are no longer the only game in town, as commercial enterprises such as contract research organizations vigorously compete for the business of clinical research.¹ Does the source of funding matter when you evaluate a study? For example should you be suspicious of an industry-funded study that suggests great benefit for its product? For that matter, how can you trust the results of any study, no matter who funds it?

Maintaining Integrity
There are several safeguards in place to promote the integrity of clinical research, including an array of regulations from bodies such as the federal Office for Human Research Protections and the Office of Research Integrity, standards promulgated by the Food and Drug Administration as part of the drug and device approval process and many others. Although imperfect, the peer-review process at the time of publication provides a more or less (depending on the journal and the reviewers) rigorous final check of the results. Nonetheless any system can be subverted, and there have been spectacular examples of data falsification, although such incidents still appear to be very much the exception.²

Spin Doctors
Even if outright fraud is unusual, more subtle (or sometimes not so subtle) biases can creep into papers. There is always more than one way of telling a story, and investigators may be just as susceptible to the temptation to “spin” their results as politicians. Very few studies, regardless of the sponsor, are planned with a negative result in mind. There may be legitimate differences of opinion regarding methods of analysis or interpretations of that analysis. On the other hand, in many cases, a positive result in a clinical trial may lead to millions, or in some cases billions, of dollars in sales. With the stakes so high, investigators with even the best of intentions must guard their integrity carefully. Guidelines regarding issues such as trial design, access to data and publication rights that help protect the independence of academic centers exist but are often not followed.³ Practices such as “ghostwriting,” in which persons employed by the sponsor actually analyze the data and write the paper without being named as authors, are particularly insidious, allowing sponsors to provide their own unvarnished presentation of the results in the guise of an “independent” study.¹ Of course, in the case of studies performed by commercial research organizations, the sponsor has total control over all aspects of the study, including the crafting of its presentation.

Judging the Results
Consumers of research papers can follow several principles to help them evaluate the results of industry-sponsored trials — or any trial, for that matter. The elements of well-performed clinical research have been codified in several forms, such as the Consolidated Standards of Reporting Trials, or CONSORT, recommendations that provide a relatively simple checklist that readers can use to judge for themselves about the quality of the trial design.⁴ We generally do a poor job of teaching young physicians how to evaluate clinical trials, so it is little wonder that so many of us are vulnerable to deception by shoddy work.

Readers also should always check the sources of funding and the stated conflicts of interest (stock ownership, etc.) of the authors. If the results of the study have the potential to further the financial interests of the sponsor or the authors, it does not necessarily invalidate the results. Indeed all authors have an interest in publishing interesting results regardless of who funds the study. It would be naïve, however, to ignore potential conflicts of interest, a fact widely recognized by disclosure policies designed to identify such conflicts. It is particularly important to identify these conflicts because conclusions favorable to industry are more likely both when industry sponsors the study and when authors have financial conflicts of interest.^5,6 Unfortunately disclosures are often incomplete and many times do not detail the role of the sponsor in study design, data analysis and manuscript preparation.⁷

The Science of PR
In our media-driven society, research findings are often trumpeted as definitive scientific “breakthroughs” with immediate relevance to our lives. We as physicians can fall into the same trap of being dazzled by the “latest and greatest” research results. Unfortunately few studies are truly definitive, and more often than not, the accumulation of several studies is required to truly define the clinical utility of new diagnostic or therapeutic modalities. It is thus important to place research results in the context of the rest of the pertinent literature, which is difficult to do in a brief sound bite, an infomercial or a print advertisement promoting a product. Indeed in our own specialty, we have seen examples where new technologies are promulgated via skilled marketing rather than good science. It also is unwise to rely upon manufacturers’ representatives as guides to the medical literature, as few of them have the background to do so and all have obvious incentives to present their product in the best possible light.

Sometimes what is not published is more important than what is published. Clinical trials showing no difference between treatment groups usually do not further the interests of the sponsor and thus may not be published. Efforts are under way to mandate the registry of clinical trials before they commence so that at least the existence of these trials can be verified.⁸ This is still a work in progress, and it will likely be some time before these registries are of practical value to clinicians.

Much industry-sponsored research is carefully performed and honestly reported. Nonetheless, because the purpose of industry is to generate a profit for shareholders, this potential for financial conflict of interest means that the prudent physician will keep this in mind when evaluating the results of these studies.

The best answer to the question of how to evaluate industry-sponsored studies may be President Ronald Reagan’s invocation of a Russian proverb in reference to arms control treaties with the Soviet Union: “doveryay, no proveryay” — trust, but verify.

References:
1. Bodenheimer T. Uneasy alliance: Clinical investigators and the pharmaceutical industry. N Eng J Med. 2000; 342:1539-1544.
2. Martinson BC, Anderson MS, de Vries R. Scientists behaving badly. Nature. 2005; 435:737-738.
3. Schulman KA, Leils DM, Timbie JW, et al. A national survey of provisions in clinical-trial agreements between medical schools and industry sponsors. N Eng J Med. 2002; 347:1335-1341.
4. Moher D, Schulz K, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001; 285:1987-1991.
5. Friedman LS, Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med. 2004; 19:51-56.
6. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA. 2003; 289:454-465.
7. Gross CP, Gupta AR, Krumholz HM. Disclosure of financial competing interests in randomized controlled trials: Cross sectional review. Brit Med J. 2003; 326:526-527.
8. DeAngelis CD, Drazen JM, Frizelle FA, et al. Clinical trials registration: A statement from the international committee of medical journal editors. JAMA. 2004; 292:1363-1364.

David O. Warner, M.D., is Professor of Anesthesiology, Mayo Clinic College of Medicine, and Vice-Chair for Research, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota.