Correspondence
Cost-Effectiveness Analyses
N Engl J Med 1995; 332:123-125January 12, 1995
- Article
To the Editor:
The Journal's policy on cost-effectiveness analyses (Sept. 8 issue)1 does more to harm the field of cost-effectiveness research than it will do to improve its validity or quality.
The new policy is based on several erroneous assumptions. The first is that there is more discretion on the part of investigators and greater opportunity for bias in economic and cost-effectiveness analyses than in clinical research. In fact, there is plenty of discretion in both. Discretionary decisions about which outcome measures to use in clinical research, for example, can easily determine whether one treatment is found to be safer or more effective than another.
The second assumption, implied in some parts of your editorial, is that all cost-effectiveness analyses involve the development of models, which in turn are put into operation with secondary data and numerous assumptions. By failing to distinguish consistently between cost-effectiveness analyses based on complex models and those based on clinical and economic data collected in randomized trials, you contribute to the misconception that such analyses cannot be “original scientific” studies.
The third erroneous assumption, and perhaps the most far-reaching in its potential effect, is that receiving a grant from a for-profit sponsor influences the probability of bias in the research being performed, independently of the contractual arrangements under which the research is conducted. Specifically, you assert that, given identical contractual arrangements, bias is less likely when a nonprofit institution conducts the research. I do not think this is the case. The contractual arrangements under which research is conducted, in contrast, may differ between nonprofit and for-profit institutions. This distinction should not be lost.
Because of these misconceptions, your new policy is flawed and misguided. In particular, your policy, under which the Journal will not even consider a cost-effectiveness analysis performed by a for-profit entity, regardless of the stringency of the contractual arrangement under which the research is conducted, is neither a sensitive nor a specific strategy for weeding out biased research. The scientific community, industry, and society would be far better served if you advocated stringent contractual arrangements for both clinical and cost-effectiveness research, regardless of who performs it, and relied on your peer reviewers to distinguish sound from unsound research of any type.
1 ReferencesEarl P. Steinberg, M.D.
Health Technology Associates, Washington, DC 20005-39341
Kassirer JP ,Angell M . The Journal's policy on cost-effectiveness analyses. N Engl J Med 1994;331:669-670
Full Text | Web of Science | Medline
To the Editor:
Cost-effectiveness analyses that undergo peer review should be critically assessed not only for the rigor (validity and completeness) of the design and measurement of clinical outcomes but also for the rigor of the design and measurement of economic outcomes. Therefore, these analyses probably require additional reviewers who are knowledgeable about the various measures that can be used to estimate the true costs associated with medical interventions from different perspectives. Such reviewers must also be able to evaluate critically and objectively the merits of the cost measures used.
Given the increasing importance of cost-effectiveness analyses, journal editors and medical educators should encourage reviewers and readers to become more discriminating evaluators of articles involving economic measures, a subject currently given limited emphasis in medical training. This is essential if physicians are to contribute meaningfully to the debate about how increasingly scarce resources are best allocated.
Neil R. Powe, M.D., M.P.H.
Johns Hopkins Medical Institutions, Baltimore, MD 21205-2223To the Editor:
Pharmacoeconomic research increasingly involves the development of economic analyses as secondary end points of phase 3 clinical trials. In these studies, the economic analysis is integrated into the clinical trial. The economic data are reported in summary fashion with the clinical trial and in a separate manuscript. If academic investigators analyze the economic data, the conduct of the research is usually governed by a sponsored-research agreement with the investigator's institution. These prospective economic analyses are different from the retrospective studies you describe in your editorial. In fact, one recent prospective economic study that was funded and conducted by a pharmaceutical company and a private consultant was reported in the September 8 issue of the Journal, 1 seemingly in conflict with your new policy. Although the Journal may not be the appropriate forum for all reports of studies involving this newer type of pharmacoeconomic research, they should be considered separately from the studies you describe in your editorial.
As a research community, we should be very uncomfortable singling out one type of research for special treatment in the peer-review process. At a minimum, this practice may provide readers with a false sense of reliability about other types of data reported in the medical literature. The Journal should improve disclosure by all investigators and sponsors and consider establishing clear policies for the conduct of all reported studies. This could be accomplished through expansion of the financial disclosure required of investigators, with specification of how the study was conducted and analyzed and whether the sponsor required written approval of the manuscript before submission.2
2 ReferencesKevin Schulman, M.D.
Georgetown University Medical Center, Washington, DC 200071
Fuchs HJ ,Borowitz DS ,Christiansen DH , et al. Effect of aerosolized recombinant human DNase on exacerbations of respiratory symptoms and on pulmonary function in patients with cystic fibrosis. N Engl J Med 1994;331:637-642
Full Text | Web of Science | Medline2
Schulman K ,Sulmasy DP ,Roney D . Ethics, economics, and the publication policies of major medical journals. JAMA 1994;272:154-156
CrossRef | Web of Science | Medline
To the Editor:
The Journal's policy on publishing cost-effectiveness articles errs in denying consideration to all studies in which any author is an employee of, consultant for, or stockholder in a sponsoring company, regardless of how well the studies meet the other criteria. This bias against industry-sponsored cost-effectiveness research is inconsistent with the Journal's policy on industry-sponsored clinical research, and another, nondiscriminatory remedy is available.
National guidelines for cost-effectiveness studies have recently been developed for Canada1 and the United Kingdom.2 Neither set of guidelines discriminates against industry-sponsored cost-effectiveness research. Both sets require widespread use of sensitivity analysis. With this technique, for each major assumption about cost and effectiveness, plausible alternative assumptions are modeled. Sensitivity analysis prevents authors from presenting only a best-case scenario.
2 ReferencesWilliam L. Simonich, Ph.D.
Amersham Healthcare, Arlington Heights, IL 600051
Guidelines for economic evaluation of pharmaceuticals, Canada. Ottawa, Ont.: Canadian Coordinating Office for Health Technology Assessment, 1994.
2
Joint Strategy Group of the Government and the Association of the British Pharmaceutical Industry. Guidelines on good practice in the conduct of economic evaluation of medicines. London: Association of the British Pharmaceutical Industry, May 20, 1994.
To the Editor:
Your refusal to review studies of health economics if any of the authors have a financial conflict of interest is based on the false assumption that for-profit research has the unique potential for being tainted. Many other sources of potential bias exist, and a number of studies have documented the effect of nonfinancial motives on scientific research.1-3 For example, academic researchers may have a biased interest in particular analytic approaches or in securing research funding for their departments, just as editors may be biased toward publishing reports of positive results.
The determination of a study's merit should be based not on who did the study but on the soundness of the methods used and the full disclosure and transparency of the research methods, analyses, and results.
3 ReferencesHugh Tilson, M.D., Dr.P.H.
James G. Kotsanos, M.D.
Pharmaceutical Research and Manufacturers of America, Washington, DC 200051
Fisher B ,Redmond CK . Fraud in breast-cancer trials. N Engl J Med 1994;330:1458-1460
Full Text | Web of Science | Medline2
Weaver D ,Albanese C ,Costantini F ,Baltimore D . Retraction: altered repertoire of endogenous immunoglobulin gene expression in transgenic mice containing a rearranged mu heavy chain gene. Cell 1991;65:536-536
CrossRef | Web of Science | Medline3
Darsee J . A retraction of two papers on cardiomyopathy. N Engl J Med 1983;308:1419-1419
Web of Science | Medline
To the Editor:
Have you considered the possibility that under your new policy authors receiving direct funding from industry (through a salary, equity interest, or paid consultancy) might remove their names from their own manuscripts lest their commercial affiliations make the manuscripts less attractive? In other words, they might become ghost authors.
For example, a clinical research associate at a pharmaceutical company who participated sufficiently in the analysis and writing of the manuscript to take public responsibility for it might, after reading your policy, request that his or her name be removed from the manuscript before submission. You would have no way of knowing this occurred. In addition to your request for written assurance of the independence of all authors, perhaps you should require assurance that no ghosts are lurking in the bylines.1 The goal of minimizing industry bias in cost-effectiveness analyses is commendable, but can you control such influences any more than you can identify ghost authors?
1 ReferencesAnnette Flanagin, R.N., M.A.
Drummond Rennie, M.D.
, Chicago, IL 606101
Rennie D ,Flanagin A . Authorship! Authorship! Guests, ghosts, grafters, and the two-sided coin. JAMA 1994;271:469-471
CrossRef | Web of Science | Medline
Author/Editor Response
The editors reply:
Financial conflicts of interest differ from other sources of bias in that they are readily identified and not an inherent part of the research enterprise. Our newly formulated policy therefore excludes from consideration only cost-effectiveness analyses whose authors have a financial conflict of interest, including direct salary support from the sponsor, an equity interest in the company, an ongoing consultancy, membership on the scientific advisory board, or a related patent pending. The policy does not exclude economic analyses that are embedded in randomized, controlled clinical trials.
Several of our correspondents argue that reviewers should be able to identify bias in cost-effectiveness analyses, yet given the discretionary nature of the items to which costs are attributed in such analyses, even the most skilled reviewers could easily fail to notice a flaw (intentional or not) that favored a particular outcome. Sensitivity analysis cannot overcome a lack of objectivity when a study's design is flawed. We are not certain what Dr. Powe means by “true costs,” but we doubt that the term has an accepted meaning.
Ms. Flanagin and Dr. Rennie wonder how we will handle manuscripts from people who might lie. Omitting the name of a qualified author is just as culpable as including the name of someone who is not qualified. As always, we will assume that our authors are honest, knowing that there is very little protection for any of us from a well-concealed lie.
Dr. Schulman's final recommendations on disclosure, identifying a study's methods, and specifying the relation between the sponsor and the researchers are the same as the conditions we outlined for considering cost-effectiveness analyses. Needless to say, we agree with them.
We continue to believe that, as we propose in our editorial, an arm's-length relationship between researchers and industry, which stands to gain from cost-effectiveness analyses, would go a long way toward reducing the opportunity for bias in such studies.
Jerome P. Kassirer, M.D.
Marcia Angell, M.D.- Citing Articles (5)
Citing Articles
1
Scott Chavers, Daniel Fife, Mary Wacholtz, Paul Stang, Jesse Berlin. (2011) Registration of Observational Studies: perspectives from an industry-based epidemiology group. Pharmacoepidemiology and Drug Safety 20:10, 1009-1013
CrossRef2
Bertrand Janne d’Othée, Michael F. Morris, Richard J. Powell, Michael A. Bettmann. (2008) Cost Determinants of Percutaneous and Surgical Interventions for Treatment of Intermittent Claudication from the Perspective of the Hospital. CardioVascular and Interventional Radiology 31:1, 56-65
CrossRef3
Bertrand Janne d'Othée, David R. Langdon, Gregory K. Bell, Michael A. Bettmann. (2006) Operating Expenses for the Diagnosis and Treatment of Peripheral Vascular Disease in an Academic Interventional Radiology Department: Cost Calculations According to a Microeconomic Method. Journal of Vascular and Interventional Radiology 17:1, 85-94
CrossRef4
Bryan R. Luce, Karl Claxton. (1999) Redefining the analytical approach to pharmacoeconomics. Health Economics 8:3, 187-189
CrossRef5
Kassirer, Jerome P., . (1995) My Years at the Journal — So Far. New England Journal of Medicine 333:10, 654-655
Full Text
