Perspective
Observational Studies of Drug Safety — Aprotinin and the Absence of Transparency
N Engl J Med 2006; 355:2171-2173November 23, 2006
- Article
The full safety profile of a new drug is rarely known at the time of approval by the Food and Drug Administration (FDA). Most drug-development programs designed for treatments of symptomatic indications are underpowered to detect any increased risk of rare drug reactions or change in background event rates attributable to the drug. Large, post-marketing, randomized, controlled trials provide robust data on drug safety but may be subject to multiple sources of bias. Observational studies of a drug's effects in clinical practice can offer additional information on risks. The recent discussions of aprotinin (Trasylol, Bayer) by the Cardiovascular and Renal Drugs Advisory Committee of the FDA, which I chair, provide insight into the strengths and weaknesses of using observational data to assess drug safety and highlight the importance of using a transparent and open process when reviewing such data.
Post-marketing observational studies permit the evaluation of drug safety in a large number of patients in a real-world setting, where practice patterns, including off-label use, can be assessed. Such studies have limited ability to determine causation, but they can detect signals that may suggest a safety concern.
In an observational study, decisions to use specific drugs are made by local physicians, according to their perceptions of the risks and benefits for particular patients. Such treatment allocation results in an imbalance in demographic and risk factors between patients given the drug of interest and those given an alternative or no treatment. An imbalance in various clinical factors will directly influence safety outcomes when patients at higher risk receive the drug. Therefore, appropriate statistical methods must be used to adjust for the nonrandom assignment to treatment groups.1
One way to address imbalances between groups is to use a propensity score that incorporates confounders and other covariates into a model predicting the probability of assignment to a particular treatment. This score can be used for adjustment or for matching patients who have similar probabilities of receiving a treatment. Differences in outcomes between treated and untreated patients (or patients treated with a comparison drug) with equal propensity scores provide a less biased estimate of treatment effect.
Aprotinin was approved by the FDA in 1993 as a means of reducing perioperative blood loss and the need for blood transfusion in patients undergoing coronary-artery bypass grafting. Neither the clinical trial database that led to approval nor the numerous randomized, controlled trials conducted after approval identified an association between aprotinin and any short-term increase in the risk of death or nonfatal cardiovascular events or any serious renal toxic effects (except for a transient increase in the serum creatinine concentration). However, in early 2006, two observational studies were published that raised serious concerns about the drug's safety.2,3
One study, by Mangano et al.,2 evaluated 4374 patients undergoing coronary-artery bypass surgery. End points were prospectively defined, and data on a large number of clinical variables were collected for each patient. The decision to use aprotinin, aminocaproic acid, or tranexamic acid to inhibit fibrinolysis or to withhold antifibrinolytic therapy was made by the treating physician. Since group assignment was not random, it is not surprising that patients assigned to the group receiving aprotinin were at much higher risk for adverse cardiovascular outcomes than were the other patients in the study, who were treated with alternative therapies. When a propensity score was used in a logistic-regression model to adjust for these baseline differences, aprotinin was associated with a nonsignificant increase in the risk of death (odds ratio, 1.59; 95% confidence interval [CI], 0.76 to 3.34) and with significant increases in the risks of renal events (odds ratio, 2.34; 95% CI, 1.27 to 4.31), cardiovascular events (odds ratio, 1.42; 95% CI, 1.09 to 1.86), and cerebrovascular events (odds ratio, 2.15; 95% CI, 1.14 to 4.06). The authors concluded that “the observed association between aprotinin and serious end-organ damage indicates that continued use is not prudent.”
The second study, by Karkouti et al.,3 compared the risks associated with aprotinin with those associated with tranexamic acid in 898 patients undergoing high-risk cardiac surgery. The authors used a propensity score to match patients who had been given the different treatments — an approach that permitted the identification of a population of patients who were subsequently fully matched according to a large number of covariates. Calculations from the study data, performed by the Colorado Prevention Center, indicated no significant difference between the two treatments in the overall risk of myocardial infarction (odds ratio, 1.20; 95% CI, 0.52 to 2.75), stroke (odds ratio, 1.15; 95% CI, 0.56 to 2.40), or death (odds ratio, 0.91; 95% CI, 0.56 to 1.46). However, among patients who had abnormal renal function at baseline, those given aprotinin had a significantly increased risk of postoperative renal dysfunction (odds ratio, 1.69; 95% CI, 1.07 to 2.69). Because this study included fewer patients than the study by Mangano et al., it had less power to detect a safety problem.
These observational studies did not permit a definitive conclusion about cardiovascular or renal risk, but they did raise concern. The FDA appropriately issued a public health advisory about the potential risks posed by aprotinin and urged physicians to monitor their patients carefully for renal, cardiac, and cerebral toxic effects.4 The agency also convened a meeting of the Cardiovascular and Renal Drugs Advisory Committee on September 21, 2006, to evaluate the evidence on the cardiovascular and renal toxic effects of aprotinin. Our committee reviewed the published studies and the global safety and efficacy database submitted by Bayer, which included 45 randomized, controlled trials involving 2249 patients who received aprotinin and 2164 who received placebo.
The Bayer safety analysis was confined to the unadjudicated adverse events reported within 7 days after the administration of aprotinin, which included 520 deaths or serious cardiovascular or renal events. The results did not reveal any increased risk of fatal or nonfatal cardiovascular events. According to the Colorado Prevention Center's calculations based on Bayer's data, the odds ratio for death was 1.14 (95% CI, 0.80 to 1.62); for myocardial infarction, 1.17 (95% CI, 0.93 to 1.49); for stroke, 0.71 (95% CI, 0.42 to 1.18); and for renal failure, 1.15 (95% CI, 0.74 to 1.78). Two studies in which myocardial infarctions were adjudicated by an independent review committee showed a nonsignificant increase in the risk of myocardial infarction, with a point estimate of the odds ratio as high as 2.24 (95% CI, 0.56 to 9.00).
How can the results of these trials and the study by Karkouti et al. appear to be so different from the findings of Mangano et al.? Part of the problem may derive from the inability of observational studies to identify and measure all relevant covariates that may influence the outcome; similar studies using a few different covariates can sometimes come to opposite conclusions.
FDA advisory committee meetings usually include an independent review of data — either those provided by the sponsor or those used by the authors of relevant published studies. This independent and transparent process becomes even more critical in the evaluation of observational studies, whose conclusions can be highly influenced by the statistical methods used and the inability to identify all confounding factors. Unfortunately, Mangano did not give the FDA or the committee full access to his data, which would have allowed the agency to perform an independent review and analysis to validate his group's findings. Although there are many legitimate concerns with regard to the sharing of data (for instance, confidentiality and the need for informed consent), the lack of independent review greatly limited the committee's ability to draw conclusions from the study. Having reviewed all the data available, the committee decided that there was insufficient evidence to require an additional warning on aprotinin's labeling and agreed that the clinical data supported an acceptable safety and efficacy profile for aprotinin.
Days after the committee meeting, the FDA was made aware of additional observational data from the sponsor that had not been presented at the meeting. Bayer evidently had commissioned an observational study involving 67,000 patients who were given aprotinin.5 According to the initial FDA review of data from that study, aprotinin may be associated with “increased risk for death, kidney failure, congestive heart failure and stroke.” The failure of Bayer to disclose all its data on aprotinin seriously undermined the advisory committee process and hindered the safety review.
Although observational studies cannot be definitive (and so should evoke measured responses), they can provide important new safety information that can direct further scientific and regulatory actions — if their findings can be confirmed. Since further analysis of the new data provided by Bayer is ongoing at the FDA, conclusions about the overall safety of aprotinin cannot be drawn at this time. Still, the aprotinin story demonstrates that full disclosure and a transparent process are essential in evaluating the findings of all studies germane to drug safety and the public health.
Source Information
Dr. Hiatt is a professor of medicine at the University of Colorado School of Medicine and president of the Colorado Prevention Center — both in Denver — and is the current chair of the Cardiovascular and Renal Drugs Advisory Committee of the FDA. Opinions expressed in this article are those of the author and do not necessarily reflect the opinions of the FDA or the advisory committee.
- References
References
1
Rubin DB . Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-763
Web of Science | Medline2
Mangano DT ,Tudor IC ,Dietzel C . The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354:353-365
Full Text | Web of Science | Medline3
Karkouti K ,Beattie WS ,Dattilo KM , et al. A propensity score case-control comparison of aprotinin and tranexamic acid in high-transfusion-risk cardiac surgery. Transfusion 2006;46:327-338
CrossRef | Web of Science | Medline4
Food and Drug Administration. Public health advisory: aprotinin injection (marketed as Trasylol). Updated September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/cder/drug/advisory/aprotinin.htm.)
5
Idem. FDA statement regarding new Trasylol data. September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/bbs/topics/NEWS/2006/NEW01472.html.)
- Citing Articles (23)
Citing Articles
1
Marco Ranucci, Solomon Aronson, Wulf Dietrich, Cornelius M. Dyke, Axel Hofmann, Keyvan Karkouti, Marcel Levi, Gavin J. Murphy, Frank W. Sellke, Linda Shore-Lesserson, Christian von Heymann. (2011) Patient blood management during cardiac surgery: Do we have enough evidence for clinical practice?. The Journal of Thoracic and Cardiovascular Surgery 142:2, 249.e1-249.e32
CrossRef2
Frédéric Jacques, Ismail El-Hamamsy, Annik Fortier, Simon Maltais, Louis P. Perrault, Moishe Liberman, Nicolas Noiseux, Pasquale Ferraro. (2011) Acute renal failure following lung transplantation: risk factors, mortality, and long-term consequences. European Journal of Cardio-Thoracic Surgery
CrossRef3
N. A. Goldenberg, A. C. Spyropoulos, J. L. Halperin, C. M. Kessler, S. Schulman, A. G. G. Turpie, A. M. Skene, N. R. Cutler, W. R. Hiatt, . (2011) Improving academic leadership and oversight in large industry-sponsored clinical trials: the ARO-CRO model. Blood 117:7, 2089-2092
CrossRef4
F Fleischmann, C Matuschek, K Orth, PA Gerber, R Mota, WT Knoefel, M Peiper, M Schick, M van Griensven, E Bdlke, W Fleischmann. (2011) Aprotinin and classic wound drainage are unnecessary in total hip replacement a prospective randomized trial. European Journal of Medical Research 16:1, 20
CrossRef5
Jerrold H. Levy, Richard P. Dutton, J Claude Hemphill, Aryeh Shander, David Cooper, Michael J. Paidas, Craig M. Kessler, John B. Holcomb, Jeffrey H. Lawson. (2010) Multidisciplinary Approach to the Challenge of Hemostasis. Anesthesia & Analgesia 110:2, 354-364
CrossRef6
Alan Lyles. (2009) Adapting pharmacoeconomic data and methods to extend the postpublication utility of analyses. Clinical Therapeutics 31:1, 192-193
CrossRef7
Ariane Maleszka, Georg Kleikamp, Armin Zittermann, Maria R.G. Serrano, Reiner Koerfer. (2008) Simultaneous Aortic and Mitral Valve Replacement in Octogenarians: A Viable Option?. The Annals of Thoracic Surgery 86:6, 1804-1808
CrossRef8
Ray, Wayne A., Stein, C. Michael, . (2008) The Aprotinin Story — Is BART the Final Chapter?. New England Journal of Medicine 358:22, 2398-2400
Full Text9
&NA;. (2008) Mortality Associated With Aprotinin During 5 Years Following Coronary Artery Bypass Graft Surgery. Survey of Anesthesiology 52:2, 56-57
CrossRef10
Dean R. Jones, H. Thomas Lee. (2008) Perioperative renal protection. Best Practice & Research Clinical Anaesthesiology 22:1, 193-208
CrossRef11
Ray, Wayne A., . (2008) Learning from Aprotinin — Mandatory Trials of Comparative Efficacy and Safety Needed. New England Journal of Medicine 358:8, 840-842
Full Text12
Mario Kluth, Jan U Lueth, Armin Zittermann, Markus Lanzenstiel, Reiner Koerfer, Kazuo Inoue. (2008) Safety of Low-Dose Aprotinin in Coronary Artery Bypass Graft Surgery. Drug Safety 31:7, 617-626
CrossRef13
Wulf Dietrich, Anna Ebell, Raimund Busley, Anna-Laure Boulesteix. (2007) Aprotinin and Anaphylaxis: Analysis of 12,403 Exposures to Aprotinin in Cardiac Surgery. The Annals of Thoracic Surgery 84:4, 1144-1150
CrossRef14
Stacy Voils. (2007) Pharmacologic Interventions for the Management of Critical Bleeding. Pharmacotherapy 27:9 Part 2, 69S-84S
CrossRef15
(2007) Current awareness: Pharmacoepidemiology and drug safety. Pharmacoepidemiology and Drug Safety 16:6, i-xii
CrossRef16
Mannucci, Pier Mannuccio, Levi, Marcel, . (2007) Prevention and Treatment of Major Blood Loss. New England Journal of Medicine 356:22, 2301-2311
Full Text17
Carl-Johan Jakobsen. (2007) Strategy of transfusion in cardiac surgery: limits of hematocrit and how much is too low?. Future Cardiology 3:2, 141-151
CrossRef18
Neel R Sodha, Munir Boodhwani, Frank W Sellke. (2007) Is There Still a Role for Aprotinin in Cardiac Surgery?. Drug Safety 30:9, 731-740
CrossRef19
R. O. Feneck. (2007) Clinical research in anaesthesia; randomized controlled trials or observational studies?. European Journal of Anaesthesiology 24:01, 1
CrossRef20
R. O. Feneck. (2007) Clinical research in anaesthesia; randomized controlled trials or observational studies?. European Journal of Anaesthesiology 24:1, 1-5
CrossRef21
Avorn, Jerry, . (2006) Dangerous Deception — Hiding the Evidence of Adverse Drug Effects. New England Journal of Medicine 355:21, 2169-2171
Full Text22
Drazen, Jeffrey M., . (2006) Research Replication. New England Journal of Medicine 355:21, 2252-2253
Full Text23
(2006) Judging the Safety of Aprotinin. New England Journal of Medicine 355:21, 2261-2262
Full Text
