Correspondence

Variation in Hospital Mortality Associated with Surgery

N Engl J Med 2010; 362:78-79January 7, 2010

Article

To the Editor:

The analysis of variation in hospital mortality by Ghaferi et al. (Oct. 1 issue)1 requires further scrutiny. The primary exposure variable used by these authors was risk-adjusted mortality. However, there are difficulties in applying risk-adjustment methods to compare patient outcomes across multiple hospitals; these difficulties result from risk relations and interactions that are not constant. Different methods of adjustment for severity frequently show different results with regard to the hospital rankings generated.2 In comparisons of standardized hospital mortality ratios, there is even evidence that case-mix adjustment may paradoxically increase the bias.3 Moreover, the size of the hospitals, which was not stated, could be expected to affect their ranking because, as compared with the larger hospitals, smaller hospitals would disproportionately appear among the very-high-mortality and very-low-mortality quintiles. The effect of the study size on the event rate is readily apparent in a funnel plot.4

James C. Hurley, M.D., Ph.D.
University of Melbourne, Melbourne, VIC, Australia
jamesh@bhs.org.au

No potential conflict of interest relevant to this letter was reported.

4 References

1. 1

   Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med 2009;361:1368-1375
   Full Text | Web of Science | Medline

2. 2

   Iezzoni LI. The risks of risk adjustment. JAMA 1997;278:1600-1607
   CrossRef | Web of Science | Medline

3. 3

   Mohammed MA, Deeks JJ, Girling A, et al. Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals. BMJ 2009;338:b780-b780
   CrossRef | Web of Science | Medline

4. 4

   Hurley JC. Profound effect of study design factors on ventilator-associated pneumonia incidence of prevention studies: benchmarking the literature experience. J Antimicrob Chemother 2008;61:1154-1161
   CrossRef | Web of Science | Medline

To the Editor:

In their article, Ghaferi et al. infer a significant difference in the ability to respond to and manage surgical complications among hospitals based on the relatively high variation in surgical mortality as compared with the more similar incidence of surgical complications. First, given that the number of deaths was small as compared with the number of complications, might the higher variation in surgical mortality be due to the fact that variance is an inverse function of the square root of the number under study, such that one would expect much greater variance in deaths as compared with complications? Second, do individual hospitals remain stably within the quintiles of hospital mortality described, or do they move about? The latter would suggest that the variation in mortality is simply a matter of random statistical distribution.

T. Flint Gray, III, M.D.
Appalachian Regional Healthcare System, Boone, NC
tfgray@apprhs.org

No potential conflict of interest relevant to this letter was reported.

Author/Editor Response

Hurley and Gray raise questions about our risk-adjustment methods. Our study benefited from the use of a very robust, clinically detailed data set — the American College of Surgeons National Surgical Quality Improvement Program. Taking full advantage of the more than 130 clinical variables collected by trained data abstractors through this program, our risk-adjustment model had a C statistic of 0.88 with excellent discrimination. The correspondents also raise the possibility that relationships between patient factors and outcomes could vary across hospitals. Although they are theoretically possible, we found no evidence of systematic interactions between risk factors and outcomes among the hospital quintiles in our study.

Hurley also raises questions about the statistical reliability of mortality estimates at individual hospitals related to issues with sample size. When studying individual operations, we are concerned about sample size.1 However, our study addressed this problem by combining multiple different operations performed at each hospital. As a result, in our study there were sufficient numbers of patients treated at hospitals to ensure the adequate reliability of our mortality estimates (the average number of patients per hospital was 455, and the average mortality rate was 5.1%). To confirm this finding, we repeated our analysis in this study and other studies2 by applying methods for reliability adjustment,3 and we found identical results.

Amir A. Ghaferi, M.D.
John D. Birkmeyer, M.D.
Justin B. Dimick, M.D., M.P.H.
University of Michigan, Ann Arbor, MI
aghaferi@umich.edu

Since publication of their article, the authors report no further potential conflict of interest.

3 References

1. 1

   Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size. JAMA 2004;292:847-851
   CrossRef | Web of Science | Medline

2. 2

   Ghaferi AA, Birkmeyer JD, Dimick JB. Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients. Ann Surg 2009;250:1029-1034
   CrossRef | Web of Science | Medline

3. 3

   Dimick JB, Staiger DO, Baser O, Birkmeyer JD. Composite measures for predicting surgical mortality in the hospital. Health Aff (Millwood) 2009;28:1189-1198
   CrossRef | Web of Science | Medline