Correspondence
Comparing Hospitals
N Engl J Med 1999; 340:2006-2007June 24, 1999
- Article
To the Editor:
Chen et al. (Jan. 28 issue)1 found that mortality rates for patients with myocardial infarction were lower at top-ranked hospitals than at others. Although the study was well done, it is premature to conclude that hospital rankings by commercial magazines are valuable. This conclusion depends on three assumptions that must be validated.
The first assumption is that the risk-adjustment procedure was adequate. Even good risk-adjustment procedures may not take into account systematic differences in risk among hospitals. Possible sources of systematic differences include variations in the following factors: the response time of paramedical services and the technology for diagnosis and treatment in the field, the distance to the hospital, cultural attitudes that affect how quickly patients seek medical attention, and admissions of patients from nursing homes or other facilities with high mortality rates. Since the study found that the top-ranked hospitals had lower predicted mortality rates than the other hospitals, it is possible that patients may access these hospitals more quickly. The analysis might be improved by adjusting for the duration of symptoms or by comparing mortality rates during a period that starts the day after admission.
The second assumption is that hospital rank is a useful distinction and not simply a proxy for another easily identified characteristic that is more strongly tied to outcome. Hospital rank may be a proxy for geographic region. When we compared rates of death associated with coronary-artery bypass surgery, we found that within New York and Pennsylvania, there were no differences between top-ranked and other hospitals. Other easily identified hospital characteristics that may account for the difference in mortality rates between top-ranked and other hospitals are the volume of patients, teaching status, and availability of staff cardiologists and intensivists throughout the night. All these factors have been found to be associated with the quality of medical care.
The third assumption is that the results for patients with myocardial infarction can be extended to patients with other conditions. Our results with patients undergoing coronary-artery bypass surgery suggest that this is not the case.2
Our comments are not intended as criticisms of the well-done study by Chen et al. Like the results of all single observational studies, however, their results should be viewed with skepticism. Because they appeal to special interests, these results and extrapolations of them may be accepted with less than the usual degree of caution.
2 ReferencesArthur Hartz, M.D., Ph.D.
Mark Graber, M.D.
Brad Doebbeling, M.D.
University of Iowa School of Medicine, Iowa City, IA 52242-10971
Chen J ,Radford MJ ,Wang Y ,Marciniak TA ,Krumholz HM . Do “America's Best Hospitals“ perform better for acute myocardial infarction? N Engl J Med 1999;340:286-292
Full Text | Web of Science | Medline2
Hartz AJ ,Pulido JS ,Kuhn EM . Are the best coronary artery bypass surgeons identified by physician surveys? Am J Public Health 1997;87:1645-1648
CrossRef | Web of Science | Medline
To the Editor:
Although we applaud the subject of your editorial “Hospitals, Heal Yourselves” (Jan. 28 issue)1 and agree with much of its content, we disagree with your assertion that comparing one type of hospital with another is a worthwhile goal. Our experience has convinced us that there is little to be gained by comparing the performance of categories of hospitals. We believe the principal goals of comparative measurement should be to enhance public accountability and improve quality. Using measurements of performance to generate lists of “best hospitals” undermines the goal of improvement in quality and leads to generalizations of the kind expressed in your editorial (“nonteaching hospitals have a substantial edge over teaching hospitals in many of the human dimensions of care”).
The report on the survey sponsored by the Massachusetts Health Quality Partnership, which you cite in your editorial, presented the results of the survey in great detail — without interpretation or labeling of hospitals. Your observation that “major teaching hospitals . . . scored substantially lower than many hospitals with a nonteaching or a minor teaching role” is not a conclusion stated in the report. Most hospitals in the survey had 21 different scores representing seven dimensions of care in three types of services (medical, surgical, and maternity). The report contained no single composite score and no rankings. In fact, because the survey scores have a margin of error (presented as a confidence interval), a definitive ranking of the hospitals is not even possible. Major teaching hospitals, minor teaching hospitals, and nonteaching hospitals were found throughout the distribution of scores (high, middle, and low). The important lesson from this project is that hospitals of any kind have the capability to meet their patients' most important needs. There are teaching hospitals that excel in the human dimensions of care, just as there are community hospitals that deliver excellent clinical services.
1 ReferencesWilliam L. Lane
Jeffrey Otten
Ronald M. Hollander
Massachusetts Hospital Association, Burlington, MA 01803-50961
Kassirer JP . Hospitals, heal yourselves. N Engl J Med 1999;340:309-310
Full Text | Web of Science | Medline
Author/Editor Response
The authors reply:
To the Editor: Hartz and colleagues raise important issues in the interpretation of the findings of our study of the treatment of myocardial infarction in “America's Best Hospitals.” Their concern that our findings may be promoted by organizations with vested interests, despite the fact that there was much overlap between top-ranked and other hospitals, is well founded.1 However, we wish to reemphasize that the intent of our study was not to endorse “America's Best Hospitals” as a valid assessment of quality of care, but rather to determine whether differences in the processes of care at these hospitals were associated with differences in outcomes.
Because the severity of illness varied according to the type of hospital, the choice and comprehensiveness of a risk-adjustment strategy are important. Although we used a validated risk-adjustment method specifically developed for myocardial infarction in Medicare patients,2 the possibility of a residual confounding effect from unmeasured factors cannot be excluded. However, further analyses taking into account nursing home admissions, the distance from home to hospital, and deaths occurring after the first hospital day did not affect the results substantively.
It is possible that characteristics common to high-ranking hospitals were responsible for the better outcomes and higher quality processes of care at these institutions. Identifying these characteristics could be an important contribution to the development of strategies to improve the care provided at all hospitals. Our study did adjust for the volume of patients and teaching status, and an additional analysis that controlled for the census region did not alter the results.
Although the results of our study may not be generalizable to other cardiac (or noncardiac) diseases, the concept that the processes involved in the care of patients after acute myocardial infarction are associated with the outcome is one that can potentially be applied to a large number of medical conditions. That substantial opportunities exist for all hospitals to do better, even those among “America's Best,” should give us further incentive to focus on process measures in the effort to improve the quality of care.
2 ReferencesJersey Chen, B.A.
Harlan M. Krumholz, M.D.
Yale University School of Medicine, New Haven, CT 06520Martha J. Radford, M.D.
Yale–New Haven Health, New Haven, CT 065041
Comarow A. The `Best Hospitals' are best. U.S. News and World Report. Vol. 126. 1999:67.
2
Daley J ,Jencks S ,Draper D ,Lenhart G ,Thomas N ,Walker J . Predicting hospital-associated mortality for Medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. JAMA 1988;260:3617-3624
CrossRef | Web of Science | Medline
Author/Editor Response
Dr. Kassirer replies:
The executives of the Massachusetts Hospital Association claim that comparing quality indicators among hospitals is not a worthwhile endeavor. I disagree. No matter what their intention was in presenting multiple raw scores, the data invite interesting comparisons. Reliable comparative information on the quality of care is hard to come by, but when it is available, institutions, groups of physicians, or even single practitioners can use it as a yardstick against which to judge their own performance. Although it is true that on certain measures individual teaching hospitals outperformed nonteaching hospitals (and vice versa), the trends I described are quite apparent.
Jerome P. Kassirer, M.D.
