Correspondence
Public Reporting and Pay for Performance
N Engl J Med 2007; 356:1782-1784April 26, 2007
- Article
To the Editor:
Lindenauer et al. (Feb. 1 issue)1 report that hospital participation in the Centers for Medicare and Medicaid Services (CMS) Premier Hospital Quality Incentive Demonstration (HQID), or pay for performance, was associated with modest but significant improvements in performance on 10 key quality measures. However, before we can conclude that pay for performance benefits patients, it is important to consider the potential for unintended negative effects on quality. Health care providers are responsible for many different aspects of patient care. As suggested in the “multitasking” literature of organizational economics,2,3 compensating providers for only a subgroup of these activities may encourage them to reallocate their efforts toward the rewarded dimensions of quality at the expense of investment in unrewarded dimensions of quality.4 For instance, a well-known response to test-based accountability programs in education has been to shift resources so that schools “teach to the test” and reduce teaching efforts in subjects that are not explicitly tested.5 Given the potential for unintended consequences, it is important to monitor both unrewarded and rewarded aspects of care — and especially patient outcomes — in assessing the overall effect of pay for performance. As we develop better methods for risk adjustment, pay-for-performance programs that directly reward outcomes, instead of process measures, may be more effective in promoting general and sustainable improvements in quality.
5 ReferencesKathleen J. Mullen, Ph.D.
Harvard University, Cambridge, MA 02138
kmullen@rwj.harvard. edu Elizabeth H. Bradley, Ph.D.
Yale School of Medicine, New Haven, CT 065201
Lindenauer PK ,Remus D ,Roman S , et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007;356:486-496
Full Text | Web of Science | Medline2
Eggleston K . Multitasking and mixed systems for provider payment. J Health Econ 2005;24:211-223
CrossRef | Web of Science | Medline3
Prendergast C . The provision of incentives in firms. J Econ Lit 1999;37:7-63
CrossRef | Web of Science4
Krumholz HM ,Normand ST ,Spertus JA ,Shahian DM ,Bradley EH . Measuring performance for treating heart attacks and heart failure: the case for outcomes measurement. Health Aff (Millwood) 2007;26:75-85
CrossRef | Web of Science | Medline5
Jacob BA . Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools. J Publ Econ 2005;89:761-796
CrossRef | Web of Science
To the Editor:
I disagree with Lindenauer et al. that hospitals engaged in pay for performance achieved “greater improvements in quality than did hospitals engaged only in public reporting.” High rates of compliance with CMS quality measures have not been shown conclusively to indicate greater quality, nor have they been associated with better clinical outcomes. Whereas some studies showed that conformance with quality measures was associated with better clinical outcomes for patients,1 others showed either no association2 or differences that were not meaningful.3,4
High adherence to quality measures may be illusory, since physicians and hospitals might have become increasingly clever at documenting questionable contraindications to standard therapies, thereby excluding many patients. Since compliance is almost not optional, it may also represent sophisticated “gaming” of quality measurements. Moreover, it is not known whether this improvement came at the expense of deterioration in unmeasured variables.
A correct conclusion for the study by Lindenauer and colleagues would be that hospitals engaged in pay for performance had an increased rate of compliance with quality-improvement measures. Such a conclusion does not mean that much was achieved.
4 ReferencesIshak A. Mansi, M.D.
Louisiana State University Health Sciences Center, Shreveport, LA 711301
Komajda M ,Lapuerta P ,Hermans N , et al. Adherence to guidelines is a predictor of outcome in chronic heart failure: the MAHLER survey. Eur Heart J 2005;26:1653-1659
CrossRef | Web of Science | Medline2
Philbin EF ,Rocco TA ,Lindenmuth NW ,Ulrich K ,McCall M ,Jenkins PL . The results of a randomized trial of a quality improvement intervention in the care of patients with heart failure. Am J Med 2000;109:443-449
CrossRef | Web of Science | Medline3
Ko DT ,Tu JV ,Masoudi FA , et al. Quality of care and outcomes of older patients with heart failure hospitalized in the United States and Canada. Arch Intern Med 2005;165:2486-2492
CrossRef | Web of Science | Medline4
Fonarow GC ,Abraham WT ,Albert NM , et al. Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA 2007;297:61-70
CrossRef | Web of Science | Medline
To the Editor:
Lindenauer et al. provide important data that help us understand the effect of public reporting and pay for performance. However, the matching of hospitals on the basis of teaching status, number of beds, region, and ownership status misses a major entry criterion and possibly biases the results. All the HQID hospitals were participants in the benchmarking database, known as Perspective, which is maintained by Premier Healthcare Informatics. By definition, this entry criterion includes hospitals that, before the study, had already valued information and presumably had staff and programs that could respond to information, provide feedback, and achieve improvements. If the HQID hospitals had been matched with hospitals that had similar decision-support databases and the staff required to maintain and use such databases to improve care, would the differences have persisted?
Dale N. Schumacher, M.D.
Rockburn Institute, Elkridge, MD 21075
dsrockinst@aol.com To the Editor:
Epstein's editorial that accompanies the report on pay for performance is insightful — in particular, his remarks on the thin link between quality-control measurements and actual high-quality care.1 The recommendation to administer antibiotics within 4 hours after arrival at the hospital in patients with suspected community-acquired pneumonia sounds sensible. However, I can think of nothing that has led to more overuse of parenteral antibiotics and their attendant consequences (e.g., antimicrobial resistance, superinfections, adverse physiological events, and Clostridium difficile–associated colitis) than this ill-conceived “quality” measure.2 Virtually any patient in the emergency department who has a low-grade fever and a cough or pulmonary infiltrate (regardless of the cause) now becomes a victim of this performance measure. I estimate that approximately five patients receive antibiotics unnecessarily in response to this dictum for every one patient with actual pneumonia. Beware the law of unintended consequences.
2 ReferencesBrian W. Cooper, M.D.
Hartford Hospital, Hartford, CT 06102
bcooper@harthosp.org 1
Epstein AM . Pay for performance at the tipping point. N Engl J Med 2007;356:515-517
Full Text | Web of Science | Medline2
Hospital Compare — a quality tool for adults, including people with Medicare. Washington, DC: Department of Health and Human Services, 2006. (Accessed April 5, 2007, at http://www.hospitalcompare.hhs.gov/.)
Author/Editor Response
Any attempt to change a system as complex as an acute care hospital will inevitably result in unintended consequences. We agree with Mullen and Bradley that in an effort to “teach to the test,” hospitals may have neglected the unrewarded aspects of care. Furthermore, we share Cooper's concern that the incentives created by both public reporting and pay for performance may lead to errors of misuse and overuse. Future studies should evaluate these issues, and public reporting and pay-for-performance programs should consider incorporating predictable, unintended effects as balancing measures in an effort to prevent their occurrence. These effects include the erroneous treatment of some patients without pneumonia due to the incentive to administer antibiotics within 4 hours to patients with pneumonia in hospital emergency departments.
Mansi highlights an inconvenient truth about the current emphasis on process measures — namely, existing measures explain only a small fraction of the variation in risk-adjusted outcomes observed among hospitals.1 In addition to several other possibilities, we agree that more thorough documentation of patient ineligibility rather than more frequent use of recommended interventions might explain why improved performance on quality measures does not always lead to improved patient outcomes. Other explanations include nominal adherence to process measures and the possibility that efficacy studies (i.e., randomized trials) have overestimated the benefits of treatment in routine practice.
Ultimately, patients care more about outcomes than processes, and we agree that the incorporation of outcome measures into public reporting and pay-for-performance programs will provide a more comprehensive assessment of quality. Thus, we applaud the plan by the CMS to add 30-day mortality and patient satisfaction measures to the reporting requirements linked to the annual payment update.
Although risk-adjustment techniques for use with administrative data are improving,2 we would not advocate the substitution of outcome measures for process measures, given that the latter provide a valuable framework for focusing the attention of providers on the translation of evidence-based recommendations into practice. Moreover, we think that the ongoing expansion of these measure sets can help mitigate concerns about teaching to the test.
Finally, Schumacher suggests that, although matched for many characteristics, HQID participants probably had a more robust quality-improvement infrastructure than the public-reporting group. Had this been the case, one might have expected HQID participants to have had superior baseline performance also; however, we observed just the opposite — for 8 of the 10 individual measures, baseline performance was worse among the hospitals subscribing to the database maintained by Premier Healthcare Informatics.
2 ReferencesPeter Lindenauer, M.D., M.Sc.
Baystate Medical Center, Springfield, MA 01199
peter.lindenauer@bhs. org Denise Remus, Ph.D., R.N.
BayCare Health System, Clearwater, FL 33760Dale Bratzler, D.O., M.P.H.
Oklahoma Foundation for Medical Quality, Oklahoma City, OK 731341
Bradley EH ,Herrin J ,Elbel B , et al. Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA 2006;296:72-78
CrossRef | Web of Science | Medline2
Krumholz HM ,Wang Y ,Mattera JA , et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation 2006;113:1683-1692
CrossRef | Web of Science | Medline
- Citing Articles (1)
Citing Articles
1
Sosena Kebede, Jeannie Mantopoulos, Shoba Ramanadhan, Emily Cherlin, Mahlet Gebeyehu, Ruth Lawson, Elizabeth H. Bradley. (2012) Educating leaders in hospital management: A pre-post study in Ethiopian hospitals. Global Public Health 7:2, 164-174
CrossRef
