Correspondence
Physician Cost Profiling
N Engl J Med 2010; 363:491-493July 29, 2010
- Article
To the Editor:
Adams et al. (March 18 issue)1 highlight problems in the use of claims data to profile physicians. We agree caution is necessary, but not all “profiling” is problematic. Three modifications can improve estimates: excluding infrequent and expensive items outside the physician's control, focusing on the primary care physician, and being condition-specific. For the profiling of resource use (i.e., cost) related to specific chronic illnesses, we recommend setting aside inpatient care.
We evaluated data on 20,073 year-long episodes of care for patients with diabetes (excluding inpatient costs) in a large medical group in 2006, 2007, and 2008. Costs were assigned to the 152 primary care physicians with whom these patients were associated. We used commercial software (Episode Treatment Groups, version 7.5, from Symmetry) to attribute the cost data to episodes of care and for risk-adjustment measures.
Of the 430 year-specific observations of average costs per primary care physician, 23% were significantly different from zero (P<0.05). Some physicians had yearly per-patient costs that were $750 above and some had per-patient costs that were $750 below the expected levels (see the figure in the Supplementary Appendix, available with the full text of this letter at NEJM.org). It was apparent when examining values for each primary care physician over time that many had quite consistent patterns over the 3 years. When appropriately focused, cost profiles can identify consistent patterns reliably.
Harold S. Luft, Ph.D.
Laura J. Eaton, M.D., M.P.H.
Palo Alto Medical Foundation Research Institute, Palo Alto, CA
lufth@pamfri.org No potential conflict of interest relevant to this letter was reported.
1 References1
Adams JL ,Mehrotra A ,Thomas JW ,McGlynn EA . Physician cost profiling -- reliability and risk of misclassification. N Engl J Med 2010;362:1014-1021
Full Text | Web of Science | Medline
To the Editor:
Adams et al. evaluate the use of episode-grouping tools for measuring the costs of physicians' services. However, the methods used in the study were notably different from recommended, common practices, and therefore the findings are not indicative of the norm. Specifically, the study evaluated the ability of a test to distinguish performance among providers reliably, whereas practical applications involve the use of tests of statistical significance. The same authors of this study used this recommended practice in a separate, related study.1
They also did not restrict measures to patients and conditions that were clinically relevant for the specialty being measured or the cost of care measurement in general, exclude episodes for nonspecific conditions, or consider the use of a procedure-centered method for surgical specialties. Such exclusions are recommended and common practice. Finally, they did not have access to the most recent version of episode-grouping tools that contain more precise severity adjustment, which increases the accuracy of measurement.
Episodes of care are vital to understanding opportunities to improve health care. They are used widely to understand costs, manage patient care, and compare providers. The use of episode-grouping tools remains a trusted, transparent method to support these assessments. We agree that careful measurement involving physicians using episodes of care is critical to accuracy and fairness.
Dan Dunn, Ph.D.
Thomas Lynn, M.D.
Ingenix Eden, Prairie, MNDrs. Dunn and Lynn report being employees of Ingenix, a company that develops and markets provider-cost and quality-measurement tools, including Episode Treatment Groups, Procedure Episode Groups, and Evidence-based Medicine Connect. Ingenix is part of the UnitedHealth Group, a global diversified health care delivery company. No other potential conflict of interest relevant to this letter was reported.
1 References1
Adams JL ,McGlynn EA ,Thomas JW ,Mehrotra A . Incorporating statistical uncertainty in the use of physician cost profiles. BMC Health Serv Res 2010;10:57-57
CrossRef | Web of Science | Medline
To the Editor:
The Clinical Performance Improvement Initiative of the Massachusetts Group Insurance Commission uses administrative data to classify Massachusetts physicians in a tiered system using episode-based cost-efficiency measures and, when sufficient data permit, for quality analysis. My colleagues and I therefore read with interest the article by Adams et al., which focuses on the method of assigning episodes of care to physicians.
The article's assertion that this type of method does not produce reliable results falls short of an adequate examination of cost-efficiency measures in the use of a tiered system, and it does not take into account the considerable improvements in the method that have been made over the past 5 years. For instance, the Group Insurance Commission program requires a minimum of 50 episodes to create the “expected norm” for an episode. RAND does not mention its minimum threshold requirement. Also, instead of a fixed cutoff point, the Group Insurance Commission allows “natural clustering” for its program with some guidelines.
The article's single model of the use of a tiered system to classify physicians does not include on-the-ground experiences with episodes. We have incorporated many such improvements and caution against extrapolating RAND's conclusions to other episode-based models.
Dolores L. Mitchell, B.A.
Group Insurance Commission, Boston, MANo potential conflict of interest relevant to this letter was reported.
To the Editor:
We appreciate the reliability index developed by Adams et al. as a useful assessment of any method for profiling physicians. Unfortunately, the authors did not clearly study any methods that are fully consistent with current industry guidelines, such as those set by the Consumer−Purchaser Disclosure Project and the physician and hospital quality standards of the National Committee for Quality Assurance.
Cigna's method is consistent with these standards, including the use of a more advanced case-mix adjustment (Symmetry software, version 7.5), a minimum of 30 episodes, and classification of physician groups with the use of a 90% statistical confidence approach. In identifying the top 33% of the most cost-efficient physicians among 23 specialties in 10 of our larger markets, the average difference in cost between top-performing physicians and the others is 19%, and the difference according to specialty is statistically significant at the 95% confidence interval for more than 95% of the specialty type−market combinations.
The authors' conclusions do not apply to health plans such as Cigna, which uses an approach that follows current national guidelines.
Jeffrey Kang, M.D., M.P.H.
Richard Salmon, M.D., Ph.D.
Cigna, Bloomfield, CT
dick.salmon@cigna. com Drs. Kang and Salmon report being employees of Cigna, which performs physician profiling. No other potential conflict of interest relevant to this letter was reported.
Author/Editor Response
In an important and rapidly evolving enterprise such as physician cost profiling, it is inevitable that there will be frequent changes in practice. The correspondents describe a variety of methods that may improve their applications; these include minimum sample-size rules, a focus on certain specialties, refined measurement algorithms, and restricted types of episodes. We applaud all of the correspondents' efforts to improve their systems. These are promising avenues for increasing reliability. However, the correspondents do not address the central issue in our article. We believe systems of cost profiling should include a transparent evaluation of reliability and misclassification rates. We would encourage all developers of physician cost-profiling systems to address the key question: “What are the reliabilities of your cost profiles and the resulting misclassification rates of your approach?”
In any system that labels physicians' performance as high or low, the key measure of adequacy is whether the labels are correct; this is true for two-category systems as well as for more elaborate multiple-category systems. To understand misclassification, it is necessary to quantify the reliability of the measures of physician performance. The ultimate criterion of whether improvements to a system are substantially better is the magnitude of the improvement in misclassification rates.
Several of the correspondents advocate the use of statistical testing. We discussed this point in a previous article.1 However, statistical tests are not a solution to the problem of low reliability. The primary feature of statistical tests in physician profiling is the exchange of one type of misclassification for another. Statistical tests may be used to increase the proportion of physicians labeled as “high cost” who indeed offer high-cost care. But the price is that more physicians who actually offer high-cost care are labeled as “average” or “low cost.”
Our article may have left the impression that we are fundamentally opposed to physician cost profiling. This is not true. We do question whether the current state of the art identifies high-performance and low-performance physicians well enough to drive change in the right way. We hope that greater transparency of the details of these systems and a careful public consideration of their reliability will lead to improvements and ultimately contribute to “bending the cost curve” in health care.
John L. Adams, Ph.D.
Ateev Mehrotra, M.D., M.P.H.
J. William Thomas, Ph.D.
RAND, Santa Monica, CA
mehrotra@rand.org Since publication of their article, the authors report no further potential conflict of interest.
1 References1
Adams JL ,McGlynn EA ,Thomas JW ,Mehrotra A . Incorporating statistical uncertainty in the use of physician cost profiles. BMC Health Serv Res 2010;10:57-57
CrossRef | Web of Science | Medline
