Skip to main content
Device Global Header

Subscribe
or Renew

Advanced Search

Special Article

Who Is at Greatest Risk for Receiving Poor-Quality Health Care?

List of authors.
  • Steven M. Asch, M.D., M.P.H.,
  • Eve A. Kerr, M.D., M.P.H.,
  • Joan Keesey, B.A.,
  • John L. Adams, Ph.D.,
  • Claude M. Setodji, Ph.D.,
  • Shaista Malik, M.D., M.P.H.,
  • and Elizabeth A. McGlynn, Ph.D.

Abstract

Background

American adults frequently do not receive recommended health care. The extent to which the quality of health care varies among sociodemographic groups is unknown.

Methods

We used data from medical records and telephone interviews of a random sample of people living in 12 communities to assess the quality of care received by those who had made at least one visit to a health care provider during the previous two years. We constructed aggregate scores from 439 indicators of the quality of care for 30 chronic and acute conditions and for disease prevention. We estimated the rates at which members of different sociodemographic subgroups received recommended care, with adjustment for the number of chronic and acute conditions, use of health care services, and other sociodemographic characteristics.

Results

Overall, participants received 54.9 percent of recommended care. Even after adjustment, there was only moderate variation in quality-of-care scores among sociodemographic subgroups. Women had higher overall scores than men (56.6 percent vs. 52.3 percent, P<0.001), and participants below the age of 31 years had higher scores than those over the age of 64 years (57.5 percent vs. 52.1 percent, P<0.001). Blacks (57.6 percent) and Hispanics (57.5 percent) had slightly higher scores than whites (54.1 percent, P<0.001 for both comparisons). Those with annual household incomes over $50,000 had higher scores than those with incomes of less than $15,000 (56.6 percent vs. 53.1 percent, P<0.001).

Conclusions

The differences among sociodemographic subgroups in the observed quality of health care are small in comparison with the gap for each subgroup between observed and desirable quality of health care. Quality-improvement programs that focus solely on reducing disparities among sociodemographic subgroups may miss larger opportunities to improve care.

Introduction

The quality of American medical care falls short of expectations.1,2 We have previously reported that U.S. adults receive about half of recommended health care services.3 How much variation is there among population subgroups in the likelihood of receiving needed care? Previous studies have found that patients who are women, older, members of racial and ethnic minorities, poorer, less educated, or uninsured are less likely to receive needed care, largely as a result of lack of access to care (determined by whether an encounter with a health care provider occurs), rather than a deficiency in the quality of care (determined by whether the encounter is necessary and provides the recommended service).4-11 Previous studies have focused on a narrow set of quality indicators and conditions in selected populations and have had a limited ability to adjust for the range of factors associated with poorer quality. Few studies have examined quality across the continuum of care for multiple conditions.

To address these issues, we assessed the quality of care in a large, community-based sample of patients who had made at least one visit to a health care provider in the previous two years. Using data from medical records and patient surveys, we measured technical process quality (whether patients were offered recommended services) across a broad spectrum of care for the leading causes of death and disability, while controlling for multiple sociodemographic characteristics and other determinants of quality. We examined the relationship between patients' characteristics and several domains of quality.

Methods

The methods of recruiting participants, performing measurements, and collecting data have been described previously3,12-14 and are summarized here.

Recruitment of Participants

Between October 1998 and August 2000, we telephoned persons who had participated in a random-digit-dial household survey conducted by the Community Tracking Study (CTS) in 12 large metropolitan areas.3,15 The participants answered questions about their experiences with the health care system and gave us written informed consent to obtain photocopies of their medical records from all health care providers seen in the previous two years.

Among the 20,028 adults in the starting sample, 2091 (10 percent) were ineligible, primarily because they had left the area. Among the 17,937 eligible adults, 13,275 (74 percent) participated in the survey. Of these, 863 (7 percent) had not visited a health care provider in the previous two years. Among the 12,412 participants who had made a visit, 10,404 (84 percent) gave oral consent and 7528 of these (61 percent) gave written informed consent for us to obtain their medical records. Among those who gave written consent, we received at least one record for each of 6712 participants (89 percent). We received 84 percent of the records of participants for whom we had consent forms. We received all records for which we had consent forms for 4612 persons (69 percent), and all but one record for 1547 persons (23 percent).

We included in the study all persons for whom we had at least one record (37 percent of the eligible sample). The participants reported having seen from 1 to 17 different providers (average, 2.6) during the previous two years.

Development of Quality Indicators

RAND's Quality Assessment (QA) Tools system contains 439 indicators of the quality of care for 30 medical conditions and preventive care. The indicators represent care for the leading causes of death, disability, and use of inpatient and outpatient services.14 Four nine-member, multispecialty expert panels chose the indicators, using the RAND–UCLA modified Delphi method.16 We classified indicators a priori according to the type (acute, chronic, or preventive) and function (screening, diagnosis, treatment, or follow-up) of care. The indicators are available at www.rand.org/health/mcglynn_appa.pdf and in the Supplementary Appendix, available with the full text of this article at www.nejm.org.

Data Collection

Using computer-assisted abstraction software, 20 trained registered nurses abstracted data from photocopies of participants' medical records of tests, treatments, use of inpatient and outpatient services, and other clinical events to determine whether the participant was eligible for care as listed in the QA Tools indicators and whether the indicated care had been received. The average interrater reliability was substantial at three levels: presence of a condition (κ=0.83), indicator eligibility (κ=0.76), and indicator scoring (κ=0.80).17 For 22 of the indicators, the survey data augmented scores based on medical records. Data on the respondents' age, sex, race or ethnic group, education, income, health status, and health insurance status were obtained from the CTS.

Statistical Analysis

We used SAS software (version 8.2) for all statistical analyses.18 Aggregate scores for overall quality, as well as for type and function of care, were calculated by dividing the total number of instances in which care was received by the total number of instances in which a participant was eligible for the recommended care in that category. The results were adjusted with sampling and nonresponse weights.

To estimate the independent effects of individual characteristics, we entered age, sex, race or ethnic group, education, annual household income, health status, and health insurance status into a logistic-regression model as mutually exclusive categorical variables. Covariates included the numbers of acute conditions, chronic conditions, outpatient visits, and hospitalizations during the study period. Odds ratios were transformed into adjusted proportions.19

We conducted a number of sensitivity analyses to test how robust our findings were to modeling assumptions, selection of indicators, and rates of nonresponse. We constructed models using indirect standardization to adjust for differences in the likelihood of receiving indicated care, as well as models that excluded variables for use of inpatient and outpatient services and models that controlled for the number of providers and the interaction between utilization and number of providers. None of these analyses changed the direction or significance of any reported results.

We also created an overall quality subscore from indicators supported by randomized, controlled trials. The main findings with regard to age, sex, race or ethnic group, income, and health insurance status persisted, although not all were statistically significant, perhaps because of power limitations. To investigate whether the selection of indicators might explain differences in quality according to race or ethnic group, we performed subgroup analyses on those indicators for which there were previously noted racial or ethnic disparities.4

Participants were more likely than nonparticipants to be white, older, female, wealthier, better educated, sicker, and more frequent users of health care services (see the Supplementary Appendix). We used two logistic regressions to model nonresponse. The first used data from the CTS to predict the rate of nonresponse to our survey; the second used survey data to predict failure to receive medical records. The predictor variables were sex, education, race or ethnic group, income, community of residence, self-reported health status, and utilization of inpatient and outpatient services. Because the adjusted and unadjusted results were similar, we present only the results adjusted for nonresponse.

Results

Table 1. Table 1. Characteristics of the 6712 Study Participants. Table 2. Table 2. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic.

Table 1 presents the characteristics of the study participants. Overall, participants received 54.9 percent of recommended care (Table 2). Women received a higher proportion of recommended care than men (56.6 percent vs. 52.3 percent, P<0.001). Quality-of-care scores declined with age (57.5 percent for those from 18 through 30 years of age vs. 52.1 percent for those 65 years of age or older, P<0.001). Participants with annual family incomes over $50,000 had quality-of-care scores that were 3.5 percentage points higher than those with incomes of less than $15,000 (P<0.001). The overall quality-of-care score was 3.5 percentage points higher for blacks than for whites (P<0.001) and 3.4 percentage points higher for Hispanics than for whites (P<0.001). The scores of Medicare beneficiaries were 3.2 percentage points higher than the scores of those without health insurance (P=0.03).

Table 3. Table 3. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Type of Care.

Table 3 presents the results according to the type of care. Women had higher scores than men for preventive care (57.8 percent vs. 50.1 percent, P<0.001) and chronic care (57.9 percent vs. 54.5 percent, P=0.01) but lower scores for acute care (51.9 percent vs. 58.4 percent, P<0.001). Adults under 31 years of age were significantly more likely to receive preventive services than those 31 through 64 years of age (difference, 3.8 percentage points; P<0.001) or those 65 years of age or older (difference, 8.8 percentage points; P<0.001). Participants 31 through 64 years of age received significantly better chronic care than those under 31 years of age (57.3 percent vs. 50.9 percent, P<0.001). Blacks had higher scores for chronic care than did whites (61.3 percent vs. 55.4 percent, P<0.001). Those with annual family incomes of at least $15,000 had significantly higher scores for preventive care (P<0.001) than those with lower incomes.

Table 4. Table 4. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Function of Care.

Women had higher scores than men for screening (56.7 percent vs. 42.9 percent, P<0.001) but lower scores for treatment (56.0 percent vs. 59.3 percent, P=0.001) (Table 4). Younger and wealthier participants also had higher scores for screening, but younger participants had lower scores for follow-up than older participants. Blacks had higher treatment scores than whites (64.0 percent vs. 56.3 percent, P<0.001), and Hispanics had higher screening scores than whites (55.9 percent vs. 51.6 percent, P=0.02). The principal advantages for Medicare beneficiaries were in diagnosis and treatment.

We conducted a number of sensitivity analyses to examine the associations between race or ethnic group and quality of care. First, when we confined our analysis to 33 indicators with known previous process-quality disparities favoring whites that were taken from the Institute of Medicine report Unequal Treatment, 4 the results for race or ethnic group were reversed (53.4 percent for blacks vs. 56.0 percent for whites), although the difference was not statistically significant. Second, we examined the influence of the rate of medical-record nonresponse (i.e., the absence of permission to release medical records). Scores for medical-record responders and nonresponders were not significantly different for five of the nine indicators that did not require review of medical records; for the remaining four, the medical-record responders' scores were higher by about 6 percentage points. Stratifying this analysis by race or ethnic group did not reveal a different pattern for blacks and whites. Finally, we calculated how low the quality-of-care score for black nonresponders would have to be for whites to have an advantage of 15 percentage points, a level of disparity commonly noted in the literature.4 If we made the response rates of blacks equivalent to those of whites, all new black respondents added to our analytic pool would have had to fail to receive all indicated care for which they were eligible to produce this level of disparity.

Discussion

We have previously reported that Americans receive about half of recommended health care and that there is remarkably little geographic variation in this rate.3,20 The present study demonstrates that the differences among population subgroups in the quality of health care, even when they are statistically significant, are small in relation to the gap between actual and optimal performance. These results underscore the profound and systemic nature of the quality-of-care problem. The variation in findings according to the type and function of care points to the complexity of the relationships between patients' characteristics and the quality of health care, making it difficult to draw conclusions about how well any group is treated on the basis of quality-of-care measures confined to any single area.

For example, we found that although women, overall, received better care than men, they had higher scores for preventive and chronic care and lower scores for acute care. One possible explanation for this difference is that women establish routine care-seeking behavior early in life by undergoing regular Pap-smear tests, receiving prenatal care, and managing well-child care in the family. The lower scores we observed among women for acute care and treatment-related care are consistent with the literature on disparities in health care between men and women.21

Overall, the proportion of recommended care that was received declined with age. This trend was particularly strong for preventive care services: the score for those under 31 years of age was nearly 10 percentage points higher than the score for those 65 years of age or older. Perhaps the trend occurs because as people age, they become eligible for more screening procedures that occur at various intervals and that may involve different physicians and, as a result, more complex patient-tracking systems are required. In contrast, the score for follow-up care was 22 percentage points higher for the oldest participants than for the youngest. This difference may reflect greater aggressiveness among both older patients and their physicians in pursuing problems once they have been identified.

We found that health insurance status was largely unrelated to the quality of care among those with at least minimal access to care. Although having insurance increases the ease of access to the health care system, it is not sufficient to ensure appropriate use of services or content of care. Indeed, within systems where access to care is more equitable, disparities in quality due to race or ethnic group or to other characteristics are often reduced or even reversed, but substantial gaps between observed and optimal quality remain.22,23 In the United Kingdom, with universal coverage, a study using our methods found that the overall proportion of recommended health care that was received was similar to what we have reported.24

Our finding that blacks had higher scores than whites for the quality of health care, even when other sociodemographic characteristics, health status, and use of inpatient and outpatient services were controlled for, is at odds with many other published studies. We considered a number of possible explanations for this result. First, we examined whether we were measuring different dimensions or indicators of quality than had previously been studied. When we confined our analysis to indicators used in previous studies that demonstrated racial or ethnic disparities, we observed a trend toward better care for whites than for blacks. Previous studies focused on invasive and expensive procedures, such as kidney transplantation and coronary-artery bypass graft surgery, rather than on the more routine care that dominates the QA Tools indicators. There is some evidence that the gap between whites and blacks is narrowing, even for these expensive procedures.25 Thus, what is measured affects the conclusions one might draw, but the overall differences between groups are small.

Next, we considered whether nonresponse bias might explain the findings. It is possible that experience with and attitudes toward the health care system — including communication problems and mistrust — might deter members of some racial or ethnic groups from obtaining care. This factor might have skewed the composition of our sample of patients who had to have made at least one visit to a health care provider in the previous two years.26-28 In contrast, differential rates of response to requests for medical records seem unlikely to explain the findings, because there was little difference according to race or ethnic group in the pattern of results for the small set of indicators that could be scored without referring to medical records. Overall, nonresponse bias could easily explain the small advantage observed for blacks. However, for nonresponse by blacks to have produced the magnitude of disparities commonly found in previous studies, nonrespondent blacks would need to have had implausibly low (near zero) quality scores.

To make our findings more concrete, we constructed profiles of two hypothetical patients. According to our study, a 50-year-old white female college graduate with private health insurance and a household income above $50,000 would receive 56.7 percent of recommended care. In comparison, a 50-year-old uninsured black man with less than a high-school education and an income under $15,000 would receive 51.4 percent of recommended care (P=0.02). Although the difference between these two hypothetical persons is significant, the gap between the care each of them receives and the standards of good practice is substantially larger than this difference and is probably much more clinically important.

We have not yet estimated the clinical significance of the deficits we observed, but Higashi et al.,29 who have developed a similar method of evaluating the quality of care delivered to vulnerable eldery persons, have established a link between mortality and scores on a comprehensive set of process measures. In that study, 18 percent of participants who received care better than the median (a group that had average scores of 62 percent) died within three years, as compared with 28 percent of participants whose quality scores were below the median (average score, 44 percent). We previously showed that the proportion of recommended care received was 19 percentage points higher for patients with controlled hypertension than for those with uncontrolled hypertension.30 The differences between the scores for any subgroup and the optimal quality of care far exceeded those thresholds, whereas the differences between subgroups did not.

Our study is one of the most comprehensive to date of the relationship between individual sociodemographic characteristics and multiple domains of the quality of care. Nonetheless, it has some limitations. We have discussed the potential for nonresponse bias above. In addition, the data obtained from medical records may differentially underrepresent quality in some circumstances as a result of incomplete or inadequate documentation by providers. A study comparing standardized patient reports and the medical record estimated that the medical record ranges from underrepresenting the quality of care by about 10 percent to overrepresenting the quality of care by about 6 percent; therefore, we do not think that poor record keeping adequately explains the observed widespread shortfalls.31 In addition, our inclusion criteria may have excluded the most vulnerable populations, such as those without telephones or those who had not seen any health care provider in the previous two years.

To make substantial improvements in the quality of health care available to all patients, we must focus on large-scale, system-wide changes. Our previous study of the quality of care delivered in the Veterans Affairs health system illustrates some of the potential for improvement. In that system, with one of the country's most mature electronic medical-record systems, decision-support tools at the point of care, automated order entry, routine measurement of and reporting on quality, and financial incentives for performance, we found that participants received 67 percent of recommended care,32 a considerably better rate than the 55 percent observed in the current study.

We have previously shown substantial deficits in the quality of care nationally and in metropolitan areas with very different market profiles.3,20 In this study, we have now shown that individual characteristics that often have a protective effect do not shield most people from deficits in the quality of care. As the Institute of Medicine has concluded,2 problems with the quality of care are indeed widespread and systemic and require a system-wide approach.

Funding and Disclosures

No potential conflict of interest relevant to this article was reported.

Author Affiliations

From RAND Health, Santa Monica, Calif. (S.M.A., J.K., J.L.A., C.M.S., S.M., E.A.M.); the Veterans Affairs Greater Los Angeles Health Care System and the Department of Medicine, David Geffen School of Medicine, UCLA — both in Los Angeles (S.M.A.); the Veterans Affairs Center for Practice Management and Outcomes Research, Veterans Affairs Ann Arbor Health Care System, and the Department of Internal Medicine, University of Michigan Medical School — both in Ann Arbor, Mich. (E.A.K.); and the Department of Medicine, Division of Cardiology, University of California, Irvine (S.M.).

Supplementary Material

References (32)

  1. 1. Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q 1998;76:517-563

  2. 2. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, D.C.: National Academy Press, 2001.

  3. 3. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003;348:2635-2645

  4. 4. Smedley BD, Stith AY, Nelson AR, eds. Unequal treatment: confronting racial and ethnic disparities in health care. Washington, D.C.: National Academy Press, 2003.

  5. 5. Ayanian JZ, Weissman JS, Chasan-Taber S, Epstein AM. Quality of care by race and gender for congestive heart failure and pneumonia. Med Care 1999;37:1260-1269

  6. 6. Kossovsky MP, Sarasin FP, Louis-Simonet M, et al. Age and quality of in-hospital care of patients with heart failure. Eur J Public Health 2004;14:123-127

  7. 7. Asch SM, Sloss EM, Hogan C, Brook RH, Kravitz RL. Measuring underuse of necessary care among elderly Medicare beneficiaries using inpatient and outpatient claims. JAMA 2000;284:2325-2333[Erratum, JAMA 2003;289:1782.]

  8. 8. Kahn KL, Pearson ML, Harrison ER, et al. Health care for black and poor hospitalized Medicare patients. JAMA 1994;271:1169-1174

  9. 9. Ferris TG, Blumenthal D, Woodruff PG, Clark S, Camargo CA. Insurance and quality of care for adults with acute asthma. J Gen Intern Med 2002;17:905-913

  10. 10. Smith JL, Rost KM, Nutting PA, Elliott CE. Resolving disparities in antidepressant treatment and quality-of-life outcomes between uninsured and insured primary care patients with depression. Med Care 2001;39:910-922

  11. 11. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable hospitalization by insurance status in Massachusetts and Maryland. JAMA 1992;268:2388-2394

  12. 12. Malin JL, Asch SM, Kerr EA, McGlynn EA. Evaluating the quality of cancer care: development of cancer quality indicators for a global quality assessment tool. Cancer 2000;88:701-707

  13. 13. Kerr EA, Asch SM, Hamilton EG, McGlynn EA, eds. Quality of care for cardiopulmonary conditions: a review of the literature and quality indicators. Santa Monica, Calif.: RAND, 2000. (Publication no. MR-1282-AHRQ.)

  14. 14. Kerr EA, Asch SM, Hamilton EG, McGlynn EA, eds. Quality of care for general medical conditions: a review of the literature and quality indicators. Santa Monica, Calif.: RAND, 2000. (Publication no. MR-1280-AHRQ.)

  15. 15. Kemper PD, Blumenthal D, Corrigan JM, et al. The design of the community tracking study: a longitudinal study of health system change and its effects on people. Inquiry 1996;33:195-206

  16. 16. Brook RH. The RAND/UCLA appropriateness method. In: McCormick KA, Moore SR, Siegel RA, eds. Clinical practice guideline development: methodology perspectives. Rockville, Md.: Agency for Health Care Policy and Research, November 1994:59-70. (AHCPR publication no. 95-0009.)

  17. 17. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174

  18. 18. SAS user's guide, version 8.2. Cary, N.C.: SAS Institute, 1999-2001.

  19. 19. Graubard BI, Korn EL. Predictive margins with survey data. Biometrics 1999;55:652-659

  20. 20. Kerr EA, McGlynn EA, Adams J, Keesey J, Asch SM. Profiling the quality of care in communities: results from the CQI study. Health Aff (Millwood) 2004;23:247-256

  21. 21. Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682

  22. 22. McBean AM, Huang Z, Virnig BA, Lurie N, Musgrave D. Racial variation in the control of diabetes among elderly Medicare managed care beneficiaries. Diabetes Care 2003;26:3250-3256

  23. 23. Gordon HS, Johnson ML, Ashton CM. Process of care in Hispanic, black, and white VA beneficiaries. Med Care 2002;40:824-833

  24. 24. Marshall MN, Roland MO, Campbell SM, et al. Measuring general practice: a demonstration project to develop and test a set of primary care clinical quality indicators. London: Nuffield Trust, 2003.

  25. 25. Escarce JJ, McGuire TG. Changes in racial differences in use of medical procedures and diagnostic tests among elderly persons: 1986-1997. Am J Public Health 2004;94:1795-1799

  26. 26. Thompson HS, Valdimarsdottir HB, Winkel G, Jandorf L, Redd W. The Group-Based Medical Mistrust Scale: psychometric properties and association with breast cancer screening. Prev Med 2004;38:209-218

  27. 27. Johnson RL, Roter D, Powe NR, Cooper LA. Patient race/ethnicity and quality of patient-physician communication during medical visits. Am J Public Health 2004;94:2084-2090

  28. 28. Collins TC, Clark JA, Petersen LA, Kressin NR. Racial differences in how patients perceive physician communication regarding cardiac testing. Med Care 2002;40:Suppl:I-27

  29. 29. Higashi T, Shekelle PG, Adams JL, et al. Quality of care is associated with survival in vulnerable older patients. Ann Intern Med 2005;143:274-281

  30. 30. Asch SM, Kerr EA, Lapuerta P, Law A, McGlynn EA. A new approach for measuring quality of care for women with hypertension. Arch Intern Med 2001;161:1329-1335

  31. 31. Luck J, Peabody JW. Using standardised patients to measure physicians' practice: validation study using audio recordings. BMJ 2002;325:679-679

  32. 32. Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med 2004;141:938-945

Citing Articles (298)

    Letters

    Figures/Media

    1. Table 1. Characteristics of the 6712 Study Participants.
      Table 1. Characteristics of the 6712 Study Participants.
    2. Table 2. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic.
      Table 2. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic.
    3. Table 3. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Type of Care.
      Table 3. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Type of Care.
    4. Table 4. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Function of Care.
      Table 4. Adjusted Percentage of Recommended Care Received by Participants, According to Characteristic and Function of Care.

      More Research