Implementation of Medical Homes in Federally Qualified Health Centers

Intervention

To support the efforts of achieving level 3 recognition, demonstration sites received payments of care management fees of $6 per Medicare beneficiary per month, technical assistance for implementing practice change, semiannual feedback on progress toward recognition, and quarterly reports on their patients’ utilization of services, quality of care, and expenditures. A detailed description of the methods that were used in this study is provided in the Supplementary Appendix.

Site Selection

Federally qualified health centers that provided medical services to at least 200 Medicare beneficiaries annually were eligible for the demonstration. Sites that served special populations only (e.g., homeless persons) were ineligible. Of the 3426 sites that billed Medicare in 2010, a total of 1558 met the eligibility criteria, and 1014 applied. The CMS selected 503 sites to ensure diversity with respect to region, degree of urbanization, use of electronic health records, medical-home readiness, and receipt of medical-home–related payments.

We identified 827 potential comparison sites from among sites that applied for the demonstration but were not selected, met the eligibility criteria but did not apply, or did not meet the requirement with respect to beneficiary volume. Using a propensity-score model that predicted participation in the demonstration on the basis of site-level characteristics, we compared the distributions of propensity scores at demonstration sites and potential comparison sites. After finding that the propensity scores for all 827 potential comparison sites fell within the range of scores at demonstration sites, we included all 827 sites in the comparison group.

Beneficiary Attribution

Medicare fee-for-service beneficiaries were included in the evaluation if they were continuously enrolled in Parts A and B and did not have end-stage renal disease during the year before the demonstration. Beneficiaries were attributed to either a demonstration site or a comparison site if the site provided a plurality of primary care visits during the year before the demonstration. We repeated the attribution process quarterly, and beneficiaries who were first attributed after the demonstration began contributed outcomes to the analysis starting in the first full year after attribution. Beneficiary outcomes were associated with the site to which the beneficiary was first attributed.

Hypotheses

We hypothesized that technical and financial assistance would help demonstration sites achieve level 3 recognition (according to the 2011 standards of the National Committee for Quality Assurance) faster than comparison sites. Medical-home recognition requires practices to implement processes to improve access, continuity, and coordination. We hypothesized that a higher rate of recognition among demonstration sites would translate into larger improvements in the utilization of primary care and in beneficiaries’ experiences (with respect to timeliness of and access to services, communication with providers, and ratings of providers) relative to comparison sites, as well as larger reductions in the utilization of acute care and in expenditures.

Data and Outcomes

The National Committee for Quality Assurance and the Health Resources and Services Administration provided data on the status of medical-home recognition. We used Medicare claims to measure the utilization of services, quality of care, and Medicare expenditures. Measures of utilization included the number of visits to federally qualified health centers, the number of primary care visits to practices that are not federally qualified health centers (including visits to physicians, nurse practitioners, or physician assistants with primary care specialties who practice in rural health clinics or offices), specialist visits (including visits to physicians, nurse practitioners, or physician assistants with specialties other than primary care who practice in federally qualified health centers, rural health clinics, or offices), emergency department visits (including observation stays), all-cause admissions and admissions for ambulatory-care–sensitive conditions (which are used to measure the utilization of inpatient services that might be averted with improved care in ambulatory settings),²⁸ and 30-day unplanned hospital-wide readmissions.²⁹

We used five process-quality measures that are part of the Healthcare Effectiveness Data and Information Set, including a four-measure composite for patients with diabetes (annual testing of glycated hemoglobin and low-density lipoprotein cholesterol levels, eye examinations, and nephropathy testing) and an annual lipid test for patients with ischemic vascular disease. Measures of Medicare expenditures included costs associated with inpatient services, with Medicare Part B (including physician and supplier claims), and with all Medicare services (including inpatient, Part B, outpatient, skilled nursing, home health, durable medical equipment, and hospice).

We conducted a longitudinal survey of beneficiaries in demonstration sites and comparison sites in two waves that were separated by 19 months. The survey consisted of items from the Clinician and Group Consumer Assessment of Healthcare Providers and Systems Survey,³⁰ which includes items to assess the experiences of patients and the Patient-Centered Medical Home Item Set,³¹ along with items to assess health outcomes and patient-reported quality of care. Approximately 16,000 beneficiaries completed the first-wave survey (which was fielded 19 to 24 months after the start of the demonstration and had a 40.7% response rate), and 10,047 responded to both first-wave and second-wave surveys (which were fielded during months 35 to 39 and had a 65.6% response rate among first-wave respondents). We oversampled beneficiaries under the age of 65 years and those with dual eligibility for Medicare and Medicaid, higher rates of comorbidities, and higher probabilities of a preference for speaking Spanish.²⁵

Qualitative data collection included conducting semistructured interviews with leaders of 20 demonstration sites and 10 comparison sites and with leaders of six state Primary Care Associations (advocacy organizations for federally qualified health centers), which oversaw the technical assistance. We also ran focus groups with practice coaches employed by state Primary Care Associations and conducted full-day visits at five demonstration sites.

Study Oversight

The human-subjects committee at RAND approved the study, including the use of informed-consent procedures for the beneficiary survey and other primary data-collection activities. Informed consent was waived for claims analyses. The CMS program team designed and oversaw the execution of the demonstration.

Statistical Analysis

We used generalized estimating equations to implement a difference-in-differences approach to data analysis. The missing-completely-at-random assumption was supported by analyses that showed equivalent rates of overall attrition and of reasons for attrition among beneficiaries at demonstration sites and comparison sites. We estimated the effects of the demonstration on the utilization of services and Medicare expenditures using two-part models, in which the first part used logistic regression and the second part used negative binomial models for utilization of services and linear models for expenditures. We used logistic regression to model hospital readmissions and process measures.

Propensity-score weighting was used to balance demonstration sites and comparison sites with respect to observable characteristics that were measured during the year preceding each beneficiary’s first attribution. We adjusted standard errors for clustering of beneficiary responses over time using the Huber–White sandwich estimator.³² All estimates are reported on their natural scales with the use of Puhani’s estimator for nonlinear difference-in-differences models.³³ Difference-in-differences estimates were generated on a year-to-year basis, in analyses that compared differences in trends between groups from the year preceding each beneficiary’s first attribution through the end of each demonstration year, and cumulatively, in analyses that pooled yearly outcomes over multiple years.

Survey analyses used logistic regression for binary items and linear regression for all scale scores. Each analysis incorporated propensity-score weights, sampling weights, nonresponse weights, and site-level clustering with the use of Huber–White adjusted standard errors.

Beneficiaries and Sites

Table 1. Table 1. Characteristics of Medicare Beneficiaries and of Sites in Demonstration.

Nearly half of the 730,353 Medicare beneficiaries who were included in the evaluation were dually eligible for Medicare and Medicaid; just under one third were either nonwhite or Hispanic (Table 1). The majority of beneficiaries at both demonstration sites and comparison sites were attributed to sites that received supplemental funding from the Health Resources and Services Administration during 2011 to support medical-home implementation or were early participants in the ongoing Patient-Centered Medical Home Recognition Program, which covers the cost of applying for recognition.

Table 2. Table 2. Medical-Home Recognition by the End of the Demonstration, According to Level of Recognition.

No sites had achieved level 3 recognition (on the basis of the 2011 standards) before the demonstration began. By the end of the demonstration, 70% of the demonstration sites and 11% of the comparison sites had achieved level 3 recognition (Table 2). More than half of the demonstration sites that achieved such recognition did so during the final 3 months of the demonstration.

Primary Care Measures

Table 3. Table 3. Cumulative Effects of the Demonstration on Utilization, Process Measures, and Medicare Expenditures.

Demonstration sites outperformed comparison sites on several primary care measures. In cumulative difference-in-differences analyses, the beneficiaries at both demonstration sites and comparison sites had decreased rates of visits to federally qualified health centers over time (Table 3). However, the reductions were relatively smaller in demonstration sites, which led to a net increase of 82.5 visits per 1000 beneficiaries per year in demonstration sites (P<0.001). Similarly, reductions in rates of eye examinations and nephropathy testing among patients with diabetes in comparison sites led to a relative increase of 1.3 percentage points in the rate of eye examinations and of 1.6 percentage points in the rate of nephropathy testing in demonstration sites, where the rates actually stayed the same (P<0.001). On a composite measure of four diabetes tests, demonstration sites improved by 0.8 percentage points more than comparison sites, which did not have significant changes over time (P=0.01).

Emergency Department Visits, Admissions, and Expenditures

Despite a relative increase in the number of primary care visits, beneficiaries at demonstration sites had increased rates of visits to emergency departments over time, whereas the rates in comparison sites remained unchanged, which resulted in a net increase of 30.3 visits per 1000 beneficiaries per year at demonstration sites (P<0.001). The utilization of inpatient services by beneficiaries increased in the two groups, but the increase was larger in demonstration sites (5.7 additional admissions per 1000 beneficiaries per year, P=0.02). We found no significant between-group differences in trends over time in Medicare expenditures except for Part B expenditures, which increased more in demonstration sites than in comparison sites ($37 more per beneficiary per year, P=0.02).

Beneficiary-Reported Outcomes

Table 4. Table 4. Effect of the Demonstration on Patient-Reported Experiences, Evidence-Based Care, and Health Status.

The demonstration was associated with few significant effects on beneficiary-reported outcomes (Table 4). Beneficiaries in demonstration sites reported having larger relative improvements in receiving answers to medical questions on the same day than did those at comparison sites (P=0.01), as well as receiving appointments as soon as they were needed, although the latter comparison was not significant (P=0.07). Both results were driven entirely by decreases among comparison sites. There were no significant between-group differences with respect to other scale scores for beneficiary experiences, evidence-based care, or health status.

Provider Experiences

During interviews, leaders at demonstration sites reported that although they valued the monthly payments of $6 per patient in care management fees, they thought that the amount was low relative to the investments required for medical-home transformation. Despite perceptions that technical-assistance components were not well coordinated during the first half of the demonstration, when the Primary Care Associations were developing technical-assistance processes, respondents at demonstration sites considered that technical assistance was valuable in charting a course of change, managing transformation efforts, and supporting the educational process for transformation. Nevertheless, technical-assistance providers and leaders at federally qualified health centers noted that the 3-year requirement to achieve level 3 recognition increased stress.

The Federally Qualified Health Center Advanced Primary Care Practice Demonstration was successful in helping federally qualified health centers achieve medical-home recognition, with 70% of demonstration sites achieving level 3 recognition within 3 years. During the same period, visits to federally qualified health centers decreased in both demonstration sites and comparison sites. However, the reductions were smaller among the demonstration sites, which may reflect better access to primary care than in comparison sites. Similar trends of worsening performance in comparison sites explained the higher relative performance in demonstration sites with respect to patients’ reports of access to care and in some process measures of quality of care for patients with diabetes. The demonstration sites had relatively larger increases in visits to emergency departments, inpatient admissions, and Part B expenditures. Despite large differences between the two groups of sites in rates of level 3 medical-home recognition, these differences did not translate into reductions in the utilization of acute care or in Medicare expenditures or into other improvements in beneficiary-reported outcomes during the 3-year period.

Several factors may explain the lack of improvements at demonstration sites across a wide array of quality measures and reductions in utilization and expenditures. First, the two groups of sites appear to have been exposed to similar levels of medical-home–related funding and technical-assistance opportunities that were not related to the demonstration, a factor that may have limited the incremental effect of the assistance of the demonstration on patient outcomes. Interviews with site leaders confirmed that both demonstration sites and comparison sites had access to numerous other medical-home–related funding sources and technical-assistance opportunities, including assistance from state Primary Care Associations, the National Association of Community Health Centers, and the National Committee for Quality Assurance. Moreover, all federally qualified health centers were motivated to achieve recognition to qualify for participation in Medicaid Health Home initiatives in many states³⁴ and for medical-home–related supplemental grant funding from the Health Resources and Services Administration.³⁵ Ultimately, 40% of comparison sites achieved some form of medical-home recognition, which suggests that assistance that was not related to the demonstration was beneficial.

Second, the care management fees that were paid to demonstration sites were both small in magnitude and paid in proportion to the size of the Medicare beneficiary population at each site. Demonstration-site leaders reported that the payments of $6 per beneficiary per month were inadequate to support the staff and other infrastructure investments that were needed to implement practice change. In addition, these payments might not have been large enough either to maintain the focus of the demonstration sites on their Medicare beneficiaries (who make up only 12% of the patients at federally qualified health centers nationally³⁶) or to allow demonstration sites to develop the more advanced medical-home capabilities that might be needed to reduce Medicare expenditures. Multipayer and shared-savings designs may help to address these issues.

Third, most demonstration sites required the entire 3-year period to achieve level 3 medical-home recognition, with a majority doing so in the final quarter of the demonstration. The full effect of medical-home–related changes on utilization, expenditures, and beneficiary experiences might be observed only with the use of an extended measurement period.

Although the reasons for increased rates of emergency department visits in demonstration sites remain unclear, several site leaders mentioned guiding their patients to seek care at emergency departments when their clinics were unable to provide timely technical or specialty services.³⁷ Thus, these increases may reflect the increased commitment of staff members at demonstration sites to encourage care-seeking behavior while leveraging their improved care-coordination systems with emergency departments. These same factors could also account for higher rates of inpatient admissions and the lack of a reduction in Medicare expenditures, especially for patients at federally qualified health centers, who often report an inconsistent history of access to medical care and a pattern of presenting late in the course of an illness.

In the absence of a randomized design, the strength of our findings rests on the validity of our difference-in-differences analyses. We examined these assumptions by assessing trends in outcomes before the demonstration, which revealed no major differences. Nevertheless, unmeasured factors such as effective clinic leadership, staff turnover, or readiness to change could have been correlated with participation in the demonstration and could have biased estimates of the effect of the demonstration. Other limitations include the use of beneficiary-level clustering with robust standard errors in claims analyses, an approach that does not account for a small amount of residual site-level clustering. The timing of the first wave of the beneficiary survey and the short duration between waves may also have resulted in an underestimation of the effects on beneficiary experiences.

The fact that patients in demonstration sites had better access to and utilization of primary care services than did those in comparison sites provides some empirical evidence to support the role of medical-home transformation among federally qualified health centers. Despite these findings, demonstration sites did not achieve significant relative reductions in key utilization and expenditure measures during a 3-year period, and the demonstration was not expanded. Any future tests of medical-home interventions in federally qualified health centers should build on these results by considering alternative designs, with attention to the magnitude of financial assistance and the duration of the evaluation, to better understand how to facilitate practice transformation and ensure that these changes translate into improved outcomes for vulnerable Medicare beneficiaries.