Randomized, Controlled Trials in Health Insurance Systems

The central activities of health insurance companies relate to the administration of benefit plans, including which services are covered, the total amount of coverage, and the level and method of patient cost sharing. Although almost all the research evaluating insurance services has relied on observational methods, variations in the design of health insurance would be the most straightforward means of testing interventions in an RCT conducted in a health insurance system. Two landmark trials, the RAND and the Oregon Health Insurance Experiments, have compared different levels of health insurance coverage, although they did not use claims data for participant identification and their primary outcomes, including health status, were measured with the use of more traditional data-collection methods.^10,11

Table 1. Table 1. Characteristics of Selected Randomized, Controlled Trials Conducted in Health Insurance Systems.

An RCT of how best to provide health insurance could also be conducted exclusively with routinely collected insurance data. For example, the Post-Myocardial Infarction Free Rx Event and Economic Evaluation (MI FREEE) trial tested the effect of providing full prescription coverage on rates of vascular events and revascularization and used claims data to identify potentially eligible participants and to evaluate all the outcomes of the trial (Table 1).¹² Claims data have also been used to conduct RCTs of population-based care-management programs, which health insurance companies use to manage financial and clinical risk. For example, Wennberg et al. used claims data to identify participants and to evaluate the effect on health spending of two telephone-based care-management strategies in an RCT involving more than 170,000 beneficiaries of two regional health plans.¹³

Individual clinical interventions, such as diagnostics and therapeutics, could also be evaluated with the use of an insurance system–based RCT, although no such studies have been published. By altering what services are covered by insurance or what level of out-of-pocket costs patients face to receive those services, different groups of patients could effectively undergo randomization to alternative therapies for a given health condition. For example, a group of plan sponsors could be assigned to make only one of the many oral medications that are currently approved for the initial treatment of diabetes or hypertension freely available on their formularies (i.e., without patient cost sharing) for patients without contraindications, with the other drugs remaining subject to usual levels of coverage. If the nature of the benefits across the plans is otherwise equivalent and if patients in different plans are equally cost-sensitive, then the rate of use of the “preferred” therapy across the different plans should be similar. The fact that not all patients will use the preferred first-line agent of the plan could be handled with the use of an instrumental-variable approach.¹⁷ Because providers interact with insurers through direct reimbursement for services and through performance-based contracting, it is also conceivable that the structure of these arrangements could be randomly altered to evaluate the use of diagnostics or therapeutics that are less subject to patient preference.

Studies of clinical interventions that are conducted in a health insurance system would be most feasible for treatments that have already been approved for use and may be considered acceptable by some patients because, by design, access to the full range of alternative treatments is preserved. Furthermore, the intervention would be evaluated in a manner in which it might ultimately be deployed in the future, thereby enhancing the generalizability of the study results.

In typical RCTs, participants are identified through the screening of medical records; active surveillance in hospitals, emergency departments, and physician offices; or, in some cases, advertising. Beneficiaries of a health insurer are an alternative group of potentially eligible trial participants. Administrative data can allow for the identification of inclusion and exclusion criteria based on sociodemographic characteristics (e.g., age and sex) that are contained in enrollment or “eligibility” files, clinical criteria that use diagnostic or procedure codes contained in “medical” claims from health care providers, or information about medication filling from prescription-drug claims (Table 1). For example, Milkman et al. evaluated strategies to encourage employees of a large company to receive their annual influenza vaccination.¹⁴ The trial focused on persons with health conditions that put them at increased risk for influenza-related adverse events, who were identified with the use of diagnostic codes that were applied to medical-claims data from the preferred provider organization used by the company.

The identification of potentially eligible study participants with claims data is most straightforward when done retrospectively — that is, when inclusion and exclusion criteria are applied to data that have been received by insurers before the start of the study. Prospective identification of participants with the use of claims is also possible but requires additional resources to query claims-processing data repeatedly. Furthermore, it may take several months from the time a service is rendered for claims to be submitted by providers, processed, and made available in a data environment that can be queried for research. Consequently, it may be challenging to prospectively identify participants in a timely fashion for studies that, for example, seek to prevent short-term adverse events.¹⁸ In the MI FREEE trial, for example, patients who were recently discharged from the hospital after myocardial infarction were identified every 2 weeks with the use of discharge diagnoses on claims submitted by hospitals to a large national insurer. As a result, the mean time from hospital discharge to randomization in this trial was 49 days rather than the many months that would have been needed if a retrospective approach had been used.¹²

After the identification of potentially eligible trial participants with health insurance data, they could be contacted and asked to undergo further screening through methods that are used in more traditional studies. Alternatively, the linkage of administrative claims to other data may help refine participant identification. Biometric information and information about risk factors (e.g., body-mass index and smoking status) that are collected by employers are sometimes provided to insurers to facilitate quality-improvement activities. Similarly, laboratory values are transmitted to insurers by providers as part of value-based contracting or are sometimes purchased by health insurers directly from laboratory companies.¹⁹ For example, the Enhancing Outcomes through Goal Assessment and Generating Engagement in Diabetes Mellitus (ENGAGE-DM) trial is evaluating the effect of a pharmacist intervention delivered to beneficiaries of a large regional health insurer who have diabetes (according to diagnoses in medical claims), are being treated with an oral hypoglycemic agent (according to drug codes in pharmacy claims), and have glycated hemoglobin levels of 8% or more (according to laboratory information).¹⁵

The assessment of outcomes in traditional RCTs is typically based on information that is prospectively collected from hospitals, at dedicated clinic visits, or through telephone outreach and that may undergo adjudication with the use of source data such as medical records and radiographic results. Although these data often provide rich clinical detail and are the only means of obtaining patient-reported outcomes such as symptoms, quality of life, and satisfaction, their collection is resource-intensive.

Data from health insurance claims allow for the evaluation of some study outcomes without the need for study-specific activities. For example, the Ontario Printed Educational Message (OPEM) trial was a cluster-randomized trial involving more than 5000 physicians caring for approximately 180,000 patients with diabetes. This trial assessed the effect of printed educational materials on the use of evidence-based care, including the receipt of diabetic retinopathy screening, and measured outcomes with the use of routinely collected data from health insurance claims.^16,20,21 Similarly, the Randomized Evaluation to Measure Improvements in Non-adherence from Low-Cost Devices (REMIND) trial used data from pharmacy claims to evaluate the effect on medication adherence of providing reminder and habit-formation devices for patients with common chronic conditions.²²

Most traditional trials that measure economic outcomes do so by applying unit costs obtained from other sources, such as hospital charges, to collect information on resource utilization prospectively, generally on the basis of patient self-report.^23,24 This process can be extremely labor-intensive, and patient self-report may not be accurate for all resource outcomes, such as physician office visits, especially as more time has elapsed between when an outcome occurred and when it is captured.^25,26 The use of administrative claims data substantially improves the efficiency of this process and may increase its accuracy as well.

Claims data may also be linked to enrollment data and data from case-report forms for participants in conventional RCTs to extend follow-up. For example, the Dialysis Clinical Outcomes Revisited (DCOR) trial compared sevelamer with calcium-based phosphate binders among patients receiving hemodialysis.²⁷ The primary study evaluated mortality and hospitalization with the use of prospectively collected data, but a preplanned companion study linked 2101 of the 2103 DCOR trial participants to claims maintained by the U.S. Renal Data System.²⁸

Although events that occur outside of health care systems cannot be evaluated with the use of claims,¹² the National Death Index can be linked to claims data through a person’s name, date of birth, or other identifiable information that is contained in enrollment files and can be used to identify out-of-hospital death as well as to assess specific causes of death. Unfortunately, these data typically do not become available until a year after the end of a particular calendar year.²⁹ The linkage of claims to laboratory data or data from electronic health records may also facilitate the evaluation of biometric outcomes, such as glycated hemoglobin levels, although few examples of the use of these data for prospective trials have been published.³⁰

Because health insurance claims are generated for administrative purposes and not to support research, claims-based algorithms may not always be accurate or complete. In the case of identifying patients for inclusion in an RCT, exposure misclassification could mean that patients who may not benefit from an intervention are offered it unintentionally, which will generally bias the effectiveness of beneficial interventions to the null. Accordingly, there is a general preference for highly specific claims-based criteria, such as the codes for acute myocardial infarction³¹ or stroke³² that were used in the MI FREEE trial. Specificity usually occurs at the expense of sensitivity; therefore, such criteria may result in the exclusion of potentially eligible participants.

Inaccuracies in the diagnostic information that is contained in insurance claims may also result in outcome misclassification. Although such misclassification is likely to introduce bias only if it differentially affects outcomes in the treatment groups, it could contribute to a loss of statistical efficiency.¹⁸ A validation study by Hlatky et al. compared the results obtained from Medicare claims data with independently adjudicated outcomes for women who participated in the Women’s Health Initiative (WHI) trial of hormone therapy and were 65 years of age or older.³³ They found good-to-excellent agreement between events of myocardial infarction and revascularization that were defined on the basis of prospectively collected case-report forms and those ascertained with insurance claims, with both sources identifying outcomes that the other had missed. The level of observed agreement between the WHI-adjudicated events and claims was also similar to the agreement between events that were adjudicated at local sites and those that were adjudicated centrally. Furthermore, the magnitude of the elevated risk of myocardial infarction from hormone therapy that was estimated on the basis of claims was very similar to that based on adjudicated outcome (hazard ratio, 1.29 vs. 1.31), with no significant differences between these two sources in 1000 bootstrapped replications.³³

Nevertheless, for diagnoses that are less accurately coded or for which the accuracy of the claims-based codes is unknown, such as eye examinations in the OPEM trial,²⁰ there is the potential for outcome misclassification. This may be particularly true for cause-specific events, such as rehospitalizations for a specific postoperative complication. Because of these concerns, it is also very unlikely that claims data alone would be sufficient for the regulatory approval of new therapies, although, similar to the approach taken in some observational studies,³⁴ it may be possible to use claims data to determine which medical records should be retrieved for adjudication, thereby reducing the burden of this process.

The cost information that is contained in adjudicated claims data comes from financial transactions among payers, providers, and patients and is therefore assumed to accurately reflect the cost of services covered by the particular insurer. Services that are paid for entirely out of pocket or that are covered by a second insurer will not be captured. Missing data may be even more common and problematic for laboratory values,¹⁹ which are subject to the idiosyncrasies of provider practice patterns or the frequency of patient follow-up. The completeness of these data almost certainly varies according to test, although missing data would bias a study only if the availability of data differed according to treatment assignment. Because the laboratory values available to insurers are those that are ordered by clinicians for the purposes of clinical care, the values themselves are assumed to be accurate.

The use of health insurance data for outcomes assessment is most practical in environments in which all the trial participants receive health benefits from a single insurer.³⁵ However, many Americans who are enrolled in employer-sponsored health insurance plans change insurers each year, either because they switch employers or because their employer switches to a different insurer. As a result, working with a single insurer may make it impossible to track patients over long periods of time and may undermine statistical power.¹⁸ Although multiple insurers may agree to provide claims for a single study, such as in the study of care management by Wennberg et al.,¹³ such agreements are often infeasible. The use of an “all payer” claims database in which insurers in a given state have agreed to pool claims³⁶ could facilitate access to claims from multiple insurers, but these data have not been used for prospective studies and in some cases are completely deidentified, thus making it impossible to link treatment assignment to individual participants for outcome evaluation. Because Medicare beneficiaries and military veterans generally retain coverage for their lifetimes, claims from these sources might be an alternative, although these populations may not be widely generalizable. Furthermore, transitions in coverage occur even in the context of Medicare when, for example, patients switch from fee-for-service plans to managed care plans, for which administrative claims are not always as easily accessible.

As with typical RCTs, investigators require oversight by institutional review boards (IRBs) to conduct trials in the context of health insurance systems. Because trial-specific activities are being conducted primarily for research (rather than quality-improvement) purposes, insurers may be required to obtain their own IRB approval in addition to that which is required for the academic investigators.³⁷ In the planning phase of a study, insurers do not necessarily need a waiver of patient authorization from an IRB on the basis of the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule. Rather, claims data may be used to identify prospective research participants, to assess potential sample sizes, or to otherwise prepare for research as long as the researchers attest in writing that the identifiable information is sought solely, and is necessary, to prepare for the proposed research and will not be removed from the premises of the insurer.³⁸ However, an IRB waiver of informed consent for the review of identifiable patient information for preparatory purposes may be required, especially for research that is federally funded and governed by the Common Rule.³⁹

From a practical perspective, accessing patient names, contact information, and other protected health information from health insurance records requires direct collaboration between investigators and insurers. In turn, because insurers provide health insurance coverage on behalf of “plan sponsors” (i.e., employers, unions, governments, or associations that sponsor a particular benefits package), the largest of which are ‘‘self-insured’’ and for whom insurers primarily process claims rather than insulating them against risk, the use of insurance data to identify and approach potential study participants may require permission from the plan sponsor.

As with the use of administrative data for participant identification, the use of such data for outcome evaluation requires IRB oversight and is governed by the Common Rule and the HIPAA Privacy Rule. For cases in which it is not feasible to obtain patient authorization to use administrative claims data, investigators may request a waiver of this requirement from an IRB if the use or disclosure of the participants’ identifiable information involves no more than minimal risk to their privacy and if the research could not practicably be done without the waiver or without access to identifiable information.³⁸ For example, in addition to meeting these criteria, Hlatky et al. were able to use Medicare claims to assess outcomes for WHI participants after receiving an IRB waiver for the requirement for additional patient authorization, because participants had already provided informed consent to participate in the parent trial.³³

Similar to the ongoing dialogue about what level of patient authorization is required in pragmatic RCTs of interventions that are delivered in clinical settings,^40-42 participant-level consent may not always be required for interventions that are conducted in health insurance systems. The requirements in this regard will no doubt depend on the potential risks associated with the interventions being studied. Accordingly, IRB waivers of the requirements for informed consent have been common for trials conducted in this context (Table 1). Before insurance-based trials are used to evaluate diagnostics or therapeutics (e.g., using differential cost sharing or insurance coverage as a randomization method), issues of consent and communication with a patient’s care team will also need to be clarified.

In the quest to create a learning health care system that makes decisions based on the best available evidence, strategies to increase the number, efficiency, and generalizability of RCTs have been widely advocated.⁴³ RCTs that are conducted in the context of the health insurance system offer a new way to achieve this goal. Although still rare, examples of trials that use health insurance data to identify potential study participants and to evaluate outcomes have been successfully conducted. In the future, trials that leverage the ways in which patients and providers use insurance may also expand the possible ways in which randomization occurs. In addition, although trials in this setting will not be possible for all patient populations, interventions, or outcomes, their conduct should be facilitated by increasing linkages between administrative claims data and other nonadministrative data sources and by the increasing experience of both academics and insurers with the practical, ethical, and methodologic challenges of these designs.