The Effectiveness of the Varicella Vaccine in Clinical Practice

Eligibility Requirements

Subjects were healthy children between 13 months and 16 years of age. Children for whom the vaccine is not routinely recommended — those who were immunocompromised because of an underlying illness (e.g., leukemia) or medications (e.g., prednisone) — were excluded. Children with a history of chickenpox were excluded, since varicella vaccine is not recommended for such children.

Children who had received the vaccine within the preceding four weeks were excluded from the study, since it takes two to four weeks for antibodies to develop and about 7 percent of vaccinated persons have a vaccine-associated rash (characterized by a mild exanthem of 6 to 10 lesions that lasts for two to three days) in the first month after vaccination.² We were interested in assessing the effectiveness of the vaccine against chickenpox and did not wish to include as children with chickenpox children who merely had minor side effects of the vaccine. Similarly, if chickenpox had been incubating at the time of vaccination, disease would have developed within four weeks.

The study was approved by the Human Investigations Committee at Yale University School of Medicine.

Identification, Evaluation, and Classification of Potential Cases

The case group consisted of children in whom chickenpox was suspected who received medical care at the 15 participating pediatric practices in greater New Haven, Connecticut. Potential cases were identified by means of active surveillance. Investigators were notified of all potential cases, regardless of severity, including many cases in children whose illness was managed over the telephone without being evaluated in person by a clinician.

Table 1. Table 1. Scale Used to Assess the Severity of Illness.

The parents of eligible children were invited to participate in the study, and written informed consent was obtained from a parent, as well as oral consent from the child, when appropriate. A research assistant (who was unaware of the vaccination status of the child) visited the home of each patient with possible chickenpox (ideally on day 3 of the illness, but as late as day 5 when necessary). A parent was interviewed, and the severity of the illness was assessed according to a modified version of a scale used in earlier clinical trials (Table 1).⁷

In addition, a lesion was unroofed with use of a nonheparinized capillary tube in which vesicular fluid was collected to test for the presence of varicella–zoster virus by the polymerase chain reaction (PCR). The base of the lesion was rubbed with a cotton-tipped swab, which was then rubbed on a slide for testing with a direct fluorescent-antibody assay. Material from the swab was analyzed by PCR if vesicular fluid was not available.

A PCR assay specific for varicella–zoster virus was used to detect the virus and to differentiate the strain contained in the vaccine from wild-type strains.¹³ In addition, to assess the adequacy of the specimen (since varicella–zoster virus is found in association with cells), all samples that were negative for varicella–zoster virus were tested for a 268-bp region of the β-globin gene that is present in all human cells; a positive result indicated the presence of cellular material in the specimen.¹³

The investigators who performed the PCR and direct fluorescent-antibody tests and interpreted the results were unaware of each child's vaccination status. The PCR assay was considered positive if the specimen was positive for varicella–zoster virus DNA and all negative controls in the batch were negative. It was considered negative if the specimen was negative for varicella–zoster virus DNA, all positive controls in the batch were positive, and the specimen was positive for β-globin (indicating the presence of amplifiable DNA). The result was considered indeterminate if the specimen was negative for both varicella–zoster virus DNA and β-globin. The results of the direct fluorescent-antibody assay were interpreted as recommended by the manufacturer (Merifluor varicella–zoster virus, Meridian Diagnostics, Cincinnati): specimens with two or more epithelial cells in the smear that exhibited typical apple-green fluorescence were considered positive for varicella–zoster virus. Specimens that were negative but that had less than one cell per high-power (×400) field were considered to be inadequate specimens. Specimens with an adequate number of cells but with fewer than two cells that exhibited immunofluorescence were considered negative. Potential cases of chickenpox were classified on the basis of the PCR results.

Selection of Controls

For each child with a potential case of chickenpox, two controls, matched according to date of birth (within one month) and pediatric practice, were selected. A list of potential controls was generated from the computerized data base of the practice; it consisted of all patients in the practice born between 30 days before and 30 days after the birth of the child with the potential case of chickenpox. A table of random numbers was used to select the order in which potential controls and their parents were contacted, screened for eligibility, and if eligible, invited to enroll in the study.

Collection of Data and Ascertainment of Vaccinations

The medical records of all the children (from all sources of care) were reviewed to obtain information about all previous immunizations. Children for whom there was written documentation that they had received varicella vaccine four weeks or more before the “focal time” — the date of onset of chickenpox or, for the controls, the date of onset in the matched children with chickenpox — were classified as vaccinated. As per current recommendations, children with potential cases of chickenpox and their matched controls who were 13 years of age or older were considered to have been vaccinated if they had received two doses of vaccine at least four weeks before the focal time.

Statistical Analysis

Matched odds ratios for vaccination, with both their associated P values (assessed with the Mantel–Haenszel chi-square test for matched data) and 95 percent confidence intervals, were calculated with the use of True Epistat and SPSS statistical software.^14-16 The effectiveness of the vaccine was calculated as 1 minus the matched odds ratio. These estimates were adjusted for the effects of possible confounders with the use of conditional logistic regression.¹⁷ The statistical significance of differences among unmatched dichotomous and continuous variables was determined by the Pearson chi-square test and Student's t-test, respectively.¹⁵ All P values are two-sided.

To assess whether bias in the selection of controls might have affected the results, we compared the proportion of children with chickenpox who had received the measles, mumps, and rubella (MMR) vaccine with the proportion of controls who had received it. Since the MMR vaccine should have been administered at about the same age as the varicella vaccine and the MMR vaccine should have had no effect on the risk of chickenpox, we expected that there would be no significant difference between the proportion of children with chickenpox and the proportion of controls who had received the MMR vaccine. If there were a significant difference, it would suggest that there had been selection bias (e.g., bias due to differential use of medical care).

We also assessed the effectiveness of the vaccine among the children with potential cases of chickenpox whose PCR test results were either negative for varicella–zoster virus or indeterminate, since the vaccine should not be effective for these patients. To provide additional information to support the accuracy of the classification of potential cases according to the results of the PCR test, parents were called at least 1 month after enrollment and asked whether they knew of anyone in whom chickenpox had developed between 10 and 21 days after having had close contact with the child with a potential case of chickenpox.

Between March 1997 and November 2000, 461 children with suspected cases of chickenpox were identified. Of these, 118 were excluded: a sample from a skin lesion could not be obtained from 10 children; 13 children were identified more than five days after the onset of the rash; 19 had been vaccinated less than four weeks before the onset of the illness; 24 could not be contacted; and 52 children or their parents declined to participate. In addition, complete data were not available for 13 children with potential cases of chickenpox. We enrolled 500 matched controls. Another 675 potential controls were not eligible because they had had chickenpox before the focal time; we were unable to reach the parents of 526 potential controls; and 33 whom we did reach declined to participate.

Table 2. Table 2. Comparison of the Results of PCR and Direct Fluorescent-Antibody Assay for 277 Suspected Cases of Chickenpox.

There were 330 children with potential cases of chickenpox for whom complete data were available. Of these, only 35 percent were seen by a nurse-practitioner or physician. The diagnosis was made over the telephone for the other 65 percent. The results of the PCR tests and the direct fluorescent-antibody assays for the suspected cases of chickenpox are shown in Table 2 (the first 53 potential cases were not included in this comparison because the research assistants were learning the proper techniques for obtaining the samples and preparing the slides for the direct fluorescent-antibody assays). Clinically useful results (either positive or negative) were obtained more frequently with the PCR test (258 of 277 [93 percent]) than with the direct fluorescent-antibody assay (203 of 277 [73 percent], P<0.001). The first 100 samples with positive results on the PCR test and the samples from all vaccinated children with chickenpox and a positive PCR result were analyzed to determine the type of varicella–zoster virus. All were wild-type virus.

Table 3. Table 3. Characteristics of All Children with PCR-Positive Cases of Chickenpox.

Overall, the PCR test was positive in 243 of the 330 children with potential cases of chickenpox (74 percent). Characteristics of these 243 children are shown in Table 3. The illness was substantially less severe among the children who had received the varicella vaccine. Children with chickenpox who had not received the vaccine were more likely to have a fever and to be treated with an antipyretic (usually acetaminophen).

Table 4. Table 4. Characteristics of Children with Chickenpox and Controls.

Table 5. Table 5. Effectiveness of the Vaccine.

Complete data were available for one or both of the matched controls for 202 of the 243 children with positive results on PCR; estimation of the effectiveness of the varicella vaccine was based on these 202 cases and their 389 controls (Table 4). The children with chickenpox were substantially less likely than the controls to have received the varicella vaccine (23 percent vs. 61 percent), but there was no difference in the proportions that had received the MMR vaccine (100 percent in both groups). The overall effectiveness of the varicella vaccine is shown in Table 5. The vaccine was 85 percent effective (95 percent confidence interval, 78 to 90 percent; P<0.001). The vaccine was 97 percent effective (95 percent confidence interval; 93 to 99 percent; P<0.001) against moderately severe and severe disease. The vaccine's effectiveness for children younger than 5 years of age was 79 percent (95 percent confidence interval, 61 to 89 percent; P<0.001); for children 5 to 10 years of age, it was 89 percent (95 percent confidence interval, 80 to 94 percent; P<0.001); and for children older than 10 years of age, it was 92 percent (95 percent confidence interval, 45 to 99 percent; P=0.03).

Information about one or two matched controls was available for 40 children with suspected cases of chickenpox whose PCR result was negative and for 19 of those whose PCR result was indeterminate. Among the groups containing a child with chickenpox-like illness but negative PCR results, 29 of 40 children with the suspected cases (72 percent) and 48 of their 74 matched controls (65 percent) had received the varicella vaccine (P=0.53); among the groups containing a child with chickenpox-like illness and indeterminate PCR results, 14 of 19 children with the suspected cases (74 percent) and 23 of their 37 matched controls (62 percent) had received the varicella vaccine (P=0.55). Thus, the vaccine was not effective in preventing illness associated with either negative or indeterminate results on a PCR assay for varicella–zoster virus.

According to the available reports of the parents, chickenpox developed within 10 to 21 days in at least one person who had had close contact with a child with chickenpox-like illness in 30 percent (58 of 195) of the PCR-positive cases (including 7 of 43 PCR-positive cases in vaccinated children). By contrast, no parents of any of the 71 children with chickenpox-like illness and negative or indeterminate results on PCR reported any contacts in whom varicella subsequently developed (P<0.001). Similarly, children with chickenpox-like illness and positive results on PCR were more likely to have had known exposure to someone with chickenpox (187 of 243 [77 percent]) than were children with chickenpox-like illness whose PCR result was either negative or indeterminate (22 of 87 [25 percent]; odds ratio, 9.9; P<0.001).

These results indicate that the effectiveness of the varicella vaccine as it is used in actual practice is excellent, at least in the short term. Virtually all the vaccinated children in whom chickenpox subsequently developed (all of whom were infected with wild-type virus) had very mild disease. These findings are consistent with those of a study by the Centers for Disease Control and Prevention that show a marked decline in the incidence of chickenpox in areas where the vaccine is widely used,¹⁸ as well as with reports of the effectiveness of the vaccine after outbreaks in day-care centers.^19-21

It may be too soon to assess the duration of vaccine-induced immunity to chickenpox. Varicella–zoster virus is still circulating in the population, so boosting of vaccine-induced immunity may still occur commonly. However, as the incidence of chickenpox declines, such natural boosting of immunity will become increasingly rare. Because ours is an ongoing study, we expect that, over time, we will be able to detect any waning of immunity that occurs.

By the PCR test, 19 percent of the samples from children in whom chickenpox was suspected were negative and nearly 7 percent had indeterminate results. The hypothesis that these samples were from children who did not truly have chickenpox is supported by the fact that a substantially lower proportion of these children than of the children with PCR-positive cases had had recent exposure to someone with chickenpox. Similarly, none of these children had contacts in whom chickenpox developed within 10 to 21 days of their illnesses (as compared with 30 percent of the children with PCR-positive cases). Most children (65 percent) in whom chickenpox was suspected were given the diagnosis over the telephone and were never seen by a clinician. This information has important public health implications because if these children had not participated in the study, some of them would be believed erroneously to have a history of chickenpox and so would not be given the varicella vaccine even though they remain susceptible to chickenpox.

Because case–control studies like ours are observational rather than experimental, they are especially susceptible to bias. We found that although a much higher proportion of controls than of children with chickenpox had received the varicella vaccine, all the children in the study had received the MMR vaccine, suggesting that selection bias related to the likelihood of vaccination was not an important factor in the study. Far fewer children with PCR-positive cases of chickenpox than their matched controls had received the varicella vaccine. If this difference (which resulted in an estimate of vaccine effectiveness of 85 percent) were due to bias, a similar difference should also have been observed between the children with cases that were negative or indeterminate by PCR and their matched controls. Such a difference would have indicated that the vaccine was also effective against illnesses associated with negative and indeterminate results on a PCR assay for varicella–zoster virus. However, as expected, the vaccine was not effective against these illnesses.

There was some risk that matching according to pediatric practice might bias the results toward the null hypothesis because of overmatching. However, the effectiveness that we found, as well as the substantial variation in the proportion of controls from each practice who had been vaccinated, suggests that possible overmatching did not have a substantial effect.

In summary, the effectiveness of the vaccine was excellent. The great majority of breakthrough cases of chickenpox among vaccinated children were mild. We conclude that, thus far, the varicella vaccine, as it is used in clinical practice, is highly effective.