Cumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction

The techniques of meta-analysis of randomized, controlled trials have been well described.^{1 2 3 4} Traditionally, the individual trials, grouped according to the class of treatment, the end point of interest, or both, are presented in order of publication or in descending order of quality. The pooled differences between treatment and control groups are then presented as a summation of the results of the individual trials, of necessity at some arbitrary point in time after the literature search has been completed. For this discussion we focus on investigations with outcomes expressed as counts in two-by-two tables, as numbers of successes and failures in the treatment and control groups.

Figure 2. Figure 2. Cumulative Meta-Analyses of 60 Trials of Intravenous Thrombolytic Agents for Myocardial Infarction by the MantelHaenszel Fixed-Effects Method and DerSimonian and Laird Random-Effects Method.

The odds ratios and 95 percent confidence intervals for an effect of treatment on mortality are shown on a logarithmic scale. With the fixed-effects method the statistical significance reaches <0.05 in 1973, and with the random-effects method in 1977. This is the largest difference attributable to heterogeneity of treatment effect in these trials of therapies for myocardial infarction. The trial drugs were streptokinase, urokinase, anisoylated plasminogen streptokinase activator complex, and alteplase (tissue plasminogen activator).

Two statistical models have been employed in our meta-analyses. The fixed-effects model described by Mantel and Haenszel⁶ assumes identical treatment effects in the studies (homogeneity of the true treatment effect), and the variances around each mean effect depend primarily on the size of each study. The random-effects model of DerSimonian and Laird⁷ includes between-study differences in treatment effects in the calculation of the variances, leading to wider confidence intervals when a given level of heterogeneity in treatment effect is observed.⁸ The MantelHaenszel results are usually reported as an odds ratio with variances calculated either by the procedure of Peto as described by Yusuf et al.¹ and Collins et al.⁹ or by the method of Robins, Greenland, and Breslow.¹⁰ The DerSimonian and Laird results have traditionally been reported as absolute percentage differences, with confidence intervals, but they can be reported as odds ratios, as in Figure 2.

Although we have used several available methods in our routine analyses, we have not found important differences in over 45 meta-analyses,¹¹ except for the anticipated wider confidence intervals of the DerSimonian and Laird method when heterogeneity in treatment effect is observed.⁸ We prefer the two-method approach; differences in the confidence intervals determined by each method occur only when there is heterogeneity.⁸ A choice can then be made between the more and the less conservative approach.

We employed the principles discussed above in cumulative meta-analysis by updating the pooled estimate of the treatment effect each time the results of a new trial were published for each of the treatments for myocardial infarction that we were evaluating. We started our literature search for meta-analyses of randomized, controlled trials of the treatment of myocardial infarction with a recent review by Yusuf et al.,¹² assembled all the additional published meta-analyses of treatment and secondary prevention, and added the results of trials reported since the last meta-analyses were submitted for publication. We also searched MEDLINE and the references of published papers for randomized, controlled trials that might not have appeared in the published meta-analyses. Some of the meta-analyses included trials of different routes of administration and different drugs of the same class (e.g., the beta-blockers). Because our principal interest was the overall treatment effect of a particular therapeutic intervention, and because there were no significant differences in mortality among the various routes of administration or individual drugs within a class, we pooled all the studies of treatments within a given drug class. Although total mortality was our preferred end point, in the rare instances in which the original paper presented only mortality from cardiovascular causes we used that figure. Although most of the data on mortality were confined to the period of hospitalization, a few studies gave data only in terms of weeks or months after infarction. For the data on short-term treatment we included mortality up to a limit of three months, because of the likelihood that any deaths up to that time were directly related to the initial infarction and its treatment. When a study reported on more than one time period, we used the one closest to the time of hospitalization. For the studies of secondary prevention we included the duration described by the authors, relying on an important characteristic of meta-analysis — the fact that differences between treatment and control groups are determined before pooling, so that each study has internal controls. In presenting the significance of the results we used only two-sided P values.

We selected the treatments to include on the basis of the availability of data. For each treatment at least one meta-analysis had been published, and we then added trials published since the original meta-analysis. Eight treatments for acute myocardial infarction have been analyzed in published meta-analyses: intravenous thrombolytic drugs,^{13 14 15 16 17 18 19 20 21 22} intravenous vasodilators (nitroglycerin and nitroprusside),²³ intravenous or oral beta-blockers,^{1 , 24 25 26} anticoagulants,²⁷ antiplatelet drugs,²⁸ lidocaine prophylaxis against primary ventricular fibrillation,^{29 30 31} calcium-channel blockers,³² and intravenous magnesium salts.³³

The seven methods for delaying death after discharge were oral beta-blockers,^{1 , 24 , 34 35 36 37 38 39} anticoagulants,^{40 41 42} antiplatelet drugs,^{43 44 45 46} calcium-channel blockers,³² cholesterol-lowering treatments,^{47 48 49} rehabilitation exercise regimens,^{50 , 51} and Class I antiarrhythmic drugs.⁵²

We obtained all but two of the original published papers that were the basis of these meta-analyses and confirmed that they were randomized, controlled trials and that their end points — death within the stated periods — could be pooled. For data published in the two papers we could not obtain and data published only in the meta-analyses, we relied on the accuracy of the authors of the meta-analyses. In pooling the results of the trials we assumed that a reasonable effort had been made by the original investigators to include all patients who presented with acute myocardial infarction unless the patients had any of the then accepted contraindications to the therapy under investigation. In the absence of established subgroup responses to given agents we included trials with various criteria for acceptance and assumed that patients omitted in the absence of any other contraindications (such as those over 70 years old or living in remote areas) would not have changed the results had they been included.

Figure 1. Figure 1. Conventional and Cumulative Meta-Analyses of 33 Trials of Intravenous Streptokinase for Acute Myocardial Infarction.

The odds ratios and 95 percent confidence intervals for an effect of treatment on mortality are shown on a logarithmic scale. A bibliography of the published trial reports is available from the authors.

The data on 33 trials that compared intravenous streptokinase with a placebo or no therapy in patients who had been hospitalized for acute myocardial infarction are presented as traditional and cumulative meta-analyses in Figure 1. In the traditional analysis the effect of treatment on total mortality for each study is presented as an odds ratio and 95 percent confidence interval, and the pooled estimate of the treatment effect with its confidence interval is presented at the bottom. Although the effect of treatment on mortality favored the active drug in 25 of the 33 trials, in only 6 was statistical significance achieved. The pooled estimate of the treatment effect, however, is significant.

The same data are also presented in Figure 1 as if a new meta-analysis had been performed each time the results of a new trial were reported. The years during which the treatment effect became statistically significant by the MantelHaenszel procedure were 1971 for a two-sided P value of <0.05 (exact P = 0.023), 1973 for a P value of <0.01 (odds ratio, 0.74; 95 percent confidence interval, 0.59 to 0.92; P = 0.007), and 1977 for a P value of <0.001. With this technique it can be seen that the very large trials — the Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI)⁵³ and the Second International Study of Infarct Survival (ISIS-2)²⁸ (without the aspirin group) — did not change the already established evidence of efficacy; published in 1986 and 1988, respectively, they narrowed the confidence intervals slightly and increased the already impressive statistical significance of the overall treatment effect. The cumulative method indicates that intravenous streptokinase could have been shown to be lifesaving almost 20 years ago, long before its submission to and approval by the Food and Drug Administration and its general adoption in practice. We have not included intracoronary streptokinase or the newer intravenous thrombolytic drugs in the data shown in Figure 1 because their effects on outcome are essentially the same, and the efficacy of thrombolysis was demonstrated before the newer agents became available.

The other intravenous drugs (urokinase, anisoylated plasminogen streptokinase activator complex, and alteplase [tissue plasminogen activator]), however, are included along with streptokinase in Figure 2, which is designed to show the different years during which a treatment may first appear to be effective when fixed-effects and random-effects models are used. The 60 trials are grouped according to year of publication to make data more legible. In this case heterogeneity in the estimates of the treatment effect postponed statistical significance until a later date when the more conservative method of analysis (that of DerSimonian and Laird⁷) was used. For all other drugs the difference was no more than one year.

Figure 3. Figure 3. An Effective Therapy for Acute Myocardial Infarction (Oral or Intravenous Followed by Oral Beta-Blockers) and One with an Unlikely Beneficial Effect (Prophylactic Lidocaine).

The odds ratios and 95 percent confidence intervals for an effect of treatment on mortality are shown on a logarithmic scale.

An example of a treatment whose effect on mortality is small enough to require tens of thousands of patients to demonstrate it, beta-blockers for acute myocardial infarction, is presented in Figure 3, along with an example of an unimpressive treatment, prophylactic lidocaine. Again in the case of beta-blockers, two large studies were compatible with the results of the previous small ones.

Table 1. Table 1. Cumulative Meta-Analyses of Treatments of Acute Myocardial Infarction and Secondary Prevention.*

Summaries of the cumulative meta-analyses are presented in Table 1. Statistically significant reductions in mortality associated with the treatment of acute myocardial infarction were demonstrated for thrombolytic agents, intravenous nitroglycerin and sodium nitroprusside, antiplatelet agents, anticoagulants, magnesium salts, and beta-blockers. For lidocaine and calcium-channel blockers the differences have not reached statistical significance, and the current difference suggests that these treatments either have no beneficial effect or may be harmful.

Statistically significant reductions in long-term mortality that were produced by secondary prevention (as indicated by the movement of the upper confidence interval to a fraction less than 1 ) were demonstrated first for beta-blockers and then for rehabilitation programs, antiplatelet agents, pooled cholesterol-lowering measures (diet, drugs, and ileal-bypass surgery), and oral anticoagulants, in that order. A significant adverse effect was observed when the results of the trials of Class I antiarrhythmic drugs were pooled, and no beneficial effect has been demonstrated for calcium-channel blockers. The benefit of antiplatelet drugs given after myocardial infarction in preventing death was statistically significant after only two studies and 2768 patients in 1976, and after four studies in 1979, but three more studies in 1980 raised the pooled P value to just above 0.05. The reversal to nonsignificance resulted from the inclusion of the Aspirin Myocardial Infarction Study (AMIS), in which the risk factors were significantly maldistributed before the start of therapy.⁵⁴ The data for aspirin are therefore presented in Table 1 with and without the AMIS results.

The upper limits of the confidence intervals occasionally fluctuate around 1 in our cumulative meta-analyses. In no instance in which one arm of a trial has become statistically significant have we later found a reversal giving the other arm statistical significance.

Table 1 shows the times at which the various independently evaluated treatments became effective at various levels of statistical significance. Certain treatments became significantly effective in a short time or after only a few thousand patients had been randomized. These treatments also had a substantial beneficial clinical effect (a final odds ratio of about 0.80 or less, corresponding to a reduction in mortality of 20 percent or more). Examples are thrombolytic drugs, intravenous vasodilators and magnesium salts, aspirin, and anticoagulants for acute myocardial infarction, and oral beta-blockers and anticoagulants for secondary prevention. For other treatments, the reduction in mortality was smaller, and it did not become statistically significant until tens of thousands of patients had been randomized (e.g., beta-blockers for the treatment of acute myocardial infarction and antiplatelet agents and various approaches to lowering the serum cholesterol for secondary prevention). In the case of prophylactic lidocaine for acute disease and calcium-channel blockers for both treatment and secondary prevention, the odds ratio for mortality was consistently above 1, suggesting a possible harmful effect, but the width of the 95 percent confidence interval prevented the exclusion of a possible beneficial effect (which is likely to be small and of little relevance if it exists at all). Finally, in the case of Class I antiarrhythmic agents for secondary prevention, statistically significant evidence of harm does exist.

Issues Raised by Cumulative Meta-Analyses

The technique of performing a new meta-analysis every time a new trial appears raises a number of methodologic and clinical issues. One involves the applicability of the concept of cumulative meta-analysis to the future monitoring of the clinical-trial literature. Should one adjust the P value for statistical significance when multiple testing of the emerging trials is carried out? This issue was previously discussed⁵⁵ and is currently the subject of methodologic research. Whether routinely to adjust the P value of meta-analyses for multiple looks, as suggested by Yusuf et al.,⁵⁶ according to the recommended corrections for multiple looks at evolving data in randomized control trials⁵⁷ is a complex issue.

We recognize that different analysts may use different protocols and therefore include somewhat different research papers, and perhaps different end points and different numerical results, and that these differences can make it difficult to determine when a cumulative meta-analysis becomes positive.

Deciding when to stop ongoing trials or to adopt or reject an investigated treatment is a multifaceted clinical and social problem that cumulative meta-analyses may simplify by supplying important data. Variations in the quality of the individual studies and potential distortions due to publication bias^{58 59 60} are important factors to be kept in mind when evaluating cumulative meta-analyses.

When does the possibility of publication bias or variation in protocols mean that the results of pooled small trials must be confirmed by very large trials? Experience with the trials of streptokinase and beta-blockers suggests that very large trials confirm the results of the numerous small trials that are reported earlier. It is clear that subgroup responses are more readily determined in big trials, but this deficiency of small trials could be remedied if investigators planning future trials thought in terms of how their data might usefully be combined with those of other studies. Against the need for replication to learn more about subgroups must be weighed the propriety of assigning patients to a control group instead of giving them a treatment shown to be effective by the meta-analysis of a number of small trials.

Another issue in all meta-analyses is the quality and reliability of the original trials. We have included all studies in which patients were assigned to their treatments at random. We would like to go further and adjust the contribution of a given study according to its technical quality.⁶¹ However, we have not been able to relate a quantitative estimate of the quality of a study to the size of the observed difference between treatments in a large data base of randomized, controlled trials.⁶²

Implications for the Treatment of Acute Myocardial Infarction

Our meta-analyses suggest that six of the eight individual treatments for reducing mortality due to acute myocardial infarction are highly effective. Similarly, five of the seven methods of preventing or postponing death after discharge are highly effective. There is evidence that these 11 effective agents are not being recommended as often as they might be.⁵ Perhaps physicians would use these agents more often if they had access to cumulative meta-analyses of the published trials. We recognize, however, that the statistical significance of the accumulated results of randomized, controlled trials is not the only determinant of changes in prescribing by physicians; many other factors may be important in the decision-making process.

In our analyses we have not included data on side effects of the various treatments for several reasons: the difference in total mortality serves as a common measure of all fatal adverse events; the data on side effects in the original publications vary enormously in frequency and quality, especially for the acute phase of infarction; in secondary-prevention studies, quality of life is of real importance, but only recently have original studies paid attention to it. One important complication affecting the quality of life, stroke, was mentioned in only 45 of the 60 trials of thrombolytic agents. The pooled rates of stroke (which could not be separated into thrombotic and hemorrhagic) were 0.7 percent in the control group and 0.8 percent in the treatment group (odds ratio, 1.05; 95 percent confidence interval, 0.85 to 1.30). In ISIS-2²⁸ there was a slight increase in the rate of hemorrhagic stroke that was statistically significant. Thus, there may be a slight increase in the rate of hemorrhagic stroke, but that possibility must be weighed against a much greater reduction in the death rate.

Future Applications of Cumulative Meta-Analysis

Our data suggest the desirability of making the results of cumulative meta-analyses available on a regular basis to physicians in every field in which clinical trials are carried out. A prototype of updated but not cumulative meta-analyses is available to obstetricians and perinatologists. The Oxford Database of Perinatal Trials is now in its fourth year of electronic publication.⁶³ The lesson to be learned from our investigation of treatments for myocardial infarction is that trials are often part of a continuum, and those that have gone before must be considered when new ones are planned. To achieve reliable conclusions in the midst of a continuum one needs an efficient way to keep track of available trials. The large task of digesting the ever-expanding literature on clinical trials and disseminating information can be simplified with cumulative meta-analysis.