Correspondence

The NSABP Trials

N Engl J Med 1994; 331:809-810September 22, 1994

Article

To the Editor:

I am stunned by Dr. Poisson's explanation (May 19 issue)1 of his behavior in regard to Protocol B-06 of the National Surgical Adjuvant Breast and Bowel Project (NSABP). It is outrageous to suggest that it is permissible for an individual investigator to ignore eligibility criteria. Each investigator has the choice to participate or not to participate in a collaborative study. Once Dr. Poisson made the choice to participate, he was obliged to follow the protocol.

Charles Levenback, M.D.
M.D. Anderson Cancer Center, Houston, TX 77030

1 References

1. 1

   Poisson R. Fraud in breast-cancer trials. N Engl J Med 1994;330:1460-1460
   Web of Science | Medline

To the Editor:

Your editorial (May 19 issue)1 seemed to agree with Dr. Fisher's claim that because the fraud had no effect on the findings of the study, there was no harm to public health. From a biostatistical standpoint, what the fraud showed was that the statistical quality control was grossly inadequate in the NSABP studies. Hence, whether or not some fraudulent cases are eliminated post hoc, any findings lack scientific validity.

Irwin D. Bross, Ph.D.
Biomedical Metatechnology, Eggertsville, NY 14226

1 References

1. 1

   Angell M, Kassirer JP. Setting the record straight in the breast-cancer trials. N Engl J Med 1994;330:1448-1450
   Full Text | Web of Science | Medline

To the Editor:

To me, the important point should have been, and should still be, to assure concerned patients that, to date, there has been no reason to believe that their treatment was inappropriate. It is regrettable that this matter has become a political football, with officials of the National Cancer Institute (NCI) attempting to place as much blame as possible on Dr. Fisher because of their obvious intimidation by Congressman Dingell.

The author of a recent editorial in the Lancet expressed the much more reasonable viewpoint that “this affair has been poorly handled” and that “the NCI and media frenzy, much of it directed personally at Fisher, cannot be justified.”1 The editorialist also pointed out that “Fisher's innovative way of thinking about and organizing breast cancer trials has been copied over and over again by other groups and stands as a testimonial to his foresight and commitment to meeting the challenge of breast cancer.”

Norman Davis, M.D.
1322 Squirrel Hill Ave., Pittsburgh, PA 15217

1 References

1. 1

   Breast cancer: clearing trails in the forest without losing our wayLancet 1994;343:1049-1050
   CrossRef | Web of Science | Medline

To the Editor:

Several of the letters discussing falsification of data in the NSABP studies stress that the results of Protocol B-06, which found no significant clinical difference between lumpectomy and total mastectomy,1,2 are not altered by the exclusion of the data collected by Dr. Poisson. The results of these reanalyses have persuaded the NCI, the National Institutes of Health, and the Office of Research Integrity that continued reliance on the original findings poses no threat to public health3.

I contend that the reanalyses alone do not provide this level of reassurance, because the original conclusions are all negative -- that is, they are findings of statistical nonsignificance. When confronted with a nonsignificant test result, one must ask, “What is the power of the test?” If the power is not substantial (usually 80 percent or more) for clinically important effect sizes, then the nonsignificance may merely reflect an inadequate study design.

This idea is relevant to Protocol B-06. In an analysis of survival, the power depends on the sample size through the number of events. Stablein's reanalysis4 indicates that in the cohort of women who were eligible for enrollment and accepted therapy, there were 933 treatment failures or deaths; elimination of the data from St. Luc Hospital reduces this total to 776. For a comparison of any two treatment groups, we can assume that there are two-thirds this many events -- a total of 622, and 517 with the St. Luc data deleted. Because this trial intends to show an equivalence, high power is required even when the effects are small -- for example, when the relative hazard of the worse treatment versus the better treatment is 1.25 (roughly a 20 percent decrease in the mean time to an event). By Schoenfeld's method,5 the chance of detecting such a difference at a 0.05 level of significance is 79.5 percent if all the data are included and 71.8 percent if the data from St. Luc Hospital are excluded. According to the usual standards, this test has barely adequate power if all the data are used and clearly inadequate power if the St. Luc data are excluded.

This is not to say that we should disbelieve the original conclusions drawn from the results of Protocol B-06. Others have corroborated the findings in independent trials, and the admitted pattern of falsification would not be expected to introduce a strong bias one way or the other. Without this information, however, the reanalyses alone do not warrant the level of reassurance that has been drawn from them.

Daniel F. Heitjan, Ph.D.
Pennsylvania State University College of Medicine, Hershey, PA 17033

5 References

1. 1

   Fisher B, Bauer M, Margolese R, et al. Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer. N Engl J Med 1985;312:665-673
   Full Text | Web of Science | Medline

2. 2

   Fisher B, Redmond C, Poisson R, et al. Eight-year results of a randomized clinical trial comparing total mastectomy and lumpectomy with or without irradiation in the treatment of breast cancer. N Engl J Med 1989;320:822-828
   Full Text | Web of Science | Medline

3. 3

   NCI issues information on falsified data in NSABP trialsJ Natl Cancer Inst 1994;86:487-489
   CrossRef | Web of Science | Medline

4. 4

   Stablein DM. Final report of a reanalysis of NSABP protocol B-06. CancerFax document 400027. Bethesda, Md.: National Cancer Institute, May 1, 1994.
   

5. 5

   Schoenfeld DA. Sample-size formula for the proportional-hazards regression model. Biometrics 1983;39:499-503
   CrossRef | Web of Science | Medline

To the Editor:

In an analysis of NSABP Protocol B-17, Fisher et al. said they eliminated 24 cases from one hospital because of problems with the quality of the data1. We now know these were Dr. Poisson's cases at St. Luc Hospital in Montreal, and the Journal appropriately criticized Dr. Fisher for failing to explain the circumstances2. But why did the Journal itself not demand an explanation from Dr. Fisher before publishing the report?

Lawrence K. Altman, M.D.
The New York Times, New York, NY 10036

2 References

1. 1

   Fisher B, Costantino J, Redmond C, et al. Lumpectomy compared with lumpectomy and radiation therapy for the treatment of intraductal breast cancer. N Engl J Med 1993;328:1581-1586
   Full Text | Web of Science | Medline

2. 2

   Angell M, Kassirer JP. Setting the record straight in the breast-cancer trials. N Engl J Med 1994;330:1448-1450
   Full Text | Web of Science | Medline

Author/Editor Response

The editors reply:

The letters of Drs. Levenback, Bross, Davis, and Heitjan speak for themselves.

Dr. Altman's question requires a response. In dealing with the manuscript Dr. Altman refers to, we assumed that there were legitimate concerns about the data that would lead prudent investigators to exclude them from the analysis. For example, some of the records might have been accidentally destroyed. We had no reason to suspect misconduct -- which would have immediately raised questions about the integrity of the rest of the work -- and so we did not “demand an explanation.”

Marcia Angell, M.D.
Jerome P. Kassirer, M.D.