Correspondence
Computer-Aided Screening Mammography
N Engl J Med 2007; 357:83-85July 5, 2007
- Article
To the Editor:
Fenton et al. (April 5 issue)1 discuss the accuracy of interpretation of screening mammography with and without the use of computer-aided detection (CAD). However, we believe that such a comparison may be unreliable, given the uncertainty associated with adjustment for differences in variables (the age of patients, the number of procedures performed, and the experience of radiologists), let alone clinically important variables not factored into the equation. The latter include diagnostic skills (not entirely accounted for by workload), the quality of mammography, and experience with percutaneous biopsy. The value of CAD performed by one radiologist should be compared with that of double reading, currently the adopted standard in screening programs in Europe and the United Kingdom.2,3 The evidence on CAD has been inconsistent and mostly indicates either no benefit2 or a modest gain in sensitivity at the expense of an increased rate of recall, as was reported by Fenton et al. A proper assessment of the effectiveness and cost of CAD should have been based on large, prospective, controlled trials rather than on observational studies.
3 ReferencesStefano Ciatto, M.D.
Istituto Scientifico Prevenzione Oncologica, 50133 Florence, Italy
s.ciatto@cspo. it Nehmat Houssami, Ph.D.
University of Sydney, Sydney, NSW 2006, Australia1
Fenton JJ ,Taplin SH ,Carney PA , et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med 2007;356:1399-1409
Full Text | Web of Science | Medline2
Ciatto S ,Ambrogetti D ,Bonardi R , et al. Second reading of screening mammograms increases cancer detection and recall rates: results in the Florence screening programme. J Med Screen 2005;12:103-106
CrossRef | Web of Science | Medline3
Khoo LA ,Taylor P ,Given-Wilson RM . Computer-aided detection in the United Kingdom National Breast Screening Programme: prospective study. Radiology 2005;237:444-449
CrossRef | Web of Science | Medline
To the Editor:
The study by Fenton et al., although large with respect to sample size and eloquently described, largely ignores four important points. First, substantial differences exist between the sensitivity levels the authors report and the results of the Digital Mammographic Imaging Screening Trial (DMIST) reported in 2005 in the Journal by some of the same authors.1 Second, as compared with other studies of CAD, the study by Fenton et al. shows substantially larger increases in recall rates (>30%), which raises questions about the training of radiologists and how CAD was actually used. Third, the effect of CAD depends on the sensitivity of screening without CAD; the high sensitivity that Fenton et al. report limits the potential for improvement. Fourth, the data for the CAD group, which most of the authors' conclusions are based on, are notably multivariable (sites, readers, and cases), and the group was quite small. The possibility of outliers with respect to specific sites, readers, or the study itself should have been analyzed and discussed.
1 ReferencesDavid Gur, Sc.D.
University of Pittsburgh, Pittsburgh, PA 15261
gurd@upmc.edu 1
Pisano ED ,Gatsonis C ,Hendrick E , et al. Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 2005;353:1773-1783[Erratum, N Engl J Med 2006;355:1840.]
Full Text | Web of Science | Medline
To the Editor:
It is worrisome that the main conclusion of Fenton et al. was drawn from a receiver-operating-characteristic (ROC) analysis with very little methodologic explanation. We find it puzzling that the ROC curves are highly symmetric around a line from upper left to bottom right, in contrast to all other screening ROC curves, which, in our experience, skew to the left. Perhaps this is an artifact of using the Breast Imaging Reporting and Data System (BI-RADS), with which most radiologists use only 0 through 2 in screening. ROC curves estimated from three-point scales are highly unreliable. We are surprised that the authors did not compare conditions with and without CAD using data only from the seven CAD facilities. This analysis would have obviated the need to make adjustments for facilities and radiologists that were neither described nor validated. Finally, the authors do not state whether they considered variability among readers in statistical testing and, if so, how. The omission of reader variability would overestimate the true significance of their findings.
Robert M. Nishikawa, Ph.D.
Robert A. Schmidt, M.D.
Charles E. Metz, Ph.D.
University of Chicago, Chicago, IL 60637
r-nishikawa@uchicago.edu Drs. Nishikawa, Schmidt, and Metz report being shareholders in and receiving royalties from Hologix. Dr. Nishikawa reports receiving research support from Hologix. No other potential conflict of interest relevant to this letter was reported.
To the Editor:
Fenton et al. claim that 51.2% of 38 radiologists read more than 2000 mammograms per year, whereas 40.9% read 1001 to 2000. This is a minimum caseload of 54,469 mammograms per year for this group of radiologists. If the mean duration of experience with CAD was 18 months, why were there only 31,186 mammograms in the population being studied?
The average caseload per facility with CAD was 4455 mammograms divided among 38 radiologists. Experience with CAD ranged from 2 to 25 months, but caseload and duration of experience with the system are not linked. What if expertise is proportional to duration of experience and the facility with the largest caseload had only 2 months of experience?
In a survey of multiple facilities where interpretive expertise is variable, less skillful operators will dilute the effectiveness of the technique. As compared with results before the implementation of CAD, the range of each performance measure increases with the use of CAD. This increased distribution indicates a variation in the expertise of the radiologists who are using the system.
The selection of a high-volume site with dedicated mammographers would have decreased variations among both radiologists and patient populations and would have allowed a truer indication of the effect of CAD on mammography performance.
James F. Ruiz, M.D.
Woman's Hospital, Baton Rouge, LA 70895
james.ruiz@womans. org To the Editor:
Five prospective clinical studies of 53,538 sequentially read screening mammograms showed that CAD increased the cancer-detection rate by 4.6 to 19.5%.1-3 The increases in the rates of recall and biopsy (7.8 to 18.5%) were concordant with the increases in cancer detection. When more cancers are detected with CAD, more recalls and biopsies will be performed.
In contrast, the study by Fenton et al. was a retrospective comparison of 31,186 screening mammograms read with input from CAD. The authors report a nonsignificant increase in the rate of detection for all cancers and a significant 33% increase (P=0.049) in the rate of detection of ductal carcinoma in situ (DCIS). We believe that prospective studies are a better method to judge the usefulness of an intervention such as CAD.
3 ReferencesStephen A. Feig, M.D.
University of California, Irvine, Orange, CA 92868
sfeig@uci.edu Robyn L. Birdwell, M.D.
Brigham and Women's Hospital, Boston, MA 02115Michael N. Linver, M.D.
Radiology Associates of New Mexico, Albuquerque, NM 871101
Birdwell RL ,Bandodkar P ,Ikeda DM . Computer-aided detection with screening mammography in a university hospital setting. Radiology 2005;236:451-457
CrossRef | Web of Science | Medline2
Morton MJ ,Whaley DH ,Brandt KR ,Amrami KK . Screening mammograms: interpretation with computer-aided detection -- prospective evaluation. Radiology 2006;239:375-383
CrossRef | Web of Science | Medline3
Dean JC ,Ilvento CC . Improved cancer detection using computer-aided detection with diagnostic and screening mammography: prospective study of 104 cancers. AJR Am J Roentgenol 2006;187:20-28
CrossRef | Web of Science | Medline
Author/Editor Response
In our study of diverse radiologists in real-world practice, the use of CAD was not associated with an increase in cancer detection, which contrasts with prospective, single-facility studies in which experienced radiologists used CAD according to defined protocols.1 We therefore concur with Gur that our study raises important questions about how community radiologists are trained to use CAD. For example, if some radiologists avert recalls on the basis of the absence of CAD marks on otherwise suspicious lesions, the system could have unanticipated and even deleterious effects on performance.2
Observational studies may be vulnerable to confounding, yet our results were not clearly explained by differences in the characteristics of patients, radiologists, or facilities. Nor were our findings attributable to outliers in the data. Although one facility used CAD for only 2 months, results were similar after excluding this facility. Finally, the sensitivities in our study are consistent with those in many previous community-based studies.3
Although some of the correspondents cite the inexperience of radiologists at facilities using CAD in our study, more than 70% of these radiologists had 10 or more years of mammography experience. Self-reported annual reading volumes among study radiologists (including those at facilities without CAD) may suggest that larger numbers of mammograms should have been available. But radiologists may interpret many mammograms outside study facilities.
The methods of the ROC analysis, including adjustment for reader characteristics, have been described elsewhere.4 The ROC curves are symmetric because the same standard deviation is assumed for the underlying latent distributions, an assumption that is supported by the fitted model and the symmetric shape of the empirical ROC distribution. The BI-RADS scale has six possible values (0 to 5), all of which were adequately represented in our large, population-based sample.
Feig et al. cite a significant increase in the rate of detection of DCIS in our study. However, the use of CAD was not associated with a significant increase in the rate of detection of either invasive breast cancer or DCIS. Rather, the proportion of total cancers detected that were DCIS increased with CAD (from 28.1% to 37.4%, P=0.049). Although the benefits of early DCIS detection are plausible, the benefits of CAD would be more certain if it clearly increased detection of invasive breast cancer. An adequately powered, randomized trial with this end point would require the interpretation of hundreds of thousands of mammograms by dozens of radiologists,5 and to our knowledge, no such study is planned. Thus, larger observational studies of the use of CAD by diverse radiologists are necessary to clarify the relative benefits and harms of a now widely disseminated technology.
5 ReferencesJoshua J. Fenton, M.D., M.P.H.
University of California, Davis, Sacramento, CA 95817
joshua.fenton@ucdmc. ucdavis. edu William E. Barlow, Ph.D.
Cancer Research and Biostatistics, Seattle, WA 98101Joann G. Elmore, M.D., M.P.H.
University of Washington, Seattle, WA 981041
Freer TW ,Ulissey MJ . Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001;220:781-786
CrossRef | Web of Science | Medline2
Alberdi E, Povyakalo AA, Strigini L, et al. Use of computer-aided detection (CAD) tools in screening mammography: a multidisciplinary investigation. Br J Radiol 2005;78:Spec. No. 1:S31-S40.
3
Yankaskas BC ,Taplin SH ,Ichikawa L , et al. Association between mammography timing and measures of screening performance in the United States. Radiology 2005;234:363-373
CrossRef | Web of Science | Medline4
Barlow WE ,Chi C ,Carney PA , et al. Accuracy of screening mammography interpretation by characteristics of radiologists. J Natl Cancer Inst 2004;96:1840-1850
CrossRef | Web of Science | Medline5
Jiang Y ,Miglioretti DL ,Metz CE ,Schmidt RA . Breast cancer detection rate: designing imaging trials to demonstrate improvements. Radiology 2007;243:360-367
CrossRef | Web of Science | Medline
- Citing Articles (5)
Citing Articles
1
N Houssami, R Given-Wilson, S Ciatto. (2009) Early detection of breast cancer: Overview of the evidence on computer-aided detection in mammography screening. Journal of Medical Imaging and Radiation Oncology 53:2, 171-176
CrossRef2
Jinshan Tang, R.M. Rangayyan, Jun Xu, I. El Naqa, Yongyi Yang. (2009) Computer-Aided Detection and Diagnosis of Breast Cancer With Mammography: Recent Advances. IEEE Transactions on Information Technology in Biomedicine 13:2, 236-251
CrossRef3
Gilbert, Fiona J., Astley, Susan M., Gillan, Maureen G.C., Agbaje, Olorunsola F., Wallis, Matthew G., James, Jonathan, Boggis, Caroline R.M., Duffy, Stephen W., . (2008) Single Reading with Computer-Aided Detection for Screening Mammography. New England Journal of Medicine 359:16, 1675-1684
Full Text4
Akira Hasegawa. (2007) FUJIFILM CAD for Mammography. Japanese Journal of Radiological Technology 63:12, 1418-1423
CrossRef5
Hiroshi Fujita. (2007) Short Survey on Current Status of Computer-aided Diagnosis (CAD). Japanese Journal of Radiological Technology 63:12, 1389-1395
CrossRef
