Comparison of Magnetic Resonance Imaging and Ultrasonography in Staging Early Prostate Cancer — Results of a Multi-Institutional Cooperative Trial

Patient Population

Five institutions participated in this study: the Cleveland Clinic, the Johns Hopkins Medical Institutions, Thomas Jefferson University Hospital, the University of Michigan, and the University of California, San Francisco. At each institution, from December 1987 to April 1989, any patient who was thought on clinical evaluation to have a surgically resectable tumor (clinical stage Ta or Tb, N0) was considered a potential candidate for enrollment in the study. Patients were excluded if they had a cardiac pacemaker, had undergone abdominoperineal resection, or had been treated for prostate cancer. If the patient and the clinical service agreed to treatment by radical prostatectomy, informed consent was obtained, and the patient was scheduled for both endorectal ultrasonography and MRI.

Imaging Studies

A uniform protocol for ultrasonography and MRI was developed. All institutions used state-of-the-art ultrasound equipment, with transducers of 5 to 7.5 MHz, biplanar capabilities, and sharply focused near fields. Units used in this project included those manufactured by Acuson (Mountain View, Calif.), Advanced Technology Laboratories (Bothell, Wash.), Aloka (Tokyo, Japan), Bruel and Kjaer (Marlborough, Mass.), Diasonics (Milpitas, Calif.), and General Electric (Milwaukee). All patients had biplanar examinations.

All MRI examinations were performed on a General Electric 1.5-tesla Signa MRI system. Initially, T₁-weighted (repetition time, 400 msec; echo time, 20 msec; thickness, 10 mm; and skip [space between images], 2 mm) sagittal images were used for localization. Intermediate and T₂-weighted (repetition time, 2500 msec; echo time, 40 or 80 msec, 60 or 120 msec, or 20 or 80 msec; thickness, 5 mm; and field of view, 24 or 28 cm) axial and sagittal scans through the prostate and seminal vesicles were obtained (for transverse scans, in an interleaved fashion so that sections were contiguous). To identify lymph nodes, T₁-weighted axial scans (repetition time, 600 msec; echo time, 20 msec; thickness, 10 mm; and skip, 2 mm) were obtained from the level of the urogenital diaphragm to the aortic bifurcation.

During the development of the protocol, the investigators agreed on a list of definitions of items to be evaluated. Sample images were circulated to the investigators to illustrate these definitions. During the study all images were interpreted prospectively, with the use of standardized forms specifically developed for this project. No reader interpreted both the MRI and ultrasound studies from the same patient. The examiners, technologists, and interpreters were unaware of the results of the digital rectal examination, the other imaging study, and other data; they knew only that the patient had a clinically suspected localized cancer.

Information was obtained about the presence and size of tumors, the location of tumors, periprostatic-fat infiltration, seminal-vesicle involvement, and pelvic lymphadenopathy (nodes larger than 1 cm in diameter) on MRI. Interpreters used a five-point grading scale appropriate for receiver-operating-characteristic (ROC) curve analysis to rate their degree of confidence that periprostatic and seminal-vesicle invasion had occurred.

All MRI and ultrasound scans were read again, except for those from patients enrolled in the first four months of the study. The two radiologists from each institution jointly reevaluated the MRI and ultrasound scans to assess possible changes in interpretation resulting from the availability of data from the alternative study. Second readings were performed on the scans for 121 patients.

Pathological Analysis

All specimens were uniformly prepared by fixation of the surgically excised whole prostate in neutral buffered formalin for at least 24 hours. The specimens were coated with India ink (to ensure proper orientation), serially sectioned in the axial plane at intervals of 5 mm (whole-mount sections) or 2 to 3 mm (routine sections), and embedded in their entirety. The sections were designated so that the location of each lesion within the prostate would be accurate. The following were determined for all cancers with at least one dimension of more than 5 mm: location; size in each of three dimensions — anteroposterior, cephalocaudal, and transverse; capsular penetration (defined as extension of the tumor into periprostatic adipose tissue); seminal-vesicle invasion (extension into the muscle wall of the seminal vesicle); and metastasis to pelvic lymph nodes.

Statistical Analysis

A modified TNM (tumor, node, metastasis) staging system was used to categorize the clinical, ultrasound, MRI, and pathological findings. Stage Ta was defined as describing a nonpalpable cancer detected in a specimen obtained by transurethral prostatectomy. Stage Tb₁—Tb₂ described a palpable cancer confined to one side of the prostate, and stage Tb₃ a palpable cancer involving both sides but confined to the prostate. Stage Tc₁—Tc₂ described a palpable lesion extending through the prostate capsule into the periprostatic fat, and Tc₃ a palpable lesion extending into the seminal vesicles. Stage N0 was defined as indicating no regional lymph-node metastasis; stage N1, microscopical regional lymph-node metastasis; stage N2, gross regional lymph-node metastasis; and stage N3, extraregional lymph-node metastasis (including inguinal, periaortic, supraclavicular, or axillary nodes).

Because imaging and pathological studies cannot be used to differentiate palpable and nonpalpable disease, clinical stages Ta and Tb (both localized cancers) were grouped together. Subcategories of stage Tc were grouped together since the foremost point of clinical relevance was to distinguish localized (stages Ta and Tb) from advanced (stage Tc) cancers. Because endorectal ultrasonography cannot be used to evaluate pelvic lymph nodes, information about stage N was available only for pathological and MRI data. In addition, patients with gross lymphadenopathy (diagnosed preoperatively) were not subjected to surgery.

Data were analyzed to assess the ability of each technique to classify patients correctly into one of two groups (those with localized vs. those with advanced disease) and to detect lesions accurately as determined by subsequent pathological measurement. The imaging and pathological studies provided information on the location of each lesion in the prostate — that is, right versus left, anterior versus posterior, and cephalad versus midportion versus caudad; a computer algorithm was developed to match lesions seen on pathological analysis with those seen on imaging.

Accuracy of MRI and Ultrasound Studies

The sensitivity of each imaging technique was determined by calculating the percentage of patients correctly identified as having advanced disease. We also calculated the specificity of the techniques by determining the percentage of patients correctly identified as having localized disease. Asymptotic standard errors were computed whenever the available sample size was adequate. The sensitivities of MRI and ultrasonography were compared by the McNemar test.¹⁷ The same procedure was used to compare the specificities, and the results of two comparisons were then combined according to Fisher's procedure.¹⁸ To test the hypothesis that the readers were all equally accurate, we also calculated the diagnostic accuracies of all readers who had interpreted the findings for more than four patients.

Because detection of local invasion (periprostatic or seminal vesicle) is critical in cancer staging, a more comprehensive analysis of these features was performed. For periprostatic invasion, ROC curves were fitted to the ultrasound and MRI data (pooled for all readers) with the use of a bivariate normal model. The calculations were made with the computer program CORROC2,¹⁹ and the areas under the two curves were compared. For seminal-vesicle invasion, sensitivities and specificities were calculated; studies in which seminal-vesicle invasion was considered by the readers to be "possibly, probably, or definitely present" were considered to be abnormal.

Lesion Identification and Size Determination

Once the locations of the lesions were matched on imaging and pathological studies, we were able to determine the fraction of malignant lesions seen on imaging. We were also able to determine the accuracy of imaging by measuring the lesion pathologically. Sizes were measured in three dimensions (anteroposterior, cephalocaudal, and transverse). The abilities of the two imaging techniques to detect lesions of various sizes were compared by the McNemar test.

Patient Population

The initial data set consisted of 249 consecutive patients from the five original institutions who were eligible for radical prostatectomy. Over a two-year period, one institution failed to enter any patients according to the prescribed protocol, and hence its data were excluded from the study. The mean age of the remaining 239 men was 60.8 years (range, 40 to 76). Among these patients, pathological examination of the specimens obtained during radical prostatectomy revealed no evidence of cancer in five; these patients had been given a diagnosis of cancer on the basis of transurethral prostate resection for hyperplasia. In four additional patients, the specimens were not sectioned according to the protocol. The data from these patients were excluded from analysis. Of the remaining 230 patients, 219 (95 percent) had ultrasound studies, 194 (84 percent) had MRI examinations, and 187 (81 percent) had both. Four patients who had ultrasound studies began the MRI studies but could not finish because of claustrophobia.

Staging Accuracy

Table 1. Table 1. Sensitivity and Specificity of Ultrasonography in Staging Operable Prostate Cancer in 219 Patients.*

When the sensitivities and specificities of the MRI and ultrasound studies were compared in the 187 patients who had both studies, the difference between the two techniques was of borderline significance (combined P = 0.09 by the McNemar test). The overall staging accuracy of ultrasonography was 58 percent (126 of 219 patients), with a standard error of 3 percent (Table 1). Of the 92 patients with localized disease, ultrasonography correctly identified 42, yielding a mean (±SE) specificity of 46±5 percent. Of the 127 men with extension of disease beyond the prostate, ultrasonography correctly identified 84, yielding a sensitivity of 66±4 percent; in these patients there was an exact match between the site of periprostatic invasion detected by ultrasonography and that identified by pathological examination 71 percent of the time (63 of 89 sites).

Table 2. Table 2. Sensitivity and Specificity of MRI in Staging Operable Prostate Cancer in 194 Patients.*

The overall staging accuracy of MRI was 69 percent (133 of 194 patients), with a standard error of 3 percent (Table 2). Of the 82 patients with localized disease, MRI correctly identified 47, yielding a specificity of 57±5 percent. Of the 112 men with advanced disease, MRI correctly identified 86, for a sensitivity of 77±4 percent; in these patients there was an exact match between the site of periprostatic invasion detected by MRI and that identified by pathological examination 76 percent of the time (71 of 94 sites). Among all patients having MRI studies, 185 had their pelvic nodes examined pathologically. MRI correctly identified 155 of 162 patients without tumor involvement of these nodes (specificity, 96 percent); its sensitivity was considerably lower (4 percent): only 1 of 23 involved nodes was detected.

When the overall accuracy of each technique was calculated for each reader in the study, there was considerable variation among readers. Although this was greater for the readers of ultrasound scans than for the readers of MRI scans, there was no significant difference among readers for either technique.

Predictive Value

When the sensitivities and specificities were converted to predictive values, the results were disappointing. For every 100 patients with ultrasound results suggesting extension beyond the prostate, only 63 would actually have had such disease; thus, 37 would have been falsely considered to have a more severe stage of disease. For every 100 patients with ultrasound results suggesting containment of the disease within the prostate, only 49 would actually have had such localized disease, and 51 would have been falsely considered to have a less severe stage of disease. For every 100 patients with MRI results suggesting extension beyond the prostate, only 71 would actually have had such disease; thus, 29 would have been falsely considered to have a more severe stage of disease. For every 100 patients with MRI results suggesting containment of the disease within the prostate, only 63 would actually have had such localized disease, and 37 would have been falsely considered to have a less serious stage of disease.

Accuracy for Detecting Periprostatic Extension

For the 187 patients who had both MRI and ultrasound studies there was no significant difference between the techniques in their ability to detect either periprostatic invasion or invasion of the seminal vesicles. For periprostatic extension, the mean (±SE) estimated area under the ROC curve for MRI was 0.67 (±5 percent) and for ultrasonography, 0.62 (±4 percent); these were not significantly different (two-sided P = 0.38). Agreement between the imaging studies about the site of penetration did not seem to increase the accuracy of the results. Pathological data agreed with imaging data in only 46 of 65 cases (71 percent) in which both imaging studies pointed to penetration at a specific site.

Ultrasonography correctly identified 14 of 65 seminal vesicles involved by tumor (sensitivity, 22 percent) and 270 of 308 seminal vesicles free of disease (specificity, 88 percent). The corresponding figures for MRI were 18 of 65 (28 percent) and 272 of 308 (88 percent).

Effect of Rereading the Ultrasound and MRI Scans

Table 3. Table 3. Effect of Rereading the Ultrasound and MRI Scans on Sensitivity, Specificity, and Accuracy.*

When the ultrasound scans were reexamined by readers with knowledge of the MRI results and when the MRI scans were reexamined by readers with knowledge of the ultrasound results, there were no statistically significant differences in accuracy between the first and second readings (Table 3). There was, however, a systematic trend among readers to judge the disease as being at a more serious stage on the second reading than on the first, so that sensitivities were higher and specificities lower.

Effect of Size on the Detectability of Lesions

In two patients there was incomplete information about the exact location of the lesion on MRI, ultrasonography, or both. Thus, there were 185 patients for whom we could compare the ability of MRI and ultrasonography to detect and localize malignant lesions of various sizes. There were 299 distinctly separate cancers with at least one measured dimension more than 5 mm on pathological examination; 110 men had one lesion, 47 had two, 20 had three, 6 had four, 1 had five, and 1 had six lesions.

Table 4. Table 4. Detection and Localization of Lesions by Ultrasonography, MRI, or Both, According to Lesion Size on Pathological Examination.

Among the 299 lesions, 143 (48 percent) were identified by both techniques, an additional 37 (12 percent) were detected only by MRI, and an additional 32 (11 percent) were detected by ultrasonography alone. A total of 212 lesions (71 percent) were identified by one or both of the imaging techniques. As expected, the ability to identify these lesions varied directly with size but minimally with the plane of imaging. For example, ultrasonography identified 53 percent of all lesions ≤1 cm in the anteroposterior dimension, whereas it identified 72 percent of the lesions that were larger than 1 cm. The corresponding percentages for MRI were 56 percent for the smaller lesions and 71 percent for the larger ones (Table 4). There was no statistically significant difference between the abilities of ultrasonography and MRI to identify lesions (combined P value >0.05 by the McNemar test).

Increasing concern about the rising costs of health care and the effectiveness of current diagnostic and therapeutic procedures has led to renewed interest in comparative clinical studies. In the field of radiology such studies are of particular interest to patients with cancer because they strongly influence treatment choices. Nonetheless, comparative data on competing imaging methods have traditionally been difficult to obtain for two reasons. First, few institutions have enough patients with cancer to perform a statistically reliable and generalized study. Second, until the Radiological Diagnostic Oncology Group was established, there was no quick and efficient way for individual institutions to pool data and thereby to enlarge samples.

Our results are disappointing. With currently available commercial equipment, neither ultrasonography nor MRI can reliably differentiate microscopical local from advanced disease in patients presenting with clinically localized disease. The overall accuracy of ultrasonography was 58 percent, and of MRI, 69 percent; both were more accurate in identifying patients with advanced disease. Ultrasonography accurately staged confined disease in only 49 percent of the patients, and MRI in 64 percent, whereas the accuracy of staging for advanced disease was 63 percent for ultrasonography and 71 percent for MRI. Moreover, making the results of the other imaging method available to the reader had no effect on the accuracy of either method. Although there was no significant difference between MRI and ultrasonography at a P level of 0.05, the performance of MRI was consistently better than that of ultrasonography in all dimensions measured (Tables 1 and 2).

Unfortunately, when these findings on the accuracy of the techniques were translated into implications for patient care, the results were equally disappointing. With either method, the disease in a significant number of cases was either incorrectly staged or classified as less serious than it actually was. For example, for every 100 patients with clinically operable disease (the entry criterion for the study) for whom ultrasonography indicated that the disease had extended beyond the prostate, only 63 would actually have had such extension. The comparable number for MRI was 71 of 100 patients. The initial hope that MRI would improve staging accuracy by allowing imaging not only of the local prostate area but also of the pelvic nodes was not substantiated. MRI and presumably other cross-sectional imaging techniques currently are not able to identify microscopic spread of disease. Although the specificity of MRI was high (96 percent), the relatively low percentage of patients with nodal involvement in this population made its predictive value low — about 12 percent of all nodes positive on MRI would be positive on pathological examination. Whether this percentage is too low to indicate a treatment strategy is unclear and would depend on a weighing of the side effects of biopsy against the benefits of the information thus obtained.

Several possible explanations can be offered for the marked differences between our results and those of earlier, more optimistic studies.^{7 8 9 10 11 12 13 14} First, all of our patients were thought, on the basis of the initial clinical examination, to have surgically resectable disease, thus minimizing the possibility of including patients with obvious advanced disease.²⁰ Second, our data were collected prospectively, and readers for one imaging examination were blinded to the results of the other as well as to the results of the pathological study. Third, the imaging techniques differed slightly from those other published reports.^{6 7 8 9 10 11 12} It is not clear to us why such changes in technique should have had such a profound effect on accuracy. Fourth, no cases were self-referred — that is, there was no overlap between detailed information obtained from the physical examination and the interpretation or performance of the imaging tests. Such an overlap could potentially have biased the results, although it is not entirely obvious in what direction the bias would have been. Finally, in previous studies the entire radical-prostatectomy specimens were not always "step-sectioned" to correlate exactly with imaging,^{10 , 12} which may mean that not all areas of microscopic tumor extension were identified. None of the imaging techniques available, including MRI, ultrasonography, and computed tomography, are able to identify microscopic disease. They can detect only macroscopic disease, whether or not there is an attempt to evaluate local disease, periprostatic or seminal-vesicle invasion, or lymph-node or distant spread of the tumor.

Another factor that may affect accuracy in this study and in others is the possibility of artifacts (hemorrhage or inflammation) due to an earlier biopsy. Theoretically, these would be expected to cause a decrease in specificity because scarring could be confused with tumor invasion. Thus, we believe that our results provide a minimal degree of accuracy that could be improved if imaging was performed before biopsy. Because of the expense of imaging studies, however, staging is usually not done until after the diagnosis of cancer is confirmed (e.g., after the biopsy). Possible pitfalls resulting from the use of biopsy will be a recurrent problem and probably will be unavoidable whatever the experimental design.

The data in Table 4 indicate that neither ultrasonography nor MRI can accurately detect and localize malignant tumors. Because this study relates to staging existing tumors, these data can offer only a lower limit of the sensitivity of these techniques for detection alone. It is generally believed that tumors do not spread beyond the confines of the prostate unless they reach a volume of at least 1.0 to 1.5 ml.^{21 , 22} Such a volume is consistent with data suggesting that cancer becomes aggressive when it attains an estimated mass of only 0.2 to 1.0 cm in diameter.^{21 , 23}

Several approaches might lead to improvements in the accuracy of ultrasonography and MRI. First, analysis of the images with special attention to diagnostic features may lead to improvements for both MRI and ultrasonography, as it has for mammography.²⁴ Second, for MRI, the use of noise reduction,²⁵ fat suppression,²⁶ and intrarectal surface coils^{27 , 28} may, as preliminary studies have suggested, be helpful and improve accuracy. Magnetic resonance spectroscopy may also be useful to determine the extent of prostate disease.^{29 30 31} The use of contrast agents in MRI may have potential benefits, and tissue characterization or diagnostic analysis may improve ultrasound results. Improved accuracy may permit the clinician to plan treatment with greater confidence than at present.

In terms of the methodologic results of this study, we suggest that the use of a study group like the Radiological Diagnostic Oncology Group is one way of providing data rapidly for use in decisions involving patient care. This particular group consisted initially of five institutions that presented preliminary data on nearly 100 patients in less than 12 months³² and on more than 200 patients in 15 months.

In summary, we believe that our results show that ultrasonography and MRI are not as accurate in staging early prostate disease as previous studies had reported. We also believe that the larger sample in this study as compared with earlier studies, its multi-institutional nature, and the use of a number of readers increase the likelihood that these conclusions can be generalized. Further refinements of these techniques should be supported because we believe that MRI and ultrasonography have the greatest potential in staging prostate cancer.