Join the 200th Anniversary Celebration

Original Article

The Diagnostic Performance of Computer Programs for the Interpretation of Electrocardiograms

Jos L. Willems, M.D., Ph.D., Cassiano Abreu-Lima, M.D., Pierre Arnaud, M.D., Jan H. van Bemmel, Ph.D., Christian Brohet, M.D., Rosanna Degani, M.Sc., Bernard Denis, M.D., Jürgen Gehring, M.D., Ian Graham, M.D., Gerard van Herpen, M.D., Hilario Machado, M.D., Peter W. Macfarlane, Ph.D., Jörg Michaelis, M.D., Spyridon D. Moulopoulos, M.D., Paul Rubel, Ph.D., and Christoph Zywietz, M.Sc.

N Engl J Med 1991; 325:1767-1773December 19, 1991

Abstract
Abstract

Background.

Computer programs for the interpretation of electrocardiograms (ECGs) are now widely used. However, a systematic assessment of various computer programs for the interpretation of ECGs has not been performed.

Methods.

We undertook a large international study to compare the performance of nine electrocardiographic computer programs with that of eight cardiologists in interpreting ECGs in 1220 clinically validated cases of various cardiac disorders. ECGs from the following groups were included in the sample: control patients (n = 382); patients with left ventricular hypertrophy (n = 183), right ventricular hypertrophy (n = 55), or biventricular hypertrophy (n = 53); patients with anterior myocardial infarction (n = 170), inferior myocardial infarction (n = 273), or combined myocardial infarction (n = 73); and patients with combined infarction and hypertrophy (n = 31). The interpretations of the computer programs and the cardiologists were compared with the clinical diagnoses made independently of the ECGs, and the computer interpretations were compared with those of the cardiologists.

Results.

The percentage of ECGs correctly classified by the computer programs (median, 91.3 percent) was lower than that for the cardiologists (median, 96.0 percent; P<0.01). The median sensitivity of the computer programs was also significantly lower than that of the cardiologists in diagnosing left ventricular hypertrophy (56.6 percent vs. 63.9 percent, P<0.02), right ventricular hypertrophy (31.8 percent vs. 46.6 percent, P<0.01), anterior myocardial infarction (77.1 percent vs. 84.9 percent, P<0.001), and inferior myocardial infarction (58.8 percent vs. 71.7 percent, P<0.0001). The median total accuracy level (the percentage of correct classifications) was 6.6 percent lower for the computer programs (69.7 percent) than for the cardiologists (76.3 percent; P<0.001). However, the performance of the best programs nearly matched that of the most accurate cardiologists.

Conclusions.

Our study shows that some but not all computer programs for the interpretation of ECGs perform almost as well as cardiologists in identifying seven major cardiac disorders. (N Engl J Med 1991;325: 1767–73.)

Article

IT is estimated that over 50 percent of the 100 million electrocardiograms (ECGs) recorded annually in the United States are interpreted with use of a computer.1 Although many studies have evaluated such computer programs,2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 systematic quality assessment has not been undertaken. Previous investigations have been limited mainly to comparisons of a few computer programs. In addition, the studies are difficult to compare, since they used different sets of data, most of them collected at a single institution.

A variety of criteria are needed to evaluate computer programs designed to interpret ECGs.22 From the clinical standpoint, diagnostic accuracy is the most important issue.4 By 1978, it was already clear that the lack of objective, common reference data made it difficult to compare the relative merits and shortcomings of various computer programs.12 The collection of an independent, representative test library of validated ECG recordings was therefore highly recommended. This goal has only recently been accomplished.32

We undertook to compare the diagnostic results obtained by nine computer programs designed to interpret ECGs and eight cardiologists who analyzed the same ECGs. The study was restricted to ECGs from patients with one of seven main diagnoses that could be validated on the basis of nonelectrocardiographic evidence, such as the results of cardiac catheterization, echocardiography, and cardiac-enzyme measurements. Evaluation of rhythm and conduction disturbances, for which the ECG itself is the standard, was not performed; we also did not assess agreement on ST-segment and T-wave changes or other descriptive statements.12

Methods

This study was initiated in 1985.32 Cases were selected entirely on the basis of clinical information unrelated to electrocardiographic findings.33 34 35 Data were collected and ECGs obtained from 1220 adults (831 men and 389 women, all white) with a mean (±SD) age of 52± 13 years.

Patients

As controls (n = 382), we used 286 ambulatory patients selected on the basis of their medical history and the results of the physical examination, chest radiography, and in some cases echocardiography (ambulatory controls) and 96 patients referred for cardiologic examination who were found to have no organic cardiac disease (catheterized controls). Most of the catheterized group underwent coronary arteriography for the evaluation of atypical chest pain or ST-segment or T-wave abnormalities on the ECG obtained at rest or after exercise. A few underwent right-heart catheterization for suspected valvular heart disease.

The diagnosis of left ventricular hypertrophy (n = 183) was based on findings of cardiac catheterization, echocardiography, or both. Patients with this diagnosis had long-standing hypertension, acquired left-sided valvular heart disease, or congenital heart disease. Patients with primary cardiomyopathy were excluded. Left ventricular hypertrophy was diagnosed primarily on the basis of left ventricular pressure overload (peak aortic valve gradient ≥50 mm Hg) or volume overload (in 115 cases). Echocardiograms, obtained for 85 percent of these patients, supported the diagnosis. In another 68 patients, all with hypertension, the diagnosis was based only on echocardiographic evidence of an increased leftventricular-mass index (the ratio of the left ventricular mass in grams to the body-surface area in square meters), defined as an index above 110 g per square meter for women and above 134 g per square meter for men. The diagnosis of right ventricular hypertrophy is difficult, but a minimal requirement was clear-cut clinical evidence of conditions expected to cause this abnormality. The patients with right ventricular hypertrophy (n = 55) had acquired or congenital right ventricular pressure overload or volume overload, primary or secondary pulmonary hypertension (mean pulmonary pressure ≥25 mm Hg), or a left-to-right shunt in the atria of more than 1.5:1. For the diagnosis of biventricular hypertrophy (n = 53), a ventricular septal defect with a left-to-right shunt greater than 1.5:1 or hemodynamic and echocardiographic measurements pointing to both left and right ventricular hypertrophy was required.

Akinesia or dyskinesia in seven different segments of the left ventriculogram, coded according to an American Heart Association reporting system,36 was the main criterion for the various types of infarction. Patients with hypokinesia in one or more wall segments together with an occluded or recanalized supply vessel and a history typical of infarction were also included. All patients had ventriculography in two orthogonal projections. Patients who had evidence of akinesia or dyskinesia in the anterolateral, anterobasal, or septal region, but not in the inferobasal, inferolateral, or true posterior wall segments, were classified as having anterior myocardial infarction (n = 170). Those with inferior infarction had evidence of akinesia or dyskinesia only in the inferobasal, inferolateral, diaphragmatic, or true posterior wall segments (n = 273). This group included nine patients with posterior akinesia and normal contraction or hypokinesia in the inferior segments. Patients who met the criteria for both anterior and inferior infarction, as well as those with isolated apical infarction, were assigned to the combined-infarction group (n = 73). Thirty-one patients with both infarction and hypertrophy were studied as a separate group.

Data Collection and Processing of ECGs

The electrocardiographs data were collected in five European centers. All standard ECG leads and the orthogonal X, Y, and Z leads were recorded simultaneously, with a sampling rate of 500 Hz. Three cardiologists checked the clinical information; a consensus was required for case selection. ECGs showing complete left or right bundle-branch block or other major intraventricular conduction defects37 and those of poor technical quality were excluded. There were no other criteria for excluding ECGs.

The ECGs selected were analyzed by eight cardiologists from seven European countries and by nine computer programs. The programs we used were the following: Marquette Electronics (Program 2), Hannover (Program 4), Hewlett-Packard (Program 5), Medis (Program 7), Nagoya (Program 8), Glasgow (Program 11), Padova (Program 13), Means (Program 15), and Leuven (Program 16). A description of these programs and their basic methods has been published recently.33 The programs developed in Hannover and Leuven use a statistical approach to classification. All others apply heuristic or deterministic methods.33 All interpretations were made in the processing centers between March and May 1990 with the latest version of each program available for routine use. Except for the age and sex of the patients, no clinical data were entered into the programs or provided to the cardiologists.

The processing centers and the cardiologists were asked to apply a scheme for translating statements into a common set of diagnostic codes, as described previously.34 Each statement could be qualified as definite, probable, or possible. A combined result based on the interpretations of all the cardiologists was derived by means of weighted averaging.35 For every diagnostic statement, each occurrence of "definite" was assigned a value of 3 points, "probable" 2 points, and "possible" 1 point. For each ECG, the scores for all the cardiologists were averaged. A combined result between 1 and 3 was thus obtained for each diagnostic category and reconverted into one of the three levels of certainty. This result is equivalent to that obtained by panel review or a majority vote of several readers. A combined result was similarly obtained for the programs. The average cardiologist was defined as the one whose total accuracy level was closest to the group average. The average program was similarly defined.

Statistical Analysis

The results of the interpretations by the computer programs and the cardiologists were compared with the diagnoses based on the independent clinical data. The programs' results were also compared with the results obtained by the average cardiologist and with the combined cardiologists' results (the group interpretation). Different classification matrixes for combinations of two or more interpretations were calculated34 , 35 (tables showing detailed results are available elsewhere*). If more than one diagnosis was supplied for a particular ECG, only that assigned the highest level of certainty was used in the analysis. The numbers in the corresponding cells of the matrix were increased by 1/2 if a program provided two statements with equal certainty for a case with a single abnormality. Similarly, when three statements were provided at the highest level with the same degree of certainty, the numbers were increased by 1/3. In this way, allowance was made for the fact that programs may provide varying numbers of diagnostic statements. For the patients with biventricular hypertrophy, combined infarction, or hypertrophy and infarction, diagnoses with lower levels of certainty were taken into consideration.34

Standard formulas,38 , 39 in which TP represents a true positive result, FN a false negative result, TN a true negative result, and FP a false positive result, were used to calculate sensitivity (TP/[TP + FN]), specificity (TN/[TN + FP]), positive predictive value (TP/[TP + FP]), and negative predictive value (TN/ [TN + FN]), of the diagnoses of all cardiologists and programs for each diagnostic group. The total accuracy level (the percentage of correct classifications) was calculated for the entire sample on the basis of the diagonal elements of the seven-by-seven misclassification matrixes.34 , 35 Wilcoxon's signed-rank test39 and Mc-Nemar's test40 , 41 were used to evaluate differences in sensitivity and total accuracy for paired data. The intraobserver variability for the cardiologists was calculated on the basis of 125 selected ECGs that were read twice. Interobserver variability was assessed for the total sample with use of kappa statistics.42 , 43 All P values are two-tailed, and P values greater than 0.05 were considered not significant.

Results

Cardiologists' Results as Compared with the Clinical Diagnoses

The cardiologists correctly classified the ECGs for the control patients in 92.7 to 97.6 percent of the cases (median, 96.0 percent) (Table 1Table 1Sensitivity of Eight Cardiologists and Nine Computer Programs and Total Accuracy Level in Identifying Major Cardiac Disorders.). The median was higher (P<0.01) for the ambulatory control group (97.9 percent) than for the catheterized control group (90.1 percent). The cardiologists' sensitivity for left ventricular hypertrophy (median, 63.9 percent; range, 54.8 to 69.3 percent) was significantly higher than that for right ventricular hypertrophy (median, 46.6 percent; range, 40.6 to 51.5 percent; P<0.03), but the corresponding number of false positive results was also higher (Fig. 1Figure 1Sensitivity and Specificity of the Computer Programs and the Cardiologists in Diagnosing Right Ventricular Hypertrophy (RVH) and Left Ventricular Hypertrophy (LVH).). The cardiologists' sensitivity for anterior myocardial infarction (median, 84.9 percent; range, 79.0 to 87.5 percent) was significantly higher than that for inferior infarction (median, 71.7 percent; range, 59.2 to 84.1 percent; P<0.001), whereas the corresponding specificities were almost equal (Fig. 2Figure 2Sensitivity and Specificity of the Computer Programs and the Cardiologists in Diagnosing Anterior Myocardial Infarction (AMI) and Inferior Myocardial Infarction (IMI).). The cardiologists' median sensitivity for biventricular hypertrophy was 42.7 percent (range, 36.8 to 70.8 percent), and that for combined myocardial infarction was 67.5 percent (range, 58.9 to 75.3 percent). Their total level of accuracy ranged between 72.6 percent and 81.0 percent (median, 76.3 percent) (Table 1).

Program Results as Compared with the Clinical Diagnoses

The ECG-reading programs correctly classified the controls in 86.3 to 97.1 percent of the cases (median, 91.3 percent) (Table 1). The median was 92.6 percent for the ambulatory controls and 87.4 percent for the catheterized controls (P not significant). The programs' sensitivity in diagnosing left ventricular hypertrophy (median, 56.6 percent; range, 50.3 to 76.2 percent) was significantly higher than for right ventricular hypertrophy (median, 31.8 percent; range, 14.5 to 52.8 percent; P<0.001) (Fig. 1). The programs' sensitivity for anterior myocardial infarction was also higher (median, 77.1 percent; range, 58.8 to 81.5 percent) than that for inferior infarction (median, 58.8 percent; range, 38.7 to 82.8 percent; P<0.001) (Fig. 2). The median sensitivity for biventricular hypertrophy was 31.8 percent, and that for combined myocardial infarction was 59.2 percent. The false positive rates were significantly higher (P<0.05) for left ventricular hypertrophy, anterior infarction, and inferior infarction (the 95 percent confidence intervals of the medians ranged from 1.9 percent to 5.6 percent) than for the other groups, for which the upper 95 percent confidence limit was 1.8 percent. The false positive rates of some programs were considerably outside these limits (Fig. 1 and 2). Table 1 shows that the total level of accuracy of the programs ranged from 62.0 percent to 77.3 percent (median, 69.7 percent). The accuracy level of the two statistically based programs (75.8 percent and 77.3 percent) was significantly higher (P<0.001) than that of the deterministic programs (median, 69.3 percent; range, 62.0 to 69.8 percent). The programs with the best performance reached levels of sensitivity and specificity similar to those reached by the best cardiologists (Fig. 1 and 2).

Program Results as Compared with the Cardiologists' Results

The Means program (Program 15) and all the programs combined (combined result) correctly diagnosed significantly more of the 382 controls than the average cardiologist, whereas for three programs the difference from the average cardiologist's results was not significant (Table 2Table 2Performance of Computer Programs and the Average Cardiologist as Compared with the Clinical Evidence). For the 838 abnormal ECGs, two programs (Hannover [Program 4] and Leuven [Program 16]) had a higher sensitivity than the average cardiologist, whereas the combined program results were not significantly different from those of the average cardiologist. Both the average cardiologist and the average program were correct in 52.5 percent and incorrect in 26.5 percent of the cases. The cardiologist was correct and the program incorrect in 14.7 percent of the cases, and the opposite was true in 6.3 percent (P<0.0001). The total percentage of correct diagnoses made by the average cardiologist (75.5 percent [921 of 1220]) was 5.8 percent higher than for the average program (69.7 percent [850 of 1220], P<0.001).

When the interpretations made by the computer programs were compared with the combined results of the cardiologists, the two agreed in 68.1 to 80.3 percent of the cases (median, 76.6 percent) (Fig. 3Figure 3Total Accuracy Level (Percentage of Correct Classifications) of the Individual Computer Programs and Cardiologists and Their Degree of Agreement with the Combined Results of the Cardiologists.). The difference between the highest and the lowest rates of agreement was statistically significant (P<0.01). The combined program results were the same as the combined cardiologists' results in 85.4 percent of the cases (95 percent confidence interval, 83 percent to 87 percent). This degree of concordance was significantly higher (P<0.001) than for any of the programs individually.

Individual cardiologists agreed with their own combined group result in 81.4 to 88.1 percent of the cases (median, 84.6 percent) (Fig. 3). The interobserver variability among the cardiologists was significantly less (P<0.001) than the variability among all the programs, with average kappa scores of 0.67±0.09 (n = 28) as compared with 0.51 ±0.11 (n = 36). The average kappa score for only the deterministic programs was 0.59±0.08 (n = 21). The intraobserver reproducibility of the cardiologists' diagnoses ranged from 76.8 percent to 90.4 percent (median, 81.8 percent) for the 125 selected ECGs that were read twice by each cardiologist.

Discussion

The primary objective of this study was to assess the performance of computer programs and cardiologists in diagnosing seven clinical conditions on the basis of ECGs recorded in patients whose diagnoses were confirmed by independent clinical data. This approach has been advocated strongly by several investigators.4 , 9 , 12 , 17 , 19 , 20 , 25 , 44 Because there is no single comprehensive method of evaluating the performance of computer programs,7 , 17 , 39 , 45 a variety of tests should be used.20 , 39 The total level of accuracy and other indexes of diagnostic precision, such as the positive and negative predictive values and the utility index,7 depend heavily on the composition of the study population represented by the sample ECGs and the proportion of patients in each group. Thus, although the total accuracy level provides a single number for use in evaluating diagnostic performance, it is often too simple and must be interpreted with caution. The predictive merits of a program should therefore be examined for each diagnostic category and in relation to different rates of prevalence.39

The results of our study demonstrate that the computer programs with the best performance are almost as accurate as the best cardiologists in classifying the ECGs of patients in seven common diagnostic groups. However, all the programs we studied can still be improved. Most cardiologists had a better performance than the programs in confirming normality and a higher sensitivity in diagnosing anterior myocardial infarction.

Our results demonstrate that, in some cases, computer interpretations of ECGs may be correct and cardiologists incorrect when their diagnoses are tested against the clinical evidence, although the opposite was twice as likely (14.7 percent vs. 6.3 percent) when the average computer program and the average cardiologist were compared. Whether the average ECG reader attains the same degree of accuracy in the ordinary clinical environment as the average cardiologist in this study is subject to debate. Six of the eight cardiologists worked in university hospitals, and all were highly motivated by their participation in the study. Hence, it is reasonable to assume that their performance was above average. All were Europeans, but because cardiologists from different centers were included, the study was not restricted to a particular school of electrocardiographic interpretation. We found that combining different programs' interpretations of ECGs can increase diagnostic accuracy, as in a panel review.35 This technique offers possibilities for further improvement of computerized electrocardiography. The degree of variability in diagnostic accuracy among the computer programs was significantly greater than the variability among the cardiologists. However, the degree of variability among the deterministic programs was on the same order as that for the cardiologists. Individual cardiologists gave different diagnoses for 10 to 23 percent of the ECGs they read twice, whereas computer programs are inherently without "intraobserver" variability. The rates of intraobserver and interobserver variability are determined largely not by the clear-cut cases but by the ECGs indicating borderline and difficult diagnoses.

Comparative evaluation of computer programs presents several problems. Almost all programs use different diagnostic terminology. Some generate only one or two diagnostic statements, whereas others provide a list of the most likely interpretations, in decreasing order of likelihood. Some use qualifiers such as "definite," "probable," and "possible," whereas others use probabilities.34 , 46 The methods used to convert the statements from the various programs into a common code (mapping), even when applied consistently, may not have allowed for all the differences in reporting. The fact that 87 percent of the data base consisted of patients with single diseases may have favored programs that use a Bayesian statistical approach. Since these programs by definition select from one of seven or eight diagnostic categories, they have a higher likelihood than other programs of classifying cases with ischemic ST—T changes, but no pathologic Q waves, as myocardial infarction. The cardiologists could also have been biased because of their knowledge of the composition of the data base, and even the designers of the computer programs could have introduced bias into mapping of their program statements. More attention might have been given to the selection of cases of differing severity, and for some diagnoses only a limited number of ECGs were available. It is also possible that patients had an increase in ventricular pressure without having ventricular hypertrophy. On the other hand, it is quite probable that some patients with myocardial infarction may have also had left ventricular hypertrophy. The magnitude and importance of these potential biases cannot be estimated, but there is no reason to believe that their effect was greater than in previous studies.26 , 44 , 45

It must be reiterated that this study did not measure some of the most important aspects of electrocardiography — for instance, the diagnosis of acute cardiac ischemia, which is best made by analyzing ST-segment and T-wave abnormalities, and the diagnosis of arrhythmias, an area in which computer programs can make serious errors.8 The review of computerized ECG reports by a physician is therefore still essential.

These results demonstrate that standard ECGreading computer programs can assist clinicians in achieving more uniform and consistent interpretations of ECGs.47 However, some programs perform at a considerably lower level than cardiologists and require improvement. Continued testing and refinement of such computer programs is necessary to enhance their performance.4 , 33

Supported by Directorate General XII of the European Commission as part of its third and fourth Medical and Public Health Research Programmes.

*Deceased.

Source Information

From the Division of Medical Informatics, University of Leuven, Belgium (J.L.W.); the Department of Cardiology, University of Porto, Portugal (C.A.-L.); Institut National de la Santé et de la Recherche Médicale (INSERM) Unité 121, Lyon, France (P.A.); the Department of Medical Informatics, Erasmus University, Rotterdam, the Netherlands (J.H.v.B.); the Department of Cardiology, Université de Louvain, Brussels, Belgium (C.B.); Research Institute for System Dynamics and Bioengineering (LADSEB), National Research Council of Italy, Padua (R.D.); the Clinique Cardiologique, Université de Grenoble, France (B.D.); Abteilung Kardiologie, Klinik Höhenried, Bernried, Germany (J.G.); the Department of Cardiology, Adelaide and Meath Hospitals, Dublin, Ireland (I.G.); the Department of Cardiology, Leyden University, the Netherlands (G.v.H.); the Department of Cardiology, University Hospital Santa Maria of Lisbon, Portugal (H.M.); the Department of Medical Cardiology, Glasgow University, Scotland (P.W.M.); the Institut für Medizinische Statistik, Johannes Gütenberg Universität, Mainz, Germany (J.M.); the Department of Medicine, University of Athens, Greece (S.D.M.); the National Institute of Applied Sciences, Lyon, France (P.R.); and the Division of Biosignal Processing, Medical School of Hannover, Germany (C.Z.). Address reprint requests to Dr. Willems at University Hospital Gasthuisberg, 49 Herestraat, 3000 Leuven, Belgium.

Appendix

The investigators and institutions participating in this study were as follows: European Working Party on Common Standards fir Quantitative Electrocardiography (CSE): Chr. Brohet, Université de Louvain, Brussels, and J.L. Willems, University of Leuven, Belgium; J. Gehring, Cardiological Clinic, Höhenried, J. Meyer and J. Michaelis, University of Mainz, S.J. Pöppl, Institute of Medical Data Processing, Munich, and Chr. Zywietz, University of Hannover, Germany; J. Damgaard Andersen, University of Copenhagen, Denmark; P. Arnaud, INSERM U121, Lyon, B. Denis, University of Grenoble, and P. Rubel, Institut National des Sciences Appliquées, Lyon, France; PJ. Bourdillon, University of London, and P.W. Macfarlane, University of Glasgow, United Kingdom; S. Moulopoulos, University of Athens, and E. Skordalakis, Technical University of Athens, Greece; I. Graham, University of Dublin, Ireland; S. Dalla Volta, University of Padua, R. Degani, Research Institute for System Dynamics and Bioengineering (LADSEB), National Research Council of Italy, Padua, and G. Mazzocca, University of Pisa, Italy; J.H. van Bemmel and J.A. Kors, Erasmus University, Rotterdam, H.J. Ritsema van Eck, Rotterdam, and G. van Herpen, University of Leiden, the Netherlands; and C. Abreu-Lima, University of Porto, and H.B. Machado, University Hospital of Lisbon, Portugal; Consultants: J.J. Bailey, National Institutes of Health; H.V. Pipberger, George Washington University; P.M. Rautaharju, Dalhousie University; and E. Robles de Medina, University of Utrecht, the Netherlands; Non-European Participants: C. Monroe and S. Charlesworth, Hewlett-Packard, and I. Rowlandson, Marquette Electronics, United States; P.M. Rautaharju and P. Macinnis, Dalhousie University, Halifax, N.S., Canada; and M. Okajima, N. Okamoto, and M. Yokoi, University of Nagoya, and M. Ohsawa, Fukuda Denshi, Japan; Representatives of the Biomedical Engineering Concerted Action Committee of the European Community: J. Marquez-Montes, University of Madrid, Spain; and U. Faust, University of Stuttgart, Germany; CSE Steering Committee: J.L. Willems (project leader), P. Arnaud, R. Degani, P.W. Macfarlane, J.H. van Bemmel, and Chr. Zywietz; CSE Board of Referees: C. Abreu-Lima, P. Arnaud, Chr. Brohet, B. Denis, I. Graham, J. Gehring, H. Machado, J. Michaelis, S.D. Moulopoulos, and G. van Herpen; CSE Coordinating Center: Division of Medical Informatics, University of Leuven, Belgium.

References

References

  1. *

    See NAPS document no. 04914 for six pages of supplementary material. Order from NAPS c/o Microfiche Publications, P.O. Box 3513, Grand Central Station, New York, NY 10163–3513. Remit in advance (in U.S. funds only) $7.75 for photocopies or $5 for microfiche. Outside the U.S. and Canada add postage of $4.50 ($1.75 for microfiche postage). There is a $15 invoicing charge on all orders filled before payment.

  2. 1

    Drazen E, Mann N, Borun R, Laks M, Berson A. Survey of computer assisted electrocardiography in the United States . J Electrocardiol 1988;21: Suppl:S98–S104.
    CrossRef | Web of Science | Medline

  3. 2

    Bruce RA, Yarnall SR. Reliability and normal variations of computer analysis of Frank electrocardiograms by the Smith-Hyde program (1968 version) . Am J Cardiol 1972;29:389–96.
    CrossRef | Web of Science | Medline

  4. 3

    Caceres CA, Hochberg HM. Performance of the computer and physician in the analysis of the electrocardiogram . Am Heart J 1970;79:439–43.
    CrossRef | Web of Science | Medline

  5. 4

    Pipberger HV, Cornfield J. What ECG-computer program to choose for clinical application: the need for consumer protection . Circulation 1973; 47:918–20.
    Web of Science | Medline

  6. 5

    Bailey JJ, Itscoitz SB, Hirshfield JW Jr, Grauer LE, Horton MB. A method for evaluating computer programs for electrocardiographic interpretation. I. Application to the experimental IBM program of 1971 . Circulation 1974; 50:73–9.
    Web of Science | Medline

  7. 6

    Meyer J, Heinrich KW, Merx W, Effert S. Computeranalyse des Elektrocardiograms mit verschiedenen Programmen . Dtsch Med Wochenschr 1974; 23:1213–23.
    CrossRef

  8. 7

    Smets P, Willems JL, Talmon J, De Maertelaer V, Kornreich F. Methodology for the comparison of various diagnostic procedures . Biom Praximetrie 1975;15:3–35.

  9. 8

    How reliable is the computerized electrocardiographic interpretation? JAMA 1975;231:1090.

  10. 9

    Pipberger HV. Comparative evaluation of ECG computer programs. In: Ostrow HG, Ripley KL, eds. Computers in cardiology. Long Beach, Calif.: IEEE Computer Society, 1976:85–8.

  11. 10

    Willems JL, Piessens J, Ector J, Kesteloot H, De Geest H. Comparison of computer and visual ECG analysis results: correlation with independent cineangiographic data. In: Ostrow HG, Ripley KL, eds. Computers in cardiology. Long Beach, Calif.: IEEE Computer Society, 1976:89–94.

  12. 11

    Feruglio GA, Feraco E, Mirolo R. Experience of a user of an ECG interpretation system. In: van Bemmel JH, Willems JL, eds. Trends in computer-processed electrocardiograms. Amsterdam: North-Holland, 1977: 239–47.

  13. 12

    Rautaharju PM, Ariet M, Pryor TA, et al. The quest for optimal electrocardiography. Task Force III: computers in diagnostic electrocardiography . Am J Cardiol 1978;41:158–70.
    CrossRef | Web of Science | Medline

  14. 13

    Bourdillon PJ, Kilpatrick D. Clinicians, the Mount Sinai program and the Veterans' Administration program evaluated against clinicopathological data derived independently of the electrocardiogram . Eur J Cardiol 1978; 8:395–412.
    Medline

  15. 14

    Rautaharju PM, Smets P. Evaluation of computer-ECG programs: the strange case of the Golden Standard . Comput Biomed Res 1979;12:3946.
    CrossRef

  16. 15

    Bonner R, Jackson LK, Hooper JK, Smith CH, Hsieh RC. A method for testing performance in electrocardiographic interpretation. In: Pryor AT, Bailey JJ, eds. Computerized interpretation of the ECG IV. New York: Engineering Foundation, 1979:126–44.

  17. 16

    Brohet C, Richman HG. Clinical evaluation of automated processing of electrocardiograms by the Veterans Administration Program (AVA 3.4) . Am J Cardiol 1979;43:1167–74.
    CrossRef | Web of Science | Medline

  18. 17

    Bailey JJ, Harris EK. Evaluation of ECG interpretation: truth versus beauty. In: Tolan GD, Pryor TA, eds. Computerized interpretation of the electrocardiogram V. New York: Engineering Foundation, 1980:179–82.

  19. 18

    Macfarlane PW, Melville DI, Horton MR, Bailey JJ. Comparative evaluation of the IBM (12-lead) and Royal Infirmary (orthogonal three-lead) ECG computer programs . Circulation 1981;63:354–9.
    CrossRef | Web of Science | Medline

  20. 19

    Laks MM. "Gold standard" for ECG diagnosis — revisited 1981. In: Bonner RE, Pryor TA, Laks MM, Cole SS, eds. Computerized interpretation of the electrocardiogram VI. New York: Engineering Foundation, 1982:26775.

  21. 20

    Bailey JJ, Berson A, Jackson L, et al. Evaluation methodologies for ECG diagnostic systems. In: Bonner RE, Pryor TA, Laks MM, Cole SS, eds. Computerized interpretation of the electrocardiogram VI. New York: Engineering Foundation, 1981:53–62.

  22. 21

    Wolf HK, Milliken JA, Gregor RD, et al. Accuracy of type "A" statements of human and computer ECG classifiers. In: Laks MM, Cole SS, eds. Computerized interpretation of the electrocardiogram VII. New York: Engineering Foundation, 1982:335–40.

  23. 22

    Bernard P, Chaitman BR, Scholl JM, Val PG, Chabot M. Comparative diagnostic performance of the Telemed computer ECG program . J Electrocardiol 1983;16:97–103.
    CrossRef | Web of Science | Medline

  24. 23

    Graham IM, Reardon B, Mulcahy D, Mulcahy R. User evaluation of a commercially available computerized electrocardiographic interpretation system. In: Willems JL, van Bemmel JH, Zywietz C, eds. Computer ECG analysis: towards standardisation. Amsterdam: North-Holland, 1986:191–3.

  25. 24

    Willems JL. The CSE project: goals and achievements. In: Willems JL, van Bemmel JH, Zywietz C, eds. Computer ECG analysis: towards standardisation. Amsterdam: North-Holland, 1986:11–21.

  26. 25

    Pipberger HV. ECG computer analysis: past, present and future. In: Willems JL, van Bemmel JH, Zywietz C, eds. Computer ECG analysis: towards standardisation. Amsterdam: North-Holland, 1986:3–10.

  27. 26

    Harris LK, Blackburn HW, Prineas RJ, Wolf HK. Electrocardiographic diagnosis of myocardial infarction by human readers and two computer programs. In: European Congress on Cardiology, August 20—24, 1976, Paris. Abstract book I. Paris: European Society of Cardiology, 1976:725. abstract.

  28. 27

    Richman HG, Brohet CR. Evaluation of a statistical decision program for automated ECG processing. In: Hofman I, Hamby RE, eds. Vectorcardiography 3. Amsterdam: North-Holland, 1976:199–207.

  29. 28

    Kornreich F, Block P, Bourgain R, et al. Multigroup diagnostic classification with a new "maximal" lead system . Adv Cardiol 1976;16:121–7.
    Web of Science | Medline

  30. 29

    Khadr NE, Bray CL, Beton DC, et al. Diagnosis of left ventricular hypertrophy and myocardial infarction by Bonner/IBM Program verified by ECGindependent evidence. In: Ripley KL, Ostrow HG, eds. Computers in cardiology. Long Beach, Calif.: IEEE Computer Society, 1979:93–7.

  31. 30

    Willems JL, Lesaffre E, Pardaens J, DeSchreye D. Multigroup logistic classification of the 12- and 3-lead ECG. In: Willems JL, van Bemmel JH, Zywietz C, eds. Computer ECG analysis: towards standardisation. Amsterdam: North-Holland, 1986:203–10.

  32. 31

    Willems JL, Lesaffre E, Pardaens J. Comparison of the classification ability of the electrocardiogram and vectorcardiogram . Am J Cardiol 1987;59:119–24.
    CrossRef | Web of Science | Medline

  33. 32

    Willems JL, Arnaud P, van Bemmel JH, Degani R, Macfarlane PW, Zywietz C. Common standards for quantitative electrocardiography: goals and main results . Methods Inf Med 1990;20:263–71.

  34. 33

    van Bemmel JH, Willems JL. Standardization and validation of medical decision-support systems: the CSE project . Methods Inf Med 1990;29:261–2.
    Web of Science | Medline

  35. 34

    Willems JL. Common standards for quantitative electrocardiography — 6th CSE progress report. Leuven, Belgium: Acco, 1986.

  36. 35

    Willems JL, Abreu-Lima C, Arnaud P, et al. Effect of combining electrocardiographic interpretation results on diagnostic accuracy . Eur Heart J 1988;9:1348–55.
    Web of Science | Medline

  37. 36

    Austen WG, Edwards JE, Frye RL, et al. A reporting system on patients evaluated for coronary artery disease. AHA Committee Report . Circulation 1975;51:Suppl:l-40.

  38. 37

    Willems JL, Robles de Medina EO, Bernard R, et al. Criteria for intraventricular conduction disturbances and pre-excitation . J Am Coll Cardiol 1985;5:1261–75.
    CrossRef | Web of Science | Medline

  39. 38

    Feinstein AR. Clinical biostatistics. St. Louis: C.V. Mosby, 1977:214.

  40. 39

    Michaelis J, Wellek S, Willems JL. Reference standards for software evaluation . Methods Inf Med 1990;29:289–97.
    Web of Science | Medline

  41. 40

    Bailey JJ, Campbell G, Horton MR, Shrager RI, Willems JL. Determination of statistically significant differences in the performance of ECG diagnostic algorithms: an improved method . J Electrocardiol 1988;21:Suppl:S188–S192.
    CrossRef | Web of Science | Medline

  42. 41

    Willems JL, Campbell G, Bailey JJ. Progress on the CSE diagnostic study: application of McNemar's test revisited . J Electrocardiol 1989;22:Suppl: 135–40.
    CrossRef | Web of Science | Medline

  43. 42

    Cohen J. A coefficient of agreement for nominal scales . Educ Psychol Meas 1960;20:37–46.
    CrossRef | Web of Science

  44. 43

    Landis JR. Koch GG. The measurement of observer agreement for categorical data . Biometrics 1977;33:159–74.
    CrossRef | Web of Science | Medline

  45. 44

    Simonson E, Tuna N, Okamoto N, Toshima H. Diagnostic accuracy of the vectorcardiogram and electrocardiogram: a cooperative study . Am J Cardiol 1966;17:829–78.
    CrossRef | Web of Science

  46. 45

    Milliken JA, Burggraf G, McCans J, Gregor RD, Wolf HK, Rautaharju PM. Evaluating ECG interpreters against expert majority and clinical data. In: Laks M, ed. Computerized interpretation of the electrocardiogram VII. New York: Engineering Foundation, 1982:341–6.

  47. 46

    Willems JL. Common standards for quantitative electrocardiography —10th CSE progress report. Leuven, Belgium: Acco, 1990.

  48. 47

    Milliken JA, Burggraf GW, McCans JL. The impact of the computer-reported vectorcardiogram on the cardiologist interpreter of the scalar electrocardiogram . J Electrocardiol 1977;10:375–8.
    CrossRef | Web of Science | Medline

Citing Articles (73)

Citing Articles

  1. 1

    Roy J. Shephard, Yukitoshi Aoyagi. (2011) Measurement of human energy expenditure, with particular reference to field studies: an historical perspective. European Journal of Applied Physiology
    CrossRef

  2. 2

    Till Ittermann, Charlotte van Noord, Nele Friedrich, Marcus Dörr, Stephan B. Felix, Matthias Nauck, Henry Völzke, Albert Hofman, Jacqueline C.M. Witteman, Bruno HCh Stricker, Henri Wallaschofski. (2011) The association between insulin-like growth factor-I and cardiac repolarization. Growth Hormone & IGF Research
    CrossRef

  3. 3

    Allison C. Hill, Christina Y. Miyake, Stafford Grady, Anne M. Dubin. (2011) Accuracy of Interpretation of Preparticipation Screening Electrocardiograms. The Journal of Pediatrics 159:5, 783-788
    CrossRef

  4. 4

    Luis Rodríguez-Padial, Blanca Rodríguez-Picón, Miguel Jerez-Valero, Julio Casares-Medrano, Finn O. Akerström, Alberto Calderon, Vivencio Barrios, Antonio Sarría-Santamera, José R. González-Juanatey, Antonio Coca, Josep Andrés, Jessica Ruiz-Baena. (2011) Precisión diagnóstica del electrocardiograma asistido por ordenador al diagnosticar hipertrofia ventricular izquierda en el bloqueo de rama izquierda. Revista Española de Cardiología
    CrossRef

  5. 5

    Luis Rodríguez-Padial, Blanca Rodríguez-Picón, Miguel Jerez-Valero, Julio Casares-Medrano, Finn O. Akerström, Alberto Calderon, Vivencio Barrios, Antonio Sarría-Santamera, José R. González-Juanatey, Antonio Coca, Josep Andrés, Jessica Ruiz-Baena. (2011) Diagnostic Accuracy of Computer-Assisted Electrocardiography in the Diagnosis of Left Ventricular Hypertrophy in Left Bundle Branch Block. Revista Española de Cardiología (English Edition)
    CrossRef

  6. 6

    Hui Yang, Satish T.S. Bukkapatnam, Trung Le, Ranga Komanduri. (2011) Identification of myocardial infarction (MI) using spatio-temporal heart dynamics. Medical Engineering & Physics
    CrossRef

  7. 7

    Jonathan J. Halford, William B. Pressly, Selim R. Benbadis, William O. Tatum, Robert P. Turner, Amir Arain, Paul B. Pritchard, Jonathan C. Edwards, Brian C. Dean. (2011) Web-Based Collection of Expert Opinion on Routine Scalp EEG: Software Development and Interrater Reliability. Journal of Clinical Neurophysiology 28:2, 178-184
    CrossRef

  8. 8

    Hui Yang. (2011) Multiscale Recurrence Quantification Analysis of Spatial Cardiac Vectorcardiogram Signals. IEEE Transactions on Biomedical Engineering 58:2, 339-347
    CrossRef

  9. 9

    J Fayn. (2011) A Classification Tree Approach for Cardiac Ischemia Detection Using Spatiotemporal Information From Three Standard ECG Leads. IEEE Transactions on Biomedical Engineering 58:1, 95-102
    CrossRef

  10. 10

    D. S. TRACHSEL, C. BITSCHNAU, N. WALDERN, M. A. WEISHAUPT, C. C. SCHWARZWALD. (2010) Observer agreement for detection of cardiac arrhythmias on telemetric ECG recordings obtained at rest, during and after exercise in 10 Warmblood horses. Equine Veterinary Journal 42, 208-215
    CrossRef

  11. 11

    Leo Bossaert, Robert E. O’Connor, Hans-Richard Arntz, Steven C. Brooks, Deborah Diercks, Gilson Feitosa-Filho, Jerry P. Nolan, Terry L. Vanden Hoek, Darren L. Walters, Aaron Wong, Michelle Welsford, Karen Woolfrey. (2010) Part 9: Acute coronary syndromes. Resuscitation 81:1, e175-e212
    CrossRef

  12. 12

    Charlotte A. Schreurs, Annemijn M. Algra, Sum-Che Man, Suzanne C. Cannegieter, Ernst E. van der Wall, Martin J. Schalij, Jan A. Kors, Cees A. Swenne. (2010) The spatial QRS-T angle in the Frank vectorcardiogram: accuracy of estimates derived from the 12-lead electrocardiogram. Journal of Electrocardiology 43:4, 294-301
    CrossRef

  13. 13

    Borys Surawicz. (2010) How to increase the accuracy of electrocardiogram's interpretation and stimulate the interest of the interpreters?. Journal of Electrocardiology 43:3, 191-193
    CrossRef

  14. 14

    J. Fayn, P. Rubel. (2010) Toward a Personal Health Society in Cardiology. IEEE Transactions on Information Technology in Biomedicine 14:2, 401-409
    CrossRef

  15. 15

    A.P. Chandrakasan, D.C. Daly, D.F. Finchelstein, J. Kwong, Y.K. Ramadass, M.E. Sinangil, V. Sze, N. Verma. (2010) Technologies for Ultradynamic Voltage Scaling. Proceedings of the IEEE 98:2, 191-214
    CrossRef

  16. 16

    Arne Pfeufer, Charlotte van Noord, Kristin D Marciante, Dan E Arking, Martin G Larson, Albert Vernon Smith, Kirill V Tarasov, Martina Müller, Nona Sotoodehnia, Moritz F Sinner, Germaine C Verwoert, Man Li, W H Linda Kao, Anna Köttgen, Josef Coresh, Joshua C Bis, Bruce M Psaty, Kenneth Rice, Jerome I Rotter, Fernando Rivadeneira, Albert Hofman, Jan A Kors, Bruno H C Stricker, André G Uitterlinden, Cornelia M van Duijn, Britt M Beckmann, Wiebke Sauter, Christian Gieger, Steven A Lubitz, Christopher Newton-Cheh, Thomas J Wang, Jared W Magnani, Renate B Schnabel, Mina K Chung, John Barnard, Jonathan D Smith, David R Van Wagoner, Ramachandran S Vasan, Thor Aspelund, Gudny Eiriksdottir, Tamara B Harris, Lenore J Launer, Samer S Najjar, Edward Lakatta, David Schlessinger, Manuela Uda, Gonçalo R Abecasis, Bertram Müller-Myhsok, Georg B Ehret, Eric Boerwinkle, Aravinda Chakravarti, Elsayed Z Soliman, Kathryn L Lunetta, Siegfried Perz, H-Erich Wichmann, Thomas Meitinger, Daniel Levy, Vilmundur Gudnason, Patrick T Ellinor, Serena Sanna, Stefan Kääb, Jacqueline C M Witteman, Alvaro Alonso, Emelia J Benjamin, Susan R Heckbert. (2010) Genome-wide association study of PR interval. Nature Genetics 42:2, 153-159
    CrossRef

  17. 17

    Charlotte Noord, Marcus Dörr, Miriam C. J. M. Sturkenboom, Sabine M. J. M. Straus, Thorsten Reffelmann, Stephan B. Felix, Albert Hofman, Jan A. Kors, Robin Haring, Frank H. Jong, Matthias Nauck, André G. Uitterlinden, Henri Wallaschofski, Jacqueline C. M. Witteman, Henry Völzke, Bruno H. Ch. Stricker. (2010) The association of serum testosterone levels and ventricular repolarization. European Journal of Epidemiology 25:1, 21-28
    CrossRef

  18. 18

    Kwang-Shik Park, Min-Seok Seo, Wan-Je Cho, Jung-Hyun Lee, Hye-Ree Lee, Jae-Yong Shim. (2010) Relationship between Metabolic Syndrome and QTc Interval Prolongation. Korean Journal of Family Medicine 31:4, 267
    CrossRef

  19. 19

    Jonathan J. Halford. (2009) Computerized epileptiform transient detection in the scalp electroencephalogram: Obstacles to progress and the example of computerized ECG interpretation. Clinical Neurophysiology 120:11, 1909-1915
    CrossRef

  20. 20

    S. C. Dijkstra, I. A. Brouwer, F. J.A. van Rooij, A. Hofman, J. C.M. Witteman, J. M. Geleijnse. (2009) Intake of very long chain n-3 fatty acids from fish and the incidence of heart failure: the Rotterdam Study. European Journal of Heart Failure 11:10, 922-928
    CrossRef

  21. 21

    M. F. Sinner, A. Pfeufer, S. Perz, E. Schulze-Bahr, G. Monnig, L. Eckardt, B.-M. Beckmann, H.-E. Wichmann, G. Breithardt, G. Steinbeck, L. Fabritz, S. Kaab, P. Kirchhof. (2009) Spontaneous Brugada electrocardiogram patterns are rare in the German general population: results from the KORA study. Europace 11:10, 1338-1344
    CrossRef

  22. 22

    P. A. Hart, A. Arora, D. L. Francis. (2009) Utility of software analysis of esophageal manometry studies in patients with aperistalsis. Diseases of the Esophagus 22:1, 80-83
    CrossRef

  23. 23

    Jan Heeringa, Jan A. Kors, Albert Hofman, Frank J.A. van Rooij, Jacqueline C.M. Witteman. (2008) Cigarette smoking and risk of atrial fibrillation: The Rotterdam Study. American Heart Journal 156:6, 1163-1169
    CrossRef

  24. 24

    Jan A. Kors, Henk J. Ritsema van Eck, Gerard van Herpen. (2008) The meaning of the Tp-Te interval and its diagnostic value. Journal of Electrocardiology 41:6, 575-580
    CrossRef

  25. 25

    Femke Atsma, Marie-Louise E.L. Bartelink, Yvonne T. van der Schouw, Jan A. Kors, Diederick E. Grobbee. (2008) Elevated blood pressure and electrocardiographic frontal T axis and spatial QRS-T angle changes in postmenopausal women. Journal of Electrocardiology 41:4, 360-364
    CrossRef

  26. 26

    Stefan P. Nelwan, Jan A. Kors, Suzanne W. Crater, Simon H. Meij, Teus B. van Dam, Maarten L. Simoons, Mitchell W. Krucoff. (2008) Simultaneous comparison of 3 derived 12-lead electrocardiograms with standard electrocardiogram at rest and during percutaneous coronary occlusion. Journal of Electrocardiology 41:3, 230-237
    CrossRef

  27. 27

    Jocelyne Fayn, Paul Rubel, Peter W. Macfarlane. (2007) Can the lessons learned from the assessment of automated electrocardiogram analysis in the Common Standards for quantitative Electrocardiography study benefit measurement of delayed contrast-enhanced magnetic resonance images?. Journal of Electrocardiology 40:3, 246-250
    CrossRef

  28. 28

    Paul Kligfield, Leonard S. Gettes, James J. Bailey, Rory Childers, Barbara J. Deal, E. William Hancock, Gerard van Herpen, Jan A. Kors, Peter Macfarlane, David M. Mirvis, Olle Pahlm, Pentti Rautaharju, Galen S. Wagner. (2007) Recommendations for the standardization and interpretation of the electrocardiogram. Heart Rhythm 4:3, 394-412
    CrossRef

  29. 29

    Christine C. Chiu, Robert M. Hamilton, Robert M. Gow, Joel A. Kirsh, Brian W. McCrindle. (2007) Evaluation of computerized interpretation of the pediatric electrocardiogram. Journal of Electrocardiology 40:2, 139-143
    CrossRef

  30. 30

    Maya E. Guglin, Neeta Datwani. (2007) Electrocardiograms with pacemakers: accuracy of computer reading. Journal of Electrocardiology 40:2, 144-146
    CrossRef

  31. 31

    Paul Kligfield, Leonard S. Gettes, James J. Bailey, Rory Childers, Barbara J. Deal, E. William Hancock, Gerard van Herpen, Jan A. Kors, Peter Macfarlane, David M. Mirvis, Olle Pahlm, Pentti Rautaharju, Galen S. Wagner. (2007) Recommendations for the Standardization and Interpretation of the Electrocardiogram. Journal of the American College of Cardiology 49:10, 1109-1127
    CrossRef

  32. 32

    H. Völzke, C. Warnke, M. Dörr, A. Kramer, L. Guertler, W. Hoffmann, J. A. Kors, U. John, S. B. Felix. (2006) Association between cardiac disorders and a decades-previous history of diphtheria. European Journal of Clinical Microbiology & Infectious Diseases 25:10, 651-656
    CrossRef

  33. 33

    Henry Chen, Victor Froehlicher. 2006. Exercise Stress Testing. .
    CrossRef

  34. 34

    Jan H. van Bemmel, Erik M. van Mulligen, Barend Mons, Marc van Wijk, Jan A. Kors, Johan van der Lei. (2006) Databases for knowledge discovery. International Journal of Medical Informatics 75:3-4, 257-267
    CrossRef

  35. 35

    Sabine M.J.M. Straus, Jan A. Kors, Marie L. De Bruin, Cornelis S. van der Hooft, Albert Hofman, Jan Heeringa, Jaap W. Deckers, J. Herre Kingma, Miriam C.J.M. Sturkenboom, Bruno H. Ch. Stricker, Jacqueline C.M. Witteman. (2006) Prolonged QTc Interval and Risk of Sudden Cardiac Death in a Population of Older Adults. Journal of the American College of Cardiology 47:2, 362-367
    CrossRef

  36. 36

    Stephen R. Benoit, Aaron B. Mendelsohn, Parivash Nourjah, Judy A. Staffa, David J. Graham. (2005) Risk factors for prolonged QTc among US adults: Third National Health and Nutrition Examination Survey. European Journal of Cardiovascular Prevention & Rehabilitation 12:4, 363-368
    CrossRef

  37. 37

    A. M. W. Spijkerman, R. M. A. Henry, J. M. Dekker, G. Nijpels, P. J. Kostense, J. A. Kors, D. Ruwaard, C. D. A. Stehouwer, L. M. Bouter, R. J. Heine. (2004) Prevalence of macrovascular disease amongst type 2 diabetic patients detected by targeted screening and patients newly diagnosed in general practice: the Hoorn Screening Study. Journal of Internal Medicine 256:5, 429-436
    CrossRef

  38. 38

    Lynne Hall, Adrian Gordon, Lynne Newall, Russell James. (2004) A development environment for intelligent applications on mobile devices. Expert Systems with Applications 27:3, 481-492
    CrossRef

  39. 39

    Nkechi E. Azie, Gregory Adams, Borje Darpo, Steven F. Francom, Emery C. Polasek, Joy M. Wisser, Joseph C. Fleishaker. (2004) Comparing Methods of Measurement for Detecting Drug-Induced Changes in the QT Interval: Implications for Thoroughly Conducted ECG Studies. Annals of Noninvasive Electrocardiology 9:2, 166-174
    CrossRef

  40. 40

    Takao Katoh, Kenji Yodogawa, Tadaaki Ohno, Hirokazu Hayakawa. (2003) Improvement of the Diagnostic Accuracy in Computer-Assisted Differential Diagnosis for Wide QRS Premature Complexes. Circulation Journal 67:12, 1036-1040
    CrossRef

  41. 41

    Jodat Seyal, Elaine Clark, Peter Macfarlane. (2002) Journal of Cardiovascular Risk 9:2, 115-121
    CrossRef

  42. 42

    Alan H. Kadish, Alfred E. Buxton, Harold L. Kennedy, Bradley P. Knight, Jay W. Mason, Claudio D. Schuger, Cynthia M. Tracy, William L. Winters, Alan W. Boone, Michael Elnicki, John W. Hirshfeld, Beverly H. Lorell, George P. Rodgers, Cynthia M. Tracy, Howard H. Weitz. (2001) ACC/AHA clinical competence statement on electrocardiography and ambulatory electrocardiography. Journal of the American College of Cardiology 38:7, 2091-2100
    CrossRef

  43. 43

    P. Baroni, G. Guida, M. Zanella. (2001) Managing uncertainty in diagnosis of acute coronaric ischemia. Artificial Intelligence in Medicine 23:2, 129-147
    CrossRef

  44. 44

    Paul O. Collinson, Linda Chamberlain. (2001) Cardiac markers in the diagnosis of acute coronary syndromes. Current Cardiology Reports 3:4, 280-288
    CrossRef

  45. 45

    Nada Lemic-Stojcevic, Ruth Dundas, Stephen Jenkins, Anthony Rudd, Charles Wolfe. (2001) Preventable Risk Factors for Coronary Heart Disease and Stroke Amongst Ethnic Groups in London. Ethnicity & Health 6:2, 87-94
    CrossRef

  46. 46

    Peter M. Okin, Richard B. Devereux, Jan A. Kors, Gerard Herpen, Richard S. Crow, Richard R. Fabsitz, Barbara V. Howard. (2001) Computerized ST Depression Analysis Improves Prediction of All-Cause and Cardiovascular Mortality: The Strong Heart Study. Annals of Noninvasive Electrocardiology 6:2, 107-116
    CrossRef

  47. 47

    Steven Morrison, Peter W. Macfarlane. (2000) Computer Detection of Atrial Flutter. Annals of Noninvasive Electrocardiology 5:4, 358-364
    CrossRef

  48. 48

    M. Lagerholm, C. Peterson, G. Braccini, L. Edenbrandt, L. Sornmo. (2000) Clustering ECG complexes using Hermite functions and self-organizing maps. IEEE Transactions on Biomedical Engineering 47:7, 838-848
    CrossRef

  49. 49

    Frank-Erik De Leeuw, Jan Cees De Groot, Matthijs Oudkerk, Jacqueline C. M. Witteman, Albert Hofman, Jan Van Gijn, Monique M. B. Breteler. (1999) A follow-up study of blood pressure and cerebral white matter lesions. Annals of Neurology 46:6, 827-833
    CrossRef

  50. 50

    Joel Morganroth, Steven S. Silber. (1999) How to Obtain and Analyze Electrocardiograms in Clinical Trials.. Annals of Noninvasive Electrocardiology 4:4, 425-433
    CrossRef

  51. 51

    Holger Holst, Mattias Ohlsson, Carsten Peterson, Lars Edenbrandt. (1999) A confident decision support system for interpreting electrocardiograms. Clinical Physiology 19:5, 410-418
    CrossRef

  52. 52

    M. L. Bots, A. W. Hoes, A. Hofman, J. C. M. Witteman, D. E. Grobbee. (1999) Cross-sectionally assessed carotid intima-media thickness relates to long-term risk of stroke, coronary heart disease and death as estimated by available risk functions. Journal of Internal Medicine 245:3, 269-276
    CrossRef

  53. 53

    N Kontodimopoulos. (1999) In-house development of test equipment for quality control and training. Case study: a prototype ECG simulator-tester. Medical Engineering & Physics 20:10, 717-721
    CrossRef

  54. 54

    Patricia Wright, Carel Jansen, Jeremy C Wyatt. (1998) How to limit clinical errors in interpretation of data. The Lancet 352:9139, 1539-1543
    CrossRef

  55. 55

    Jan A Kors, Martine C de Bruyne, Arno W Hoes, Gerard van Herpen, Albert Hofman, Jan H van Bemmel, Diederick E Grobbee. (1998) T axis as an indicator of risk of cardiac events in elderly people. The Lancet 352:9128, 601-605
    CrossRef

  56. 56

    Holst, Ohlsson, Peterson, Edenbrandt. (1998) Intelligent computer reporting `lack of experience': a confidence measure for decision support systems. Clinical Physiology 18:2, 139-147
    CrossRef

  57. 57

    M. Sunemark, L. Edenbrandt, H. Holst, L. Sörnmo. (1998) Serial VCG/ECG Analysis Using Neural Networks. Computers and Biomedical Research 31:1, 59-69
    CrossRef

  58. 58

    R.-D. Bousseljot, D. Kreiseler. (1998) Ergebnisse bei der EKG-Analyse durch Signalmustervergleich. Biomedizinische Technik/Biomedical Engineering 43:s1, 234-235
    CrossRef

  59. 59

    Xinhua Li, Monique M. B. Breteler, Martine C. Bruyne, Harry Meinardi, W. Allen Hauser, Albert Hofman. (1997) Vascular Determinants of Epilepsy: The Rotterdam Study. Epilepsia 38:11, 1216-1220
    CrossRef

  60. 60

    Bharat N Nathwani, Kenneth Clarke, Thomas Lincoln, Costan Berard, Clive Taylor, Kc Ng, Ramesh Patil, Malcolm C Pike, Stanley P Azen. (1997) Evaluation of an expert system on lymph node pathology. Human Pathology 28:9, 1097-1110
    CrossRef

  61. 61

    M.C. de Bruyne, J.A. Kors, A.W. Hoes, D.A.C. Kruijssen, J.W. Deckers, M. Grosfeld, G. van Herpen, D.E. Grobbee, J.H. van Bemmel. (1997) Diagnostic interpretation of electrocardiograms in population-based research: Computer program research physicians, or cardiologists?. Journal of Clinical Epidemiology 50:8, 947-952
    CrossRef

  62. 62

    Fabio Badilini, Jocelyne Fayn, Pierre Maison-Blanche, Antoine Leenhardt, Marie Claire Forlini, Isabelle Denjoy, Philippe Coumel, Paul Rubel. (1997) Quantitative Aspects of Ventricular Repolarization.. Annals of Noninvasive Electrocardiology 2:2, 146-157
    CrossRef

  63. 63

    Jean-Raoul Scherrer. (1997) Editorial commentary: Aims and tasks of medical informatics. International Journal of Medical Informatics 44:1, 45-52
    CrossRef

  64. 64

    Marvin E. Gozum, Andrew S. Kanter, Dawn E. DeWitt. (1995) Benefits of computer diagnostic assistants. Journal of General Internal Medicine 10:7, 413-414
    CrossRef

  65. 65

    FRANCIS D. MURGATROYD, BAIYAN XIE, XAVIER COPIE, IVAN BLANKOFF, A. JOHN CAMM, MAREK MALIK. (1995) Identification of Atrial Fibrillation Episodes in Ambulatory Electrocardiographic Recordings: Validation of a Method for Obtaining Labeled R-R Interval Files. Pacing and Clinical Electrophysiology 18:6, 1315-1320
    CrossRef

  66. 66

    Jari J. Forsström, Kevin J. Dalton. (1995) Artificial Neural Networks for Decision Support in Clinical Medicine. Annals of Medicine 27:5, 509-517
    CrossRef

  67. 67

    Greg C. Larsen, John L. Griffith, Joni R. Beshansky, Ralph B. D’Agostino, Harry P. Selker. (1994) Electrocardiographic left ventricular hypertrophy in patients with suspected acute cardiac ischemia—Its influence on diagnosis, triage, and short-term prognosis. Journal of General Internal Medicine 9:12, 666-673
    CrossRef

  68. 68

    T. -F. Yang, B. Devine, P. W. Macfarlane. (1994) Artificial neural networks for the diagnosis of atrial fibrillation. Medical & Biological Engineering & Computing 32:6, 615-619
    CrossRef

  69. 69

    DAVID CUBANSKI, DAVID CYGANSKI, ELLIOTT M. ANTMAN, CHARLES L. FELDMAN. (1994) A Neural Network System for Detection of Atrial Fibrillation in Ambulatory Electrocardiograms. Journal of Cardiovascular Electrophysiology 5:7, 602-608
    CrossRef

  70. 70

    L. Edenbrandt, A. Houston, P. W. Macfarlane. (1994) Vectorcardiograms synthesized from 12-lead ECGs: A new method applied in 1792 healthy children. Pediatric Cardiology 15:1, 21-26
    CrossRef

  71. 71

    Steven D. Hillson, Donald P. Connelly. (1992) Computer-assisted test interpretation: Considerations in patient care. Journal of Medical Systems 16:5, 195-205
    CrossRef

  72. 72

    (1992) Computer Interpretation of Electrocardiograms. New England Journal of Medicine 326:24, 1634-1635
    Full Text

  73. 73

    Laks, Michael M., , Selvester, Ronald H. Startt, . (1991) Computerized Electrocardiography — An Adjunct to the Physician. New England Journal of Medicine 325:25, 1803-1804
    Full Text