The Effect of Different Diagnostic Criteria on the Prevalence of Dementia

The data we analyzed were collected as part of the Canadian Study of Health and Aging (CSHA), a national, multicenter epidemiologic study of dementia.¹⁰ The CSHA surveyed randomly selected samples of people 65 years of age or older throughout Canada. Of the 10,263 people surveyed, 9008 were living in the community and 1255 in long-term care institutions. An extensive neurologic and neuropsychological examination was performed on all subjects in the institutions, those in the community who had cognitive impairment on screening with the Modified Mini–Mental State Examination,¹¹ and a subsample of those in the community who did not have cognitive impairment on screening. The neuropsychological examination assessed the cognitive domains included among the various criteria for dementia: memory, abstract thinking, judgment, presence or absence of aphasia, presence or absence of apraxia, presence or absence of agnosia, and constructional abilities (particular tests used to assess each domain are described by Tuokko et al.¹² and Steenhuis and Østbye¹³).

On the basis of the neuropsychological tests, background information, and established normative information, the study neuropsychologists determined whether participants had impairment within each of the cognitive domains and made a preliminary diagnosis. The assessments by the physicians included a mental-status evaluation (including the Modified Mini–Mental State Examination), as well as physical and neurologic examinations. A knowledgeable informant (a family member, friend, or formal care giver) supplied historical information through an interview using the questions in the CAMDEX.⁹ The physician made an independent preliminary diagnosis. The final diagnosis of normality or abnormality in each domain of function was based on the results of the clinical examination, the neuropsychological evaluation, and the interview with the informant. At a clinical-consensus meeting, the subjects were classified as having no cognitive loss, cognitive loss but not dementia, or dementia. Dementia was rated as mild, moderate, or severe according to the guidelines of the DSM-III-R.⁴ The cause of dementia was classified as possible or probable Alzheimer's disease, according to the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association¹⁴; possible or probable vascular dementia; or other causes, according to the ICD-10 criteria.³ For a subgroup made up of 210 participants, a second diagnosis was made, on the basis of the clinical information. The agreement between this diagnosis and the original clinical-consensus diagnosis (dementia or no dementia) had a kappa value of 0.81.¹⁰

All 1879 subjects who underwent clinical evaluation, including the neuropsychological examination, were included in the present analyses. The classification systems used for diagnosing dementia were those of the DSM-III, DSM-III-R, DSM-IV, ICD-9, ICD-10, and CAMDEX. The diagnoses according to these systems were not based on assessments by different clinicians. Rather, we first identified the domains used in the various diagnostic systems, then identified those variables from the data set (based on a consensus of a neurologist, neuropsychologist, and nurse) that most closely corresponded to these domains. Given the amount and detail of information that had been collected, we had sufficient information to determine whether the subject had evidence of impairment in each domain.

Table 1. Table 1. Criteria for Dementia in the Classification Systems.

Table 1 shows graphically the sets of diagnostic criteria for dementia. The DSM-III, for example, includes difficulties with short-term or long-term memory, difficulties in one or more of the areas of abstract thinking, judgment, higher cortical functions, or personality, and impairment in work or social functioning. A decline from a level before the onset of illness is specified, with normal consciousness and an assumed organic cause. The ICD-9 and ICD-10 criteria required more factors to be present than the other systems.

Figure 1. Figure 1. Subjects Identified as Having Dementia According to Various Diagnostic Classification Systems.

A total of 1879 subjects were evaluated. One subject, in whom dementia was diagnosed according to DSM-IV and CAMDEX, is not shown.

Table 2. Table 2. Demographic Characteristics of 1879 Subjects in the CSHA Cohort with and without Dementia According to Various Classification Systems.

Table 3. Table 3. Prevalence of Dementia in the CSHA Cohort as Diagnosed by Various Classification Systems, According to Age Group.

The contribution of the various factors to the diagnosis according to different classification schemes was calculated, and the degree of overlap among the patients identified as having dementia was examined (Figure 1). Given the relatively large proportion of subjects without dementia by any classification in our sample, we chose as a measure of concordance the proportion of subjects in whom dementia was diagnosed according to each system divided by the total number of subjects in whom it was diagnosed by any system. The demographic characteristics of the subjects with and without dementia were compared (Table 2 and Table 3). The relative frequency of moderate and severe dementia, Alzheimer's disease, and vascular dementia and the duration of symptoms were examined. Finally, to determine the relative contribution of various factors to agreement or disagreement between DSM-IV and ICD-10, between DSM-IV and CAMDEX, and between ICD-10 and CAMDEX, multivariate logistic-regression models were fitted. The SAS software package (version 6.07; SAS Institute, Cary, N.C.) was used for data management and analysis.

The mean age of the 1879 subjects in the sample was 80.4 years; 62.4 percent were women, 42.5 percent had more than nine years of education, and 69.7 percent lived in the community.

The frequency of dementia in the CSHA cohort when diagnosed according to the criteria for dementia in the six different classification systems varied considerably (Table 1). The proportion of subjects with dementia was 29.1 percent when the DSM-III criteria were used, 17.3 percent with the DSM-III-R criteria, 13.7 percent with the DSM-IV criteria, 5.0 percent with the ICD-9 criteria, 3.1 percent with the ICD-10 criteria, and 4.9 percent with the CAMDEX. The frequency of dementia according to the CSHA clinical-consensus diagnosis was 20.9 percent. Because the prevalence in the CSHA cohort was not adjusted for oversampling of the very old, it cannot be compared directly with the figures published by the CSHA for the prevalence of dementia in Canada.¹⁰

A step-by-step evaluation of the changes in prevalence as each factor was entered into the algorithm showed, for instance, that about half as many subjects had both short- and long-term memory dysfunction as had only defective short-term memory. That is, the inclusion of long-term memory impairment as a requirement for the diagnosis of dementia had a substantial effect on prevalence and probably accounted for most of the difference between the DSM-III criteria and the later DSM-based systems (45.4 percent of the subjects had short-term memory problems as required by DSM-III, whereas only 23.3 percent of the subjects had both short-term and long-term memory problems, as required by the DSM-III-R and DSM-IV). The ICD-9 and ICD-10 classifications required more types of impairment and thus identified fewer subjects as having dementia. Far fewer subjects (only 5.4 percent) met the requirement for impaired executive function (abstract thinking, judgment, and problem solving) than met the less restrictive criterion of dysfunction in one of a variety of higher cortical functions that included abstract thinking and judgment (52.3 percent). The inclusion of impairment in basic activities of daily living as a criterion in ICD-10 and of evidence of progressive deterioration in CAMDEX also produced a change in the proportions in whom dementia was diagnosed.

Regardless of which diagnostic scheme was used, the subjects with dementia were slightly older than those without dementia, but the difference was not significant (Table 2). The proportion of women among those who had dementia was similar to the proportion among those without dementia. The subjects who were given a diagnosis of dementia according to the DSM-III and the DSM-IV criteria had less education than those without dementia, but the level of education was similar for the other classification systems. Not surprisingly, for all the classification systems, subjects with dementia were more often institutionalized than those without dementia. Overall, the classification systems differed little according to the subjects' age, sex, amount of education, or living conditions.

The frequency of dementia increased with increasing age regardless of which diagnostic classification system was used (Table 3). According to the criteria of DSM-III-R, for example, the frequency of dementia was 10.5 percent among subjects 65 to 74 years old, 16.0 percent among those 75 to 84 years old, and 24.4 percent among those 85 years of age or older.

The classification systems that identified a higher percentage of the study population as having dementia (DSM-III, DSM-III-R, and DSM-IV) included more cases with mild dementia, and there was a trend toward a shorter mean duration of symptoms. Each successive revision of the DSM appeared to extend the diagnosis to fewer subjects with mild dementia. The ICD-10 and CAMDEX classifications identified the smallest proportion of mild cases. The distribution of Alzheimer's disease and vascular dementia did not differ markedly among most of the classification systems. The exception was the CAMDEX, which assigned the diagnosis of dementia mainly to those with Alzheimer's disease.

Figure 1 shows the relation among the cases identified by the ICD-10, DSM-IV, and CAMDEX criteria and the CSHA clinical-consensus process. There was complete overlap between the CAMDEX and clinical-consensus diagnoses, although the CAMDEX identified only a subgroup of the cases of dementia identified by the CSHA consensus process. Neither DSM-IV nor ICD-10 identified a complete subgroup of the cases identified by a different classification system. The groups of subjects identified as having dementia by the CSHA consensus process and by the DSM-IV criteria had the least overlap, with fewer than half the clinical-consensus cases identified by DSM-IV. Conversely, about a third of the DSM-IV cases were not classified as dementia by consensus.

The simple concordance between the clinical-consensus diagnosis of dementia or no dementia and the diagnosis according to the DSM-IV, ICD-10, and CAMDEX criteria were 79.2 percent (1489 of 1879), 78.7 percent (1479 of 1879), and 83.8 percent (1574 of 1879), respectively. Since most of this apparent agreement is due to the large number of our subjects who did not have dementia according to any diagnostic system, we also calculated the proportion of subjects who were classified as having dementia according to one system as a proportion of those who had dementia according to any system. The results were as follows: ICD-10, 12.9 percent (58 of 450); DSM-IV, 57.1 percent (257 of 450); and CAMDEX, 20.4 percent (92 of 450).

Table 4. Table 4. Odds Ratios for Disagreement between Classification Systems in Multivariate Logistic-Regression Models.

Factors that most often accounted for disagreement between the classification schemes were further studied by multivariate logistic-regression analysis. The factors that best predicted disagreement between DSM-IV and ICD-10 were long-term memory, executive function, presence or absence of aphasia, social activities, and duration of symptoms (Table 4). DSM-IV and CAMDEX were differentiated by the weight given to social activities and progressive deterioration, whereas long-term memory, social function, progressive deterioration, and the presence or absence of an assumed organic cause differentiated between ICD-10 and CAMDEX.

In this large, population-based study, the frequency of dementia varied dramatically when different systems of diagnostic classification were used. Although there was substantial overlap among the groups of subjects identified by the various systems as having dementia, many individual subjects assigned a diagnosis of dementia by one classification system were not so identified by another. This finding arouses concern about the validity of comparisons among studies that use different criteria to diagnose dementia. Questions about validity also arise with regard to the ICD-based systems, which are more likely to identify advanced cases of dementia in which the diagnosis is quite apparent.

Our large sample drawn from a population-based study allowed us to examine the effects of various classification systems on the frequency of the diagnosis of dementia. The CSHA is one of a few studies in which detailed and structured clinical and neuropsychological evaluations have been used to assess cognitive functions in the domains incorporated into diagnostic classifications.

In our study, the frequency of dementia was 3.1 percent when the ICD-10 criteria were used, 4.9 percent with the CAMDEX, 5.0 percent with ICD-9, 13.7 percent with DSM-IV, 17.3 percent with the DSM-III-R, 20.9 percent according to the CSHA clinical-consensus method, and 29.1 percent with the DSM-III criteria. The highest prevalence (with the DSM-III criteria) was approximately 10 times the lowest (with ICD-10). Increasing frequency of dementia with increasing age and lower educational levels and the absence of difference between the sexes have been reported previously.^2,15,16 Other comparative studies^2,5 also found that fewer cases were identified by the ICD-10 criteria than by the DSM-III-R criteria. In our study, the rate of dementia diagnosed with the DSM-III-R criteria was roughly six times that with the ICD-10, whereas the earlier studies reported a twofold difference. This discrepancy is likely to be due to the more detailed methods used by the CSHA and the characteristics of the sample (i.e., community and institutional).

In a study of 486 patients with stroke (mean age, 72 years), Pohjasvaara et al.¹⁷ reported a frequency of 25 percent for dementia when the DSM-III criteria were used, 20 percent with the DSM-III-R criteria, 18 percent with the DSM-IV criteria, and 6 percent with the ICD-10 criteria. In another study of stroke, Tatemichi et al.¹⁸ also found varying proportions of patients identified as having dementia according to the criteria of the DSM-III (30 percent), the National Institute of Neurological Disorders and Stroke–Association Internationale pour la Recherche et l'Enseignement en Neurosciences (27 percent),¹⁹ and Cummings and Benson (41 percent).²⁰ In contrast, in a series of 167 selected patients with suspected dementia (mean age, 72 years), Wetterling et al.²¹ found that the numbers of cases diagnosed by four classification systems were similar (86 cases with DSM-IV and 85 cases with ICD-10; the other two systems were not included in our study), but the groups included different patients.

In the present study, the various classification systems yielded distinct groups; only 20 subjects were given a diagnosis of dementia according to all six classification systems. Thus, the problem is not simply that some systems are more restrictive than others. Rather, the systems identify different individual subjects as having dementia. Similarly, the work of Wetterling et al.²¹ yielded distinct groups of subjects, of whom only 35 percent met the criteria of all the classification systems used. The same results have been found when different clinical criteria are used for the diagnosis of vascular dementia.²² In the Canberra study,² the kappa statistic for the agreement between the DSM-III-R and the ICD-10 criteria was only 0.48. A similar value was reported in a cohort with stroke (kappa = 0.43).¹⁷ Thus, the literature indicates a lack of diagnostic agreement and only a moderate degree of concordance between the various classification systems.

Based on our analysis of the algorithms used, the main factors related to the differences in the frequency of dementia in the study sample included the requirement that both short-term and long-term memory be impaired in the DSM-III-R, DSM-IV, and CAMDEX; the requirement of impairment in abstract thinking, judgment, and problem solving in ICD-9 and ICD-10; the requirement of impairment in basic activities of daily living in ICD-10; and the requirement of evidence of progressive deterioration in CAMDEX. The multivariate regression analysis supported those observations and also identified long-term memory and executive function as important. The duration of symptoms, the presence or absence of aphasia, and social functioning also contributed. Similarly, in the series of Pohjasvaara et al.,¹⁷ the factors influencing diagnostic differences were long-term memory, executive function, and the duration of cognitive symptoms.

Our findings support the need for validation of the criteria used to diagnose dementia. Inconsistency of diagnosis is not unique to dementia, however. For example, there are similar problems with diagnostic concordance and validity with the DSM and ICD criteria for substance use disorder,²³ personality disorder,²⁴ somatization disorder,²⁵ and affective and psychotic disorders,²⁶ with different systems emphasizing different aspects of the conditions.

One possible conclusion is that universal standards are needed for diagnosis — that is, that all investigators and clinicians should use the same classification scheme and criteria. Certainly, such uniformity would make international comparisons of prevalence and incidence more meaningful. However, a more basic question relates to the validity of the criteria and the underlying theoretical and empirical foundations of diagnosis. In the context of cerebrovascular disease, for example, recent discussions have highlighted issues related to the validity of current criteria for the diagnosis of dementia.^19,27,28 The current criteria for the diagnosis of Alzheimer's disease are also constrained by the concept of dementia.¹⁴ New discoveries (e.g., the importance of early medial-temporal-lobe atrophy, characteristics of memory impairment, and the apolipoprotein E genotype) may soon permit the diagnosis of probable Alzheimer's disease before the dementia is clinically obvious.^29-32

All investigators should use the same minimal set of standardized, validated measures and record key demographic characteristics, so that patients can be reclassified and the findings reinterpreted in the light of emerging knowledge.²⁷ The validity (construct, content, and criterion) of diagnostic classifications is essential for studies of the epidemiology, risk factors, prevention, and treatment of any disorder. This use of standardized measures is critical, especially at the early stages of disease, when early detection of pathologic processes is essential if any intervention is to prevent or retard cognitive impairment. One proposal is that the focus should be on the spectrum of cognitive impairment, and the label “dementia” may even be abandoned.²⁷

From the viewpoint of research, a given person may or may not qualify for a therapeutic protocol, depending on the label applied to his or her condition. Clinically, it makes a great difference whether a patient is labeled as having dementia. The diagnosis often sets the threshold for investigation, treatment, and prognosis. Third-party reimbursement depends on the diagnosis, as does the patient's ability to obtain insurance. Legally, the diagnosis of dementia may deprive a person of the right to drive, manage personal affairs, and make a will. Diagnostic methods that generate prevalence figures that vary by a factor of 10 have important implications for health care planning. It makes a substantial difference whether 3 percent or 29 percent of the population over 65 years of age has dementia; the resources needed for prevention, treatment, and long-term care differ dramatically in these two cases. Our findings, and the prospect of the early diagnosis and treatment of dementia, point to the urgency of further debate and studies to redefine and refine the characterization of categories of cognitive impairment and dementia.