Correspondence
Assessment in Medical Education
N Engl J Med 2007; 356:2108-2110May 17, 2007
- Article
To the Editor:
I am disappointed that Epstein's review of assessment in medical education (Jan. 25 issue)1 does not include the short case, which is an excellent instrument used in many educational settings.2 In the short case, the student is asked to perform a supervised focused physical examination (e.g., of only the abdomen) of a real patient, with little knowledge of the patient's history, and is then assessed on the basis of the technique of the examination and the ability to elicit physical signs and interpret these findings correctly. Several cases are used to improve validity and reliability.
By making physical examination an integral part of trainee evaluation, the short case represents a tool for curtailing the decline in physical-examination skills.3 Furthermore, the short case uniquely prepares a trainee to perform an accurate assessment and place physical findings in context. These skills are especially valuable in the current era of duty-hour restrictions and increasing shift work,4 since a house officer is likely to be called to evaluate a patient he or she hardly knows.
4 ReferencesKapil Parakh, M.D., M.P.H.
Johns Hopkins University School of Medicine, Baltimore, MD 21224
kparakh1@jhmi.edu 1
Epstein RM . Assessment in medical education. N Engl J Med 2007;356:387-396
Full Text | Web of Science | Medline2
Fowell SL ,Maudsley G ,Maguire P ,Leinster SJ ,Bligh J . Student assessment in undergraduate medical education in the United Kingdom, 1998. Med Educ 2000;34:Suppl 1:1-49
CrossRef | Web of Science | Medline3
Jauhar S . The demise of the physical exam. N Engl J Med 2006;354:548-551
Full Text | Web of Science | Medline4
Fletcher KE ,Underwood W III ,Davis SQ ,Mangrulkar RS ,McMahon LF Jr ,Saint S . Effects of work hour reduction on residents' lives: a systematic review. JAMA 2005;294:1088-1100
CrossRef | Web of Science | Medline
To the Editor:
In his review of assessment methods, Epstein misses an opportunity to discuss the importance of providing constructive feedback to the trainee. This failure is painfully illustrated by the video that accompanies the article. The video shows two students separately examining a standardized patient. The students are subsequently confronted with (long lists of) their shortcomings and mistakes. If the assessment does not provide clear, positive feedback and reassurance and does not focus on specific points to improve, students may fail to learn from their mistakes, lose motivation to engage in assessment in the future, and develop a disproportionate fear of failure in their contact with real patients.1 Moreover, although in subsequent examinations, students may show improvement in areas in which they previously received a negative assessment, they may fail in areas in which they performed well earlier but without positive feedback.
The purpose of assessment is to improve the clinical skills of trainees and produce competent physicians who can provide high-quality care. This requires formative feedback that encourages self-reflection, actively reinforces good medical skills, and provides specific strategies for improving performance.2
2 ReferencesBastiaan E. de Galan, M.D., Ph.D.
Petra J. van Gurp, M.D.
Paul M. Stuyt, M.D., Ph.D.
Radboud University Nijmegen Medical Center, 6500 HB Nijmegen, the Netherlands
b.degalan@aig. umcn. nl 1
Kilminster S ,Jolly B ,van der Vleuten CPM . A framework for effective training for supervisors. Med Teach 2002;24:385-389
CrossRef | Web of Science | Medline2
Brukner H ,Altkorn DL ,Cook S , et al. Giving effective feedback to medical students: a workshop for faculty and house staff. Med Teach 1999;21:161-165
CrossRef | Web of Science | Medline
To the Editor:
As Klass notes in the editorial1 accompanying the review by Epstein, current assessment systems do not perform acceptably for evaluating physicians in practice. We need to move away from hours of continuing medical education or attendance at meetings or symposia and allow practice members to design their own continuing education on the basis of specific practice characteristics. All members of a practice, rather than individual physicians, should be held accountable for the progress of the practice's population. When electronic medical records are widely available, practice assessments could be developed, evaluated, and adjusted annually on the basis of the practice demographics (e.g., the numbers of patients in the practice according to diagnosis and health status).2 We also need to move away from measures focused strictly on process to an emphasis on patient outcomes. For example, the Health Plan Employer Data and Information Set (HEDIS) looks at data for the proportion of eligible patients who have been immunized, but the system currently includes information only about shots given, not about immunization schedules completed. Similarly, the proportion of visits for hypertension can be analyzed, but individual outcomes (e.g., control of blood pressure) cannot. This assessment system would be greatly enhanced by better training in epidemiology for all physicians.
2 ReferencesChristopher M. Buttery, M.B., B.S., M.P.H.
Virginia Commonwealth University, Richmond, VA 23298
cbuttery@vcu.edu 1
Klass D . Assessing doctors at work -- progress and challenges. N Engl J Med 2007;356:414-415
Full Text | Web of Science | Medline2
Buttery CMG . Clinical investigation in general practice: the use of a simplified data-recording system. South Med J 1963;56:650-653
CrossRef | Web of Science | Medline
To the Editor:
Though Klass states that “little attention has been paid to the assessment of doctors who are already in practice,” 24 member boards of the American Board of Medical Specialties, including the American Board of Internal Medicine (ABIM), offer maintenance of certification — a process for assessing physician competence that promotes lifelong learning and high-quality care. Recent articles in the Journal 1-3 describe the program's importance and value.
Through certification and maintenance of certification, ABIM assesses physician competence throughout a lifetime of practice. ABIM's secure examination tests medical knowledge and assesses the clinical reasoning and judgment required of an excellent internist. Our self-evaluation modules allow physicians to examine their clinical knowledge and performance in practice and to use these data to direct improvements in patient care.
Maintenance of certification is a rigorous mechanism for assessing physicians in practice. It addresses the important need that Klass identifies. Unfortunately, participation is not universal. Perhaps that is the issue to which too “little attention has been paid.”
3 ReferencesChristine K. Cassel, M.D.
American Board of Internal Medicine, Philadelphia, PA 191061
Baron RJ . Personal metrics for practice -- how'm I doing? N Engl J Med 2005;353:1992-1993
Full Text | Web of Science | Medline2
Brennan TA . Recertification for internists -- one “grandfather's” experience. N Engl J Med 2005;353:1989-1992
Full Text | Web of Science | Medline3
Steinbrook R . Renewing board certification. N Engl J Med 2005;353:1994-1997
Full Text | Web of Science | Medline
Author/Editor Response
The short case may hold promise as a method for assessment of clinical skills and may prove to be a robust and economical alternative or adjunct to long cases and standardized patient examinations. However, a Medline search with the search terms “short case” and “education” or “assessment” yielded no articles describing and validating its use in English-speaking or Western European countries. Three articles mentioned its use.1-3 An article from Kuwait indicated that the performance of the short case compared favorably with that of the long case.4 A word search of the study by Fowell and colleagues5 cited by Parakh showed that the short case was simply included in a list of assessment methods commonly used in the United Kingdom; there was no further description. I am certain that there are many other assessment methods that will ultimately prove to have merit, but just as with diagnostic tests in clinical medicine, we must await empirical validation before promoting their use.
I agree with de Galan and colleagues that feedback must be balanced, timely, and accurate. The purpose of feedback is to promote an organized approach to reflection and remediation of clinical skills. Feedback should lead to greater mindfulness, attentive self-observation, and critical curiosity during actual clinical practice and should enhance the motivation for change. The list of deficiencies in the video that accompanies the article was intended to emphasize the kinds of skills that can be observed in simulated settings; it was not intended to suggest that all those items should be presented to the student. In real-life settings, feedback should be based on respect and trust, should build on strengths, should be given in bite-size pieces that can be assimilated, and should be framed to increase self-confidence, skills, and autonomy. Feedback should be embedded in longitudinal mentoring relationships, which are all too rare in clinical teaching and in medical education in general. Otherwise, the feedback runs the risk of being haphazard and will be less likely to build incrementally on the trainees' emerging skills and competence.
5 ReferencesRonald Epstein, M.D.
University of Rochester School of Medicine and Dentistry, Rochester, NY 14610
ronald_epstein@urmc.rochester. edu 1
Wass V ,Van der Vleuten C ,Shatzer J ,Jones R . Assessment of clinical competence. Lancet 2001;357:945-949
CrossRef | Web of Science | Medline2
Singh R ,Gupta P ,Singh K ,Koirala S . Undergraduate pediatric education at BPKIHS integrated with an innovative curriculum. Indian Pediatr 1999;36:43-50
Medline3
Mekasha A . Assessment methods in medical education. Ethiop Med J 2004;42:63-71
Medline4
Hijazi Z ,Premadasa IG ,Moussa MA . Performance of students in the final examination in paediatrics: importance of the “short cases.” Arch Dis Child 2002;86:57-58
CrossRef | Web of Science | Medline5
Fowell SL ,Maudsley G ,Maguire P ,Leinster SJ ,Bligh J . Student assessment in undergraduate medical education in the United Kingdom, 1998. Med Educ 2000;34:Suppl 1:1-49
CrossRef | Web of Science | Medline
Author/Editor Response
I agree with the comments of both Buttery and Cassel. I would add to what Buttery says that although the task of making valid judgments about practices on the basis of outcomes is an important objective, it must be undertaken carefully because of the tortuous and poorly understood relationships between process and outcome in most medical situations. And I should apologize to Cassel if my suggestion of a relative lack of attention to assessments in practice over time disparaged the efforts of many organizations to address this discrepancy. In particular, the leadership of John J. Norcini and F. Daniel Duffy (former and current staff members, respectively, at ABIM) in this area is notable. Editorial constraints limited my ability to be more complete in attribution, so I thank Cassel for using this venue to draw attention to the important work in this area by ABIM and also by the American Board of Medical Specialties.
Daniel Klass, M.D., C.M.
College of Physicians and Surgeons of Ontario, Toronto, ON M5G 2E2, Canada
