Clinical Decisions
American Board of Internal Medicine Maintenance of Certification Program
N Engl J Med 2010; 362:948-952March 11, 2010
- Article
Poll Results
Vignette
A 55-year-old physician, who graduated from medical school in 1979, completed his internship and junior residency in internal medicine in 1981. He did 3 years of fellowship training in endocrinology, followed by a third year of residency in internal medicine. He then completed and passed the American Board of Internal Medicine (ABIM) qualifying examinations in internal medicine and endocrinology; the ABIM issued to him certificates of unlimited duration, specifying that he held board certification in internal medicine and endocrinology. For the past 24 years, he has been in the practice of general internal medicine with an emphasis on endocrinology. He estimates that about half of his patients see him for endocrine problems exclusively and the remainder see him for issues regarding general internal medicine. He is on the faculty of the medical school from which he graduated, but he spends most of his time caring for patients in an outpatient setting. For 6 months of each year, he has third-year residents, training in internal medicine, shadowing him at his outpatient practice. For one 3-week block each year, he serves as an attending physician on the general medical service of the hospital where he has inpatient privileges, a minor teaching hospital of his medical school at which he trained. During this 3-week period, he has two medical students, two interns, and a senior medical resident under his direct supervision.
He attends the grand rounds lecture series regularly. Once a year, he leaves his practice to attend a weeklong postgraduate course, alternating between internal medicine and general endocrinology. He has never considered enrolling in the maintenance of certification (MOC) program in either internal medicine or endocrinology. His reasoning is that he is up to date in his practice, that he makes frequent use of point-of-care reference services to check on the latest diagnostic or therapeutics practices when he encounters a condition that he does not see frequently, and that the money and time needed to undertake MOC would not be well spent. He has received a communication from the ABIM urging him to undergo the MOC process. He consults you, as a friend and colleague, for advice about what to do.
- Recommendation 1
- Recommendation 1Enroll in the MOC Program as Currently Configured
Like our colleague in the vignette, approximately 69,000 diplomates of the American Board of Internal Medicine (ABIM) have time-unlimited certification, often called “grandfather” status. We are among the less than 1% of such physicians who have chosen to become recertified by participating in the maintenance of certification (MOC) program. As members of the ABIM Board of Directors, we were required to participate in MOC. We found that participation in MOC fit with our commitment to lifelong learning and gave us more confidence (and tangible evidence) that we are up to date in our fields.
Many physicians question the value of MOC, and — we must confess — so did we. We used to believe that years of practice build knowledge and competence and that ongoing participation in continuing medical education (CME) reinforces a basic understanding of clinical concepts. But the evidence does not bear that out.1,2
Considerable research shows that traditional CME does not deliver when it comes to improved physician performance and knowledge.1-3 However, MOC, as a structured assessment process, is more likely to stop the decline over time in a physician's skills, knowledge, and performance4 than more passive CME. MOC requires physicians to complete a self-assessment of their knowledge, participate in practice-based quality-improvement activities, and pass a secure, knowledge-based examination.
Initial ABIM certification and MOC have both been associated with better care.5-10 Physicians who are certified in their field provide higher-quality care — such as in the management of acute myocardial infarction7,8 and delivery of preventive services9 — than noncertified physicians. Similarly, physicians who participate in MOC have better clinical outcomes.5,6,10 For example, internists scoring in the top quartile on the ABIM MOC examination were more likely to provide better care for patients with diabetes than physicians with scores in lower quartiles; those who did not pass the examination performed least well in terms of benchmarks of quality care.6
What physicians think they know and do in practice does not match what they actually know and do. In fact, physicians are not good at assessing their own skills — fewer than 30% of physicians examine their own performance data,11 and physicians' ability to independently and accurately assess and evaluate themselves has been shown to be poor.3 Furthermore, physicians typically overestimate their compliance with quality standards.10,12
Physicians with grandfather status may need the recertification process more than their younger peers do. A systematic review4 of 62 studies showed that most found declines in physician performance with greater numbers of years in practice4: decreasing medical knowledge; less adherence to the standards of appropriate diagnosis, screening, preventive care, and therapy; and worse actual health care outcomes. Also, clinical skills tend to decline over time, and more clinical experience does not necessarily lead to better outcomes or improvement of skills.3,4
Although participation in MOC is not required by the ABIM, physicians with grandfather status are being called on to enroll in the program by external stakeholders. Patients are becoming more sophisticated in seeking information that will help them to select physicians who provide high-quality care. Hospitals are required by accreditation bodies, like the Joint Commission, to routinely attest to physicians' competence. Payers are interested in assessing the quality of care so they can distinguish “high-quality physicians.” We believe the MOC process, developed and continually refined by physicians, is a better program than those developed by payers or by the government.
The biggest concerns we hear from physicians about becoming recertified relate to cost, time, relevance, and fear.13,14 The physician in the vignette should understand that the ABIM recommends that physicians maintain the certifications that best reflect their practice; in his case, we would recommend that he maintain his endocrinology certification. We believe that the one-time fee, although considerable ($1,570 to $1,720), is reasonable, since certification lasts for 10 years, resulting in a cost of about $160 per year. The cost of his annual weeklong postgraduate course is much more than that — approximately $1,000 per course per year. Enrollment in MOC provides CME credits for completion of the knowledge self-assessment modules and gives access to performance-improvement modules throughout the 10-year period — at no additional cost.
The ABIM currently has 19 subspecialties and continues to get requests to create more focus areas; this is reflected in the cost. The process of producing, maintaining, and administering the examinations and modules is expensive. Teams of physicians in practice and academia create the examinations and approve all MOC products. Furthermore, to ensure quality, accuracy, relevance, and security, questions for the secure examination and self-assessment modules are updated continually.
In our experience, MOC requires a time commitment, to prepare for the examination and to complete the performance-improvement modules. The fact that the assessment is meaningful to patients, physicians, and other stakeholders requires the ABIM to create a program that is rigorous and robust and that requires real effort by the participant. Indeed, completing the performance-improvement modules can take hours, because the test taker has to gather data, develop and implement a quality-improvement plan, and remeasure performance. For us, preparing for the examination had the added value of providing a means of continuous learning.
Physicians express concern that the examination is about recall but does not give them the chance to use the resources they would use in their practice. The questions in the examination are designed to test diagnostic reasoning and clinical judgment — that is, to test what the certified internist or subspecialist is expected to know without access to medical resources. The examination is continually revised by internists, who select the most relevant questions; in the future, the ABIM may provide resources in which test takers can look up information during the examination.
Finally, fear of failure is a hurdle for physicians with time-unlimited certification. The examination is a high-stakes one, and failure can affect a physician's practice (although physicians with grandfather status do not lose their certification if they fail the examination). It is worth noting that 84% of first-time test takers pass and that the ultimate pass rate is around 95% — evidence that learning and feedback are effective.
The ABIM's MOC program is not perfect, and we on the Board of Directors have been working with input from our diplomates to address many shortcomings of the program. We share the topic areas of the examination questions so that physicians know what will be covered. Given physicians' concerns that the secure examination be relevant to their practices, especially for those who are very specialized, the ABIM is exploring plans to implement a MOC track of focused practice. This track will consist of a program that better aligns with what physicians are doing in practice.
In addition, the ABIM is working to make sure that, if appropriate, multiple resources can be used to complete the MOC process. Many tools are being developed to assist physicians in assessing their performance in practice. Physicians can often use the work they are doing within the MOC program to get credit for other programs important to their practice, thereby reducing redundant effort.
We recommend to our colleague that he take up the ABIM challenge and enroll in MOC. In doing so, he can show his patients, peers, and himself that he is up to date in medical knowledge, participates in quality improvement in his practice, and has met a rigorous, external standard. As a faculty member, he can serve as a role model to students and residents, all of whom will be required to participate in MOC throughout their careers. As public expectations increase and as self-regulation is challenged on the basis of high costs of care and gaps in quality,15 it is time for all “grandfathers” to participate in MOC.
Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
Source Information
From the Department of Medicine, University of Toronto, Toronto (W.L.); and the Department of Medicine, University of California, San Francisco, San Francisco (T.E.K.).
- Recommendation 2
- Recommendation 2Do Not Enroll in the Current MOC Program
We three “grandfathers,” despite having substantially different backgrounds and current roles, strongly support the concept of recertification but cannot advise the physician in the vignette to engage in the American Board of Internal Medicine's (ABIM) maintenance of certification (MOC) program as currently constituted. We express this view with a sense of disappointment but also with hope and expectation that the process will be greatly improved.
From a variety of perspectives, we would be pleased to endorse a recertification process that reasonably and cost-effectively assessed a physician's competence, in terms of knowledge and professionalism, in the skills and tasks appropriate to the physician's specific medical practice. We would be even more confident in this recommendation if there were definitive evidence to show that the actual process of recertification benefitted those who became recertified, as well as their patients. The fact that so few physicians with grandfather status voluntarily choose to be recertified shows that most older, experienced internists do not believe the current recertification process to be a worthwhile method of helping them maintain their competence to care for their patients.13
Although nonrandomized data show an association between initial board certification and improved performance,8,9,16 there are little or no data showing improved outcomes of care related to recertification. Unfortunately, the logical belief that continuing education will improve a physician's knowledge (and hence, performance) has not been confirmed in randomized trials.1,2 Board recertification in internal medicine relies heavily on, and culminates with, physicians' passing a secure examination of knowledge. This strategy is used by many professional boards because of the inconsistency of oral examinations and the psychometric validity ascribed to the written examination.
We do not dispute the correlation of performance on a written examination with other process-performance measures, but we wonder whether the correlation is based on examination scores being a surrogate marker, rather than a definition, of competency. Medicine is replete with surrogate markers that correlate well with an outcome but either are not truly causal or, even if they are causal, may represent misplaced therapeutic targets. Similarly, even if scores on a secure examination are a useful surrogate for overall competence, do the outcomes justify spending tens, if not hundreds, of hours studying to pass an examination that may have little to do with one's practice? Physicians do need a solid knowledge base, but much more importantly, physicians must have a well-developed ability to access and use relevant information resources to support decision making. The question of whether heavy emphasis on increasing personal knowledge will improve outcomes of care can be tested, but so can the question of whether improving one's ability to utilize available information will improve care. In the absence of relevant data, the current approach to recertification relies more on the logical assumption that it may be beneficial than on demonstrated outcomes.
Even if the current approach by the ABIM is correct, we believe that the MOC process falls short in terms of relevance and the time, effort, and expense it requires of candidates. The one-size-fits-all secure examination, which requires many hours of preparation and the review of volumes of material, much of which may be irrelevant to one's daily practice, is a key component of the MOC program. If you fail the examination, you fail to become recertified.
The relevance of MOC should and could be enhanced by customizing the process — by having the candidates identify the most common problems and diagnoses encountered in their practice (e.g., through the review of electronic medical records or billing data) and focusing the study, practice-improvement, and testing materials on the customized practice (presumably from a bank of available material). The customized learning agenda could be supplemented by a limited amount of core material focusing on a small set of must-not-miss diagnoses and management priorities pertinent to all internists, rather than requiring all candidates to reread about all of internal medicine in an unfocused fashion. The “final examination” (which could still be used, if data could show its relevance) should allow candidates to access the information typically available to a practitioner so that real-world skills, not factual recall, are assessed.
We do not oppose tests. Rather, we fear that continued overreliance on factual recall runs the risk that physicians will be inappropriately encouraged to trust their “recertified memories” rather than reminded that they should always consult easily accessible, authoritative resources to make sure they are doing the right thing.
In an ABIM survey of board-certified internists whose certification was expiring, only 38% of respondents felt that the requirements for recertification were appropriate; 41% did not.8 Among the respondents, only 53% had completed the requirements for MOC. The major reasons for not participating in the process were the time required, the lack of relevance to their practice, the expense of the process, and the fact that recertification was not required for employment. These results suggest that it might be better for the ABIM to address these issues definitively so as to enhance the legitimacy of recertification for those who must participate, before embarking on a campaign to convince physicians with time-unlimited certification to participate voluntarily.
The ABIM has had, for all intents and purposes, an absolute monopoly on certification and recertification in internal medicine and its subspecialties. Since the advent of recertification, the financial reserves of the ABIM and the ABIM Foundation have increased substantially; in 2006 and 2007, the ABIM transferred $13 million to its foundation.17 Reserves of this magnitude create a responsibility both to reduce the costs of recertification and to fund research and development to improve the certification and recertification processes. We contend that this goal should become the stated priority of the ABIM, pursued explicitly and aggressively in a manner that the public can trust and the profession should endorse.
It can be argued that recertification captures the high ground and helps forestall more onerous and capricious external regulation; we hope this can be made the case. However, the low rate of voluntary recertification confirms that physicians with grandfather status are not convinced of the value of the process. We believe that the true success of recertification will be defined not by whether physicians with time-limited certification maintain their certification but by whether the ABIM can improve and reform the process of recertification sufficiently so that all physicians will choose to become recertified in a truly voluntary fashion. Our view is that the entire internal medical community would be delighted to work with the ABIM to make recertification a more relevant, evidence-based, cost-effective process.
In its current form, recertification is great in theory but disappointing in practice. As a result, we would reluctantly advise our colleague to spend his time and money elsewhere.
Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
Source Information
From the Columbia University Medical Center, New York (L.G.); Harvard Medical School and Massachusetts General Hospital — both in Boston (A.H.G.); and Eureka Internal Medicine, Eureka, CA (B.K.).
- References
References
1
Mazmanian PE ,Davis DA . Continuing medical education and the physician as a learner: guide to the evidence. JAMA 2002;288:1057-1060
CrossRef | Web of Science | Medline2
Davis DA ,Thomson MA ,Oxman AD ,Haynes RB . Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274:700-705
CrossRef | Web of Science | Medline3
Davis DA ,Mazmanian PE ,Fordis M ,Van Harrison R ,Thorpe KE ,Perrier L . Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006;296:1094-1102
CrossRef | Web of Science | Medline4
Choudhry NK ,Fletcher RH ,Soumerai SB . Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med 2005;142:260-273
Web of Science | Medline5
Turchin A ,Shubina M ,Chodos AH ,Einbinder JS ,Pendergrass ML . Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus. Circulation 2008;117:623-628
CrossRef | Web of Science | Medline6
Holmboe ES ,Wang Y ,Meehan TP , et al. Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries. Arch Intern Med 2008;168:1396-1403
CrossRef | Web of Science | Medline7
Norcini JJ ,Kimball HR ,Lipner RS . Certification and specialization: do they matter in the outcome of acute myocardial infarction? Acad Med 2000;75:1193-1198
CrossRef | Web of Science | Medline8
Chen J ,Rathore SS ,Wang Y ,Radford MJ ,Krumholz HM . Physician board certification and the care and outcomes of elderly patients with acute myocardial infarction. J Gen Intern Med 2006;21:238-244
CrossRef | Web of Science | Medline9
Pham HH ,Schrag D ,Hargraves JL ,Bach PB . Delivery of preventive services to older adults by primary care physicians. JAMA 2005;294:473-481
CrossRef | Web of Science | Medline10
Holmboe ES ,Meehan TP ,Lynn L ,Doyle P ,Sherwin T ,Duffy FD . Promoting physicians' self-assessment and quality improvement: the ABIM diabetes practice improvement module. J Contin Educ Health Prof 2006;26:109-119
CrossRef | Web of Science | Medline11
Audet AM ,Doty MM ,Shamasdin J ,Schoenbaum SC . Measure, learn, and improve: physicians' involvement in quality improvement. Health Aff (Millwood) 2005;24:843-853
CrossRef | Web of Science | Medline12
McGlynn EA ,Asch SM ,Adams J , et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003;348:2635-2645
Full Text | Web of Science | Medline13
Lipner RS ,Bylsma WH ,Arnold GK ,Fortna GS ,Tooker J ,Cassel CK . Who is maintaining certification in internal medicine -- and why? A national survey 10 years after initial certification. Ann Intern Med 2006;144:29-36
Web of Science | Medline14
Brennan TA . Recertification for internists -- one “grandfather's” experience. N Engl J Med 2005;353:1989-1992
Full Text | Web of Science | Medline15
Shaw K ,Cassel CK ,Black C ,Levinson W . Shared medical regulation in a time of increasing calls for accountability and transparency: comparison of recertification in the United States, Canada, and the United Kingdom. JAMA 2009;302:2008-2014
CrossRef | Web of Science | Medline16
Haas JS ,Orav EJ ,Goldman L . The relationship between physicians' qualifications and experience and the adequacy of prenatal care and low birthweight. Am J Public Health 1995;85:1087-1091
CrossRef | Web of Science | Medline17
Prospect Research Online (PRO) home page. (Accessed January 20, 2010, at http://www.iwave.com/.)
- Citing Articles (9)
Citing Articles
1
Brian J. Hess, Weifeng Weng, Eric S. Holmboe, Rebecca S. Lipner. (2011) The Association Between Physiciansʼ Cognitive Skills and Quality of Diabetes Care. Academic Medicine1
CrossRef2
Cassel, Christine K., Reuben, David B., . (2011) Specialization, Subspecialization, and Subsubspecialization in Internal Medicine. New England Journal of Medicine 364:12, 1169-1173
Full Text3
Kevin C. Chung, Philip J. Clapham, Donald H. Lalonde. (2011) Maintenance of Certification, Maintenance of Public Trust. Plastic and Reconstructive Surgery 127:2, 967-973
CrossRef4
John M. James, Mark Corbett. (2010) The American Board of Allergy and Immunology maintenance of certification program: “To do or not to do? That is the question.”. Annals of Allergy, Asthma & Immunology 105:6, 485-488
CrossRef5
Gary J. Becker, Matthew A. Mauro, Jeanne M. Laberge, Jennifer L. Bosma, David Laszakovits. (2010) Should “Dr. S” Maintain Her American Board of Radiology Subspecialty Certificate in Vascular and Interventional Radiology?. Journal of Vascular and Interventional Radiology 21:9, 1323-1329
CrossRef6
R. D. Zimmerman. (2010) Maintenance of Certification: Where We Are, Why We Are Here, and Where We Need to Be. American Journal of Neuroradiology 31:8, 1373-1376
CrossRef7
(2010) The ABIM and Recertification. New England Journal of Medicine 362:25, 2428-2430
Full Text8
Kritek, Patricia A., Drazen, Jeffrey M., . (2010) American Board of Internal Medicine Maintenance of Certification Program — Polling Results. New England Journal of Medicine 362:15,
Full Text9
Drazen, Jeffrey M., Weinstein, Debra F., . (2010) Considering Recertification. New England Journal of Medicine 362:10, 946-947
Full Text
- Letters
