Book Review

Scientific Errors and Controversies in the U.S. HIV/AIDS Epidemic: How They Slowed Advances and Were Resolved

N Engl J Med 2008; 359:981-982August 28, 2008

Article

Scientific Errors and Controversies in the U.S. HIV/AIDS Epidemic: How They Slowed Advances and Were Resolved
By Scott D. Holmberg. 228 pp. Westport, CT, Praeger, 2008. $49.95. ISBN: 978-0-313-34717-7

This book is an autopsy of errors. Drawing on his 20 years as chief of epidemiology in the division for the prevention of the human immunodeficiency virus (HIV) and the acquired immunodeficiency syndrome (AIDS) at the Centers for Disease Control and Prevention (CDC), Scott Holmberg dissects “a litter of discarded theories and opinions” about the epidemic to learn how they arose, were propagated, and were eventually corrected. The result is an instructive illustration of how errors afflict science at all levels, and it provides some thoughtful, if overly optimistic, observations on how to reduce their incidence.

Holmberg highlights several types of errors, ranging from standard statistical problems to more intractable cognitive biases. “Ascertainment bias” occurs when observers draw conclusions about an entire group from an atypical sample. An example of this type of bias occurred early in the epidemic, when women typically presented with HIV later than did gay men. From this, researchers erroneously concluded that women as a whole had a shorter conversion period from HIV to AIDS than men.

“Ecologic fallacies” work the other way around, inspiring false conclusions about individual people based on the characteristics of the group as a whole. These fallacies often involve mistaking a coincidence for a causal connection. For instance, a journalist noted an overlap between countries where a polio vaccine was administered in the 1950s and where HIV first spread a decade later. With scant evidence, he hypothesized in a popular book that the vaccine had been cultured on monkey kidneys infected with simian immunodeficiency virus, which then evolved into HIV in humans. It ultimately took an international scientific conference to discredit the theory.

Another vexing challenge involves “confounding,” when an apparent relationship between two variables is actually the result of a third, unmeasured variable. For instance, early HIV researchers mistakenly thought that illicit use of inhaled nitrites (also called “poppers”) raised the risk of developing Kaposi's sarcoma, an AIDS-opportunistic infection. It was later determined that popper use was merely a marker for unsafe sexual behavior, which was inducing the spread of a newly discovered human herpesvirus, HHV-8. This herpesvirus, and not drug use, is now known to be the causative agent of Kaposi's sarcoma.

These and other types of error are common in research and can be minimized through careful study design. More difficult to control are analytic errors, cognitive biases, and simple human blunders. Analytic errors stem in part from the growing complexity of new laboratory and statistical techniques. Holmberg justly fears that most medical personnel are not well trained to understand such tools and thus are unable to effectively question their applications or the assumptions they embody. Cognitive bias takes many forms, perhaps none as common as sheer wishful thinking, which kept many theories alive well after the evidence suggested they were fatally flawed. Human error has also played a continual part. Holmberg documents how contaminated samples at a prestigious laboratory were to blame for the contentious early dispute over which virus was the cause of AIDS.

In his fine closing chapter, the author proposes several reforms: reversing the recent politicization of decision making in public health agencies, improving the education of medical and public health students in epidemiology and statistics, and imposing quality-control standards on journal peer reviews. Each could help to reduce the rate of error. Holmberg's larger project, however, seems to be to eliminate error altogether. He argues that it “can only delay the ultimate discovery . . . of reality.” Many scientists would take issue with this. Science may be a grand quest for truth, but the scientific process itself is largely a search for error. Even a mistaken theory can hasten useful discoveries by inspiring other researchers to correct it.

Ironically, the book itself at times embodies yet another type of error — hindsight bias. While some errors in HIV research were indeed avoidable, many were probably inescapable, as researchers, working under intense public pressure, struggled to make sense of an unprecedented virus. It is unrealistic to imply that the source of error in every case could have been foreseen at the outset. And although Holmberg is forthright about his own errors, he leaves the impression that the CDC as an agency could do no wrong. He variously typifies the work of his colleagues as “seminal,” “paradigm,” or “perspicacious,” while describing those who differed with the CDC as “irritating.” The book also suffers from a more prosaic form of error; typographical errors abound, including some that change the meaning of sentences. The writing is uneven, occasionally awkward, and in places almost unintelligible.

Flaws aside, however, this book offers a well-documented account that will be of interest not only to HIV and AIDS scholars but to anyone concerned with the role of error in modern medicine.

Keith Hansen, M.P.A., J.D.
World Bank, Washington, DC 20001
khansen@worldbank.org