In this second installment of our series on medical malpractice in the media, we’ll take a closer look at the study which sparked the most recent wave of news coverage. In the study, Martin Makary and Michael Daniel argue that lethal medical errors occur much more frequently than commonly believed. They provide a revised estimate of the rate of incidence of medical errors, which is an average of the rate found in four previous studies. The dataset this argument rests on is deeply flawed.

Death certificates often fail to convey the role that medical errors play in causing death, so those who study medical errors have devised systems by which errors can be detected indirectly. These systems work by identifying markers with which medical errors are correlated in medical records. Researchers then survey the available set of patient records for those markers and examine the cases that bear them to determine the role of medical errors in causing death.

The largest study in Makary and Daniel’s meta-analysis looked for markers called Patient Safety Indicators (PSIs), defined as top-level medical conditions “of concern to patient safety,” in a database of 37 million patient admissions. The size of the data set precluded analysis of the individual medical records. Accordingly, the deaths the study identified could only be said to be “potentially attributable” to PSIs, which are by definition only potentially preventable.

The other three studies Makary and Daniel evaluated used a system called the Global Trigger Tool (GTT), which looks for a much larger number of more specific markers, and is not limited to medical conditions. The GTT does not assess preventability of the adverse events it flags, so a physician must review GTT-identified case records for a study to produce data on preventable events. This analysis appears to be consistent among two of the three GTT studies, but the study that found the highest rate of adverse events (twice the average of the other studies) featured no such analysis. Additionally, all three GTT studies had sample sizes too small to apply their findings to the 35 million patient admissions in 2013, as Makary and Daniel do.

Another major weakness of the studies Makary and Daniel considered involves the populations each study surveyed. Two of the papers sampled Medicare patients—a population that has been found to experience patient safety incidents more often than members of other patient groups. All four of the papers surveyed only deaths in the inpatient context, meaning deaths that occurred in the home-care and nursing home contexts were not represented.

Although medical malpractice is a serious concern for the community, it is important that the public is not misled. Makary and Daniel’s assertion that lethal medical error is far more prevalent than commonly believed is based on a figure that is by their own admission inaccurate when applied to the broader US population. Unfortunately for the general public, this weakness went unreported in the news articles reporting the study.