Unreliability of Experts

Forensic scientists had a strong tendency to overestimate their ability to draw correct conclusions from evidence.

Erik Eckholm reports that expert testimony about bite marks, considered trustworthy enough to send people to prison, has turned out to be extremely unreliable. Mark Cheney was sentenced to life for murder "after a dental expert testified that it was virtually certain that his teeth had caused marks on an arm of the victim, a drug dealer who was stabbed to death." After 28 years in prison, he was recently freed because "studies have shown that dental experts cannot reliably claim that a bite wound was caused by a particular individual. They cannot even consistently agree on whether wounds were caused by bites at all."

Forensic science more broadly is in turmoil as prosecutors, defense lawyers and judges confront evidence that many long-used methods, like handwriting analysis and microscopic hair comparisons, were based more on tradition than science and do not hold up under scrutiny. Even fingerprint and certain kinds of DNA matches are not quite as certain as many once believed, scientists say. ...
Put under rigorous scrutiny, some forensic tools, including comparisons of lead chemistry in bullets and the matching of aural voice prints, have already been largely discarded. The accuracy of many supposed signs of arson, like burn patterns that seemed to be caused by a liquid, has also been disproved.

This year, the Federal Bureau of Investigation admitted that examiners at its vaunted crime lab had for decades overstated the reliability of microscopic hair matches.
- Erik Eckholm, Lives in Balance, Texas Leads Scrutiny of Bite-Mark Forensics, NYT, Dec. 12, 2015

The sheer number of people who were imprisoned using faulty science called into question the premise of forensics itself. Just how reputable were these methods, and what exactly were expert witnesses’ opinions based on?

In 2009, the National Academy of Sciences sought to answer those questions with a groundbreaking, and damning, report. Its authors found that many forensic disciplines — including the analysis of blood spatter, hairs, bite marks, shoe and tire impressions and handwriting — were not as scientific as they often purported to be. Rather than being firmly grounded in hard data and rigorous, peer-reviewed research, many of these disciplines relied on the individual judgments of practitioners. The report included a sobering appraisal of bloodstain interpretation. Analysts’ opinions were often “more subjective than scientific,” its authors warned, and open to “context bias.” They noted that “some experts extrapolate far beyond what can be supported.”

A 2009 report by the National Academy of Sciences found that “the opinions of bloodstain-pattern analysts are more subjective than scientific,” and, “The uncertainties associated with bloodstain pattern analysis are enormous.”

And yet judges in many states have accepted these experts’ testimony as scientifically valid — not because of any concrete evidence that it is, but because other courts have accepted it before. In other words, it’s a good bet that there are other Joe Bryans sitting in prisons around the country because of highly unreliable forensic testimony.

That unreliability is not unique to bloodstain-pattern analysis. As DNA testing has revolutionized forensic science and helped to exonerate hundreds of wrongfully convicted people, it has also shined a light on the inadequacy of earlier methods. The National Academy of Sciences report found significant problems with the analysis of bite marks, tire treads, arson and hair samples. In 2015, the F.B.I. released an initial review of hundreds of convictions it had won and found that over two decades, the bureau’s “elite” forensic hair-sample analysts testified wrongly in favor of the prosecution 96 percent of the time. Thirty-two of the defendants in those cases were sentenced to death, and 14 of those were executed or died in prison.
- The Editorial Board, Bad Blood, NYT, May 31, 2018

Show php error messages