Limitations of fingerprint identification recognized
A Baltimore County judge has ruled that fingerprint evidence, a mainstay of forensics for nearly a century, is not reliable enough to be used against a homicide defendant facing a possible death sentence – a finding that national experts described yesterday as unprecedented and potentially far-reaching.
Baltimore County Circuit Judge Susan M. Souder’s order bars prosecutors from using at trial the partial fingerprints lifted from the Mercedes of a Security Square Mall merchant who was fatally shot last year during an attempted carjacking at the shopping center. Prosecutors say the fingerprints – as well as those found in a stolen Dodge Intrepid in which witnesses said the shooter fled the mall parking lot – link a 23-year-old Baltimore man to the killing.
In her ruling, Souder outlined the long history of fingerprinting as a crime-solving tool but says that such history “does not by itself support the decision to admit it.” In explaining her reasoning in a 32-page decision, the judge leaned heavily on the case of an Oregon lawyer mistakenly linked through fingerprint analysis to the 2004 Madrid train bombings.
With defendant Bryan Keith Rose scheduled to go to trial today in Towson, prosecutors and defense attorneys in the capital case declined to comment yesterday on the judge’s ruling.
But others who have researched the issue and litigated cases involving fingerprint evidence said the decision – if it stands up on appeal – could have implications that reach even beyond the use of fingerprint evidence in criminal courts.
“The repercussions are terrifically broad,” said David L. Faigman, a professor at the University of California’s Hastings College of the Law and an editor of Modern Scientific Evidence: The Law and Science of Expert Testimony.
“Fingerprints, before DNA, were always considered the gold standard of forensic science, and it’s turning out that there’s a lot more tin in that field than gold,” he said. “The public needs to understand that. This judge is declaring, not to mix my metaphors, that the emperor has no clothes.
“There is a lot of forensic science that is considered second to fingerprinting,” Faigman added, mentioning firearms and toolmark analysis, hair identification, bite pattern analysis and evidence used in arson investigations as examples. “If fingerprinting turns out to not be so good, people could start questioning that science as well.”
The technology has come under scrutiny in recent years.
Stephan Cowans, a Boston man who spent six years in prison for the shooting of a police sergeant, was released in 2004 after the discovery that the fingerprint used to convict him was not his.
That same year, the FBI mistakenly linked Brandon Mayfield, an Oregon lawyer, to a fingerprint lifted off a plastic bag of explosive detonators found in Madrid after commuter train bombings there killed 191 people. Two weeks after Mayfield’s arrest, Spanish investigators traced the fingerprint to an Algerian man.
The U.S. Justice Department issued a formal apology last year to Mayfield and awarded him $2 million.
Souder, the Baltimore County judge, referred repeatedly in her opinion to that case, as well as a March 2006 report from the Justice Department’s internal investigators on the FBI’s handling of the matter.
In the Mayfield case, three FBI fingerprint examiners and an independent court-appointed fingerprint analyst determined that the fingerprint on the bag of detonators belonged to the Oregon attorney.
“Up to that point, [the government] had maintained that if you have a competent examiner, the technique of fingerprinting can’t produce a misidentification. Mayfield exposed that as a fallacy,” said Robert Epstein, an assistant federal defender in Philadelphia who in 1998 was among the first lawyers to challenge the reliability of latent fingerprint identifications.
In the Baltimore County murder case, defense attorneys challenged the admissibility of fingerprint evidence that linked Rose to the killing Jan. 5, 2006, of Warren T. Fleming, the owner of a Cingular Wireless store at Security Square Mall.
Rose was arrested 13 days after the shooting after police received a call saying the “ringleader” of the attempted carjacking was a man called “Sticky,” a nickname that was recognized as Rose’s, according to court records.
At a pretrial hearing in May, prosecutors argued that fingerprint evidence has been accepted by the courts and relied upon for nearly 100 years. Defense attorneys countered that there is no similar history of subjecting the evidence to scientific review.
“The state is correct that fingerprint evidence has been used in criminal cases for almost a century,” Souder, the judge, wrote in her decision. “While that fact is worthy of consideration, it does not prove reliability. For many centuries, perhaps for millennia, humans thought that the earth was flat.”
She criticized the common method of fingerprinting as overly subjective and lacking in standards. She discounted the proficiency tests that the state’s expert witness testified about. And she characterized as “neither credible nor persuasive” testimony that fingerprinting is an infallible methodology.
Souder acknowledged that the crime lab technicians’ conclusions that Rose’s fingerprints match those found on the cars “appear to be the heart of the state’s case.”
Issued late Friday afternoon, the judge’s decision has already attracted significant attention within the world of forensic sciences. The decision was included in yesterday’s edition of The Detail, an e-mail newsletter distributed by fingerprint examiners.
The FBI has consistently refused to undertake any scientific tests of fingerprint identification, although they seem more open to it after the errors mentioned above. (The FBI also, FWIW, refuses to videotape their interrogations of suspects.)
The fingerprint problem, BTW, is not with a complete and clear fingerprint—those are seldom found at a crime scene—but with partial fingerprints and smudged fingerprints, which are more typical. Attaching those to a particular fingerprinted person is a matter of judgment, and judgments can be wrong.
The New Scientist notes in an article:
A century of fingerprint practice went by before the first attempt to measure its accuracy was published earlier this year (Journal of Forensic Identification, vol 56, page 55). The FBI recently laid out an ambitious research agenda to place fingerprinting on a more scientific footing, but only after it was forced to admit that a false fingerprint identification had wrongly implicated an Oregon lawyer in the Madrid train bombings of 2004.
“Many forensic disciplines appear ill equipped or disinclined to take a rigorous empirical approach”
A good illustration of the lack of research underpinning much forensic science is the use by the FBI of an esoteric technique called comparative bullet lead analysis (CBLA), which the FBI discontinued in 2004. When bullet fragments were too badly damaged to allow for the examination of striation marks, which can match a bullet to a particular gun, the FBI would instead analyse their metallurgical composition. If bullets from a crime scene contained similar proportions of trace elements to bullets found in a suspect’s possession, FBI scientists argued this was convincing evidence that they came from the same source.
CBLA rested on untested assumptions about the diversity of lead sources and their chemical consistency. The validity of these assumptions was only addressed in the 1990s. It turned out that lead sources are neither entirely heterogeneous nor internally homogeneous (New Scientist, 20 April 2002, p 4). In 2004, the FBI discontinued CBLA after a critical report from the National Research Council (NRC).
What does this say about the state of forensic science? At a National Academy of Sciences (NAS) colloquium in Washington DC last November, Ronald Singer, former president of the American Academy of Forensic Sciences, said the CBLA episode showed forensic science was “self-correcting”. That is a generous interpretation, considering that it took more than 30 years for the FBI to fund the NRC report.
Here’s another way of viewing it: the FBI’s use of CBLA turned the usual relationship between law and science on its head. Rather than performing scientific research to determine what can responsibly be said about a form of evidence, and then offering such testimony in court, it appears the FBI testified first and did the science later.
And a New Scientist editorial speaks to the issue:
In June 2002, a corpse turned up near Las Vegas. A fingerprint taken from the body matched one from Kathleen Hatfield of Sonoma county, California. Hatfield’s mother was notified, funeral arrangements were made and her grave dug. But before the body could be interred, Hatfield turned up alive and well.
Neville Lee was arrested in 1991 in Nottinghamshire, UK, for allegedly raping an 11-year-old girl. The case rested on fingerprint evidence. He was assaulted in jail and his home was wrecked by vigilantes. It was only after another person confessed to the crime that Lee was released.
Last year, an Oregon lawyer named Brandon Mayfield was held in connection with the Madrid bombings after his fingerprint was supposedly found on a bag in the Spanish capital. Only after several weeks did the Spanish police attribute the print to Ouhnane Daoud, an Algerian living in Spain.
These are just 3 of 22 cases of mismatched fingerprints collected by Simon Cole, a criminologist at the University of California, Irvine (Journal of Criminal Law & Criminology, vol 95, p 985). In many of these cases, people were wrongly convicted and spent time in jail. Yet despite all this evidence to the contrary, fingerprint examiners still say their technique is infallible. What is going on?
Fingerprint matching is undoubtedly a valuable tool for catching criminals but it suffers from one major flaw: nobody knows how often fingerprint examiners make a wrong call. In US federal courts, there are standards that scientific evidence must live up to. One of these “Daubert criteria” is that techniques must have a known rate of error. Yet, somehow, fingerprint matching has not been required to meet this standard, and the Department of Justice has shied away from systematic research that would reveal the error rate (New Scientist, 31 January 2004, p 6).
US fingerprint examiners make a distinction between “methodological error” and “practitioner error”. The first enshrines the notion that if an examination is done correctly the error will always be zero. Practitioner error is an admission that people can get things wrong, but FBI examiners argue that this error should be excluded from the Daubert criteria.
It is a seminal idea in science that all errors should be understood and quantified, whether they come from the method used, the equipment or human errors. But ultimately, it is the overall error that counts. Splitting error in two, as examiners do, may be useful for calculating overall risk, but applying Daubert criteria to only one of them makes no sense.
Take automated fingerprint recognition, which is being adapted for use in passports, as a digital identity for banking and even as an alternative to car keys. Just deciding which parts of the hardware and software play the role of the methodology and which the practitioner would tax a philosopher. But all that really counts is the overall error rate: how often do immigration officials fail to recognise you, how often are you prevented from withdrawing cash or locked out of your car.
The error rates of these automated systems are well understood, and are likely to cause plenty of problems for us all as the use of biometrics spreads (see “Privacy and prejudice: whose ID is it anyway”). Understanding the error rates in judicial fingerprint matching will also create problems: any criminal whose conviction relied heavily on fingerprint evidence is likely to appeal. But that is no reason to ignore the issue.
Innocent people are being wrongly convicted, and the numbers could be substantial. One disturbing aspect of Cole’s list is that two-thirds of the mistakes emerged only in exceptional circumstances, such as an unexpected confession. How many other errors have failed to come to light from mundane cases?
Ignoring the existence of error also prevents fingerprint analysis being improved. Examiners’ decisions are clearly influenced by what they are told before they examine a potential match (see “Examiners’ objectivity called into question”). Such biases will remain so long as examiners refuse to acknowledge them.
As more mistakes are exposed and defence lawyers intensify their challenges, it is only a matter of time before judges and juries reject fingerprint evidence. That would be a dreadful waste of a powerful tool. Far better to do some research now so we know once and for all how confident we can be about fingerprints.
Here’s the bit on “Examiner’s objectivity called into question”:
Examiners’ objectivity called into question
Fingerprint examiners can be heavily influenced by external factors when making judgements, according to research in which examiners were duped into thinking matching prints actually came from different people.
The study, by Itiel Dror and Ailsa Péron at the University of Southampton, UK, suggests that subjective bias can creep into situations in which a match between two prints is ambiguous. So influential can this bias be that experts may contradict evidence they have previously given in court. “I think it’s pretty damning,” says Simon Cole, a critic of fingerprint evidence at the University of California, Irvine.
Dror and Péron arranged for five fingerprint examiners to determine whether a “latent” print matched an inked exemplar obtained from a suspect. A latent print is an impression left at a crime scene and visualised by a technique such a dusting. The examiners were also told by a colleague that these prints were the same ones that had notoriously and incorrectly been matched by FBI fingerprint examiners last year in the investigation into the Madrid bombings. That mismatch led to Portland lawyer Brandon Mayfield being incorrectly identified as one of the bombers.
What the three examiners didn’t know was that the prints were not from the bombing case at all. Each pair of prints, different for each examiner, had previously been presented in court by that same expert as a definite match.
Yet in the experiment only one of the experts correctly deemed their pair as matches. “The other four participants changed their identification decision from the original decision they themselves had made five years earlier,” says Dror. Three claimed the pair were a definite mismatch, while the fourth said there was insufficient information to make a definite decision. Dror will present the results at the Biometrics 2005 conference in London next month.
One solution, says Cole, might be for each forensics lab to have an independent official who distributes evidence anonymously to the forensic scientists. This would help to rule out any external case-related influences by forcing the scientists to work in isolation, knowing no more about each case than is necessary. At the moment fingerprint examiners asked to verify decisions made by their colleagues do not receive the evidence “blind”. They already know the decision colleagues have made.
Paul Chamberlain, a fingerprint examiner with the UK Forensic Science Service who has more than 23 years experience, says: “The FSS was aware of the need for a more robust scientific approach for fingerprint comparison.” But he questions the relevance of the expert study. “The bias is unusual and it is, in effect, an artificial scenario,” he says.
And this New Scientist article spells out the number of misidentifications that are likely:
A high-profile court case in Massachusetts is once again casting doubt on the claimed infallibility of fingerprint evidence. If the case succeeds it could open the door to numerous legal challenges.
The doubts follow cases in which the testimony of fingerprint examiners has turned out to be unreliable. The most high-profile mistake involved Brandon Mayfield, a Portland lawyer, who was incorrectly identified from crime scene prints taken at one of the Madrid terrorist bombings on 11 March 2004. Despite three FBI examiners plus an external expert agreeing on the identification, Spanish authorities eventually matched the prints to an Algerian.
Likewise, Stephan Cowans served six years in a Massachusetts prison for shooting a police officer before being released last year after the fingerprint evidence on which he had been convicted was trumped by DNA.
No one disputes that fingerprinting is a valuable and generally reliable police tool, but despite more than a century of use, fingerprinting has never been scientifically validated. This is significant because of the criteria governing the admission of scientific evidence in the US courts.
The so-called Daubert ruling introduced by the Supreme Court in 1993 set out five criteria for admitting expert testimony. One is that forensic techniques must have a known error rate, something that has never been established for fingerprinting.
The reliability of fingerprinting is at the centre of an appeal which opened earlier this month at the Massachusetts Supreme Court in Boston. Defence lawyers acting for Terry Patterson, who was convicted of murdering an off-duty policeman in 1993, have launched a so-called “interlocutory” appeal midway through the case itself to test the admissibility of fingerprinting. Patterson’s conviction relies heavily on prints found on a door of the vehicle in which the victim died.
A key submission to the appeal court is a dossier signed by 16 leading fingerprint sceptics, citing numerous reasons for challenging the US Department of Justice’s long-standing contention that fingerprint evidence has a “zero error rate”, and so is beyond legal dispute. Indeed, fingerprint examiners have to give all-or-nothing judgements. The International Association for Identification, the oldest and largest professional forensic association in the world, states in a 1979 resolution that any expert giving “testimony of possible, probable or likely [fingerprint] identification shall be deemed to be engaged in conduct unbecoming”.
Material in the dossier includes correspondence sent to New Scientist in 2004 by Stephen Meagher of the FBI’s Latent Fingerprint Section in Quantico, Virginia, author of a pivotal but highly controversial study backing fingerprinting. The so-called “50K study” took a set of 50,000 pre-existing images of fingerprints and compared each one electronically against the whole of the data set, producing a grand total of 2.5 billion comparisons. It concluded that the chances of each image being mistaken for any of the other 49,999 images were vanishingly small, at 1 in 1097.
But Meagher’s study continues to be severely criticised. Critics say that showing an image is more like itself than other similar images is irrelevant. The study does not mimic what happens in real life, where messy, partial prints from a crime scene are compared with inked archive prints of known criminals.
When New Scientist highlighted these issues in 2004 (31 January 2004, p 6), Meagher’s response to our questions arrived too late for publication. He wrote that critics misunderstood the purpose of his study, which sought to establish that individual fingerprints are effectively unique – unlike any other person’s print. “This is not a study on error rate, or an effort to demonstrate what constitutes an identification,” he wrote (the letter can be read at www.newscientist.com/article.ns?id=dn7983). By the time New Scientist went to press, the FBI had not responded to our requests for comment.
But critics of fingerprinting have seized on this admission and included it in the dossier as evidence that the 50K study doesn’t back up the infallibility of fingerprinting. “It shows that the author of the study says it doesn’t have anything to do with reliability,” says Simon Cole, a criminologist at the University of California, Irvine and one of the 16 co-signatories of the dossier.
Cole says that Meagher’s replies to New Scientist demolish claims by the courts, the FBI and prosecution lawyers that the 50K study is evidence of infallibility. He says the letter has already helped to undermine fingerprint evidence in a recent case in New Hampshire.
Whatever the decision in the Patterson case, the pressure is building for fingerprinting’s error rate to be scientifically established.
One unpublished study may go some way to answering the critics. It documents the results of exercises in which 92 students with at least one year’s training had to match archive and mock “crime scene” prints. Only two out of 5861 of these comparisons were incorrect, an error rate of 0.034 per cent. Kasey Wertheim, a private consultant who co-authored the study, told New Scientist that the results have been submitted for publication.
But evidence from qualified fingerprint examiners suggests a higher error rate. These are the results of proficiency tests cited by Cole in the Journal of Criminal Law & Criminology (vol 93, p 985). From these he estimates that false matches occurred at a rate of 0.8 per cent on average, and in one year were as high as 4.4 per cent. Even if the lower figure is correct, this would equate to 1900 mistaken fingerprint matches in the US in 2002 alone.