The Permanent Record / Dossier · Biometrics & Identity
The False Match Rate
Face recognition does not tell anyone who you are. It tells them who you look most like, and federal testing shows how often "most like" is somebody else.
In 2019 the National Institute of Standards and Technology ran 189 face recognition algorithms against millions of photos and found that the rate of false matches often varied by a factor of 10 to beyond 100 depending on the race, sex and age of the people in the pictures. The error has a technical name. When police treat its output as an identification, it also has a human one. In Detroit it was Robert Williams, arrested on his front lawn for a theft a computer search had pointed at him.
Join the discussion →
Every face recognition system answers the same narrow question. It takes two images, reduces each face to a set of numbers, and produces a score for how similar the two sets are. If the score clears a threshold that someone chose, the system calls it a match. Nothing in that process establishes who anyone is. It establishes that two pictures are alike enough, by one vendor's arithmetic, to be worth a second look.
That difference is the whole story of this entry. A score above the threshold is a probability dressed up as a fact. Who is on the receiving end of the error depends on the algorithm, on the photo, and, federal testing shows, on what the person in the photo looks like.
Two Kinds of Wrong
NIST, the federal agency that has tested face recognition algorithms for vendors and governments for years, sorts errors into two kinds. A false negative is the failure to associate one person in two images: the system looks at two photos of you and says they are different people. A false positive is the erroneous association of two different people: the system looks at you and a stranger and says they are the same.
The two errors trade against each other through the threshold. Raise it, and the system makes fewer false matches but misses more true ones. Lower it, and it finds more true matches while pulling in more strangers. NIST's glossary defines the false match rate plainly as the proportion of impostor comparisons producing false matches, an impostor here meaning simply a photo of somebody else.
NIST also separates the two ways the technology is used. One-to-one verification compares your face against a single claimed identity, the way a phone or a passport gate does: are you who you say you are? One-to-many identification searches one face against a whole gallery, a database of mugshots or licence photos, and returns the people who look most like it. NIST measures the first with a false match rate and the second with a false positive identification rate. The distinction matters because the consequences are not symmetrical. A false negative at a passport gate means a second attempt. A false positive in a mugshot search means a stranger's name on an investigator's desk.
A false negative at a passport gate means you try again. A false positive in a mugshot search means a stranger's name lands on a detective's desk.
What NIST Found
In December 2019 NIST published Interagency Report 8280, the third part of its Face Recognition Vendor Test, devoted entirely to demographic effects. It processed 18.27 million images of 8.49 million people through 189 mostly commercial algorithms from 99 developers. The photographs came from four sources: domestic mugshots, application photos from people seeking immigration benefits, visa photos, and border-crossing images. The mugshots carried race metadata; the others carried country of birth, which NIST used as a proxy, limited to 24 countries in seven regions with lower levels of long-distance immigration.
Its main result was about false positives. Contemporary algorithms, the report said, exhibit demographic differentials of various magnitudes, and false positive differentials are much larger than those for false negatives and exist broadly, across many but not all of the algorithms tested. Across demographics, false positive rates often varied by factors of 10 to beyond 100 times. False negatives tended to be more algorithm-specific and often varied by factors below 3.
The details depended on the photos. In the higher-quality application photos, false positive rates were highest in West and East African and East Asian people and lowest in Eastern European individuals, a gap NIST described as generally large, with a factor of 100 more false positives between countries. A number of algorithms developed in China reversed part of that pattern, giving low false positive rates on East Asian faces. In the domestic law-enforcement mugshots, the highest false positives were in American Indians, with elevated rates in African American and Asian populations, the relative ordering depending on sex and varying by algorithm.
Sex mattered too. NIST found false positives between 2 and 5 times higher in women than in men, the multiple varying with algorithm, country of origin and age, and present for most algorithms and datasets. It found elevated false positives in the elderly and in children, largest at the oldest and youngest ages and smallest in middle-aged adults.
The report was careful about one thing the headlines were not. Accuracy varied enormously from developer to developer, and the most accurate algorithms produce many fewer errors, so they can be expected to show smaller demographic differentials. That is a finding about the market, not a reassurance. A police department does not buy "face recognition." It buys one vendor's algorithm, set to one threshold, run against one gallery of photos of whatever quality it happens to hold. NIST measured the spread. Each agency lives somewhere inside it.
Who Is Running the Searches
In June 2021 the Government Accountability Office published the public version of a review of federal law enforcement's use of the technology, GAO-21-518. It surveyed 42 federal agencies that employ law enforcement officers. Twenty reported owning systems with face recognition or using systems owned by other entities: other federal agencies, states, localities and private companies.
The agencies reported using it for criminal investigations, among other things. Six reported using it on images of the unrest, riots or protests following the death of George Floyd in May 2020. Three reported using it on images of the events at the U.S. Capitol on January 6, 2021.
The finding that drew the recommendations was about knowing. All 14 agencies that reported using face recognition to support criminal investigations also reported using systems owned by non-federal entities. Only one had awareness of which non-federal systems its employees were using. GAO made two recommendations to each of 13 agencies: put in place a mechanism to track which non-federal systems employees use, and assess the risks of those systems, including privacy and accuracy risks. Twelve agreed with both. The Postal Service agreed with one and partially agreed with the other.
Put that beside the NIST result. The accuracy of a search depends on which algorithm runs it, and most of the agencies doing investigative searches could not say which systems their own people were searching through.
Robert Williams's Driveway
Robert Julian-Borchak Williams sued the City of Detroit, its police chief and a detective in federal court in April 2021. What follows is what his complaint alleged; the case ended in a settlement that, by its own terms, is not an admission of liability.
On October 2, 2018, according to the complaint, someone walked into a Shinola store in Detroit and stole five watches. The store's surveillance footage was poorly lit and the shoplifter never looked directly into the camera. Months later a still made from that footage was sent for a face recognition search, and on March 11, 2019 the Michigan State Police returned an "Investigative Lead Report" naming Williams. The photo it matched was his expired driver's licence picture. The complaint says the report carried, prominently, this warning: "THIS DOCUMENT IS NOT A POSITIVE IDENTIFICATION. IT IS AN INVESTIGATIVE LEAD ONLY AND IS NOT PROBABLE CAUSE TO ARREST." It also says the report did not include the similarity score or the other candidates the system would have returned.
The report said, in capital letters, that it was not an identification and not probable cause to arrest. The complaint says that is roughly what it became.
The next step, the complaint alleged, was a photo array shown not to an eyewitness but to a loss-prevention contractor who had not been in the store and had only watched the same footage. She picked the expired licence photo. On that basis a warrant was requested. On January 9, 2020, Williams, described in the complaint as a 43-year-old Black man living in Farmington Hills with his wife and two young daughters, pulled into his driveway after work and was arrested in front of his family. He was held about 30 hours. At his probable cause conference later that month, the Wayne County Prosecutor's Office dismissed the charges, without prejudice.
What the Settlement Changed
The case was dismissed with prejudice on June 28, 2024, on a stipulated order signed by District Judge Laurie J. Michelson, with the court keeping jurisdiction to enforce the settlement for four years. The agreement filed with it is about procedure, not money; its damages terms are in a release the parties agreed to redact in full.
Detroit agreed to implement a revised face recognition directive and new lead and vetting report forms. It agreed that for four years it would not propose changes that would let investigators conduct a lineup based on a face recognition lead without first developing an independent and reliable basis for it, or seek an arrest warrant based only on such a lineup combined with a face recognition lead. Nor, for those four years, will it change its rules to allow lineups shown to non-eyewitnesses, lineups containing a photo that came from a face recognition search, or telling a witness that face recognition had been used. And it agreed to audit every case since February 2017 in which a face recognition lead was followed by an arrest or a warrant, checking whether there was independent evidence, and to notify the prosecutor where there was not.
Read as a list, those terms describe the gap the complaint alleged. The system produced a candidate. The safeguards that were supposed to turn a candidate into evidence, an eyewitness, a blind lineup, an independent reason, were the part that went missing.
The Store Version
Police are not the only ones running searches. In December 2023 the Federal Trade Commission sued Rite Aid in federal court in Philadelphia over the face recognition system the pharmacy chain had used in a subset of its stores. The complaint alleged that cameras captured the faces of consumers as they entered and moved through those stores and compared them against a database of people Rite Aid suspected or accused of wrongdoing, a database it said held at least tens of thousands of enrollments. Rite Aid, the complaint said, failed to take reasonable steps to keep low-quality images out of it.
The complaint alleged the system generated thousands of false-positive matches. Between December 2019 and July 2020, it said, the system produced over 5,000 match alerts in stores more than 100 miles from the store that had created the enrollment. Employees acting on alerts, it alleged, subjected people to searches, wrongly accused them of shoplifting, asked them to leave and called the police. One example in the complaint: employees stopped and searched an 11-year-old girl on the basis of a false-positive match. The FTC alleged the harm fell especially on Black, Asian, Latino and women consumers.
On February 26, 2024 the court entered a stipulated order for permanent injunction. Its first provision prohibits Rite Aid, for five years, from deploying or using any facial recognition or analysis system in any retail store, retail pharmacy or online retail platform. Rite Aid neither admitted nor denied the allegations. What it gave up was the system.
A Match Is Not a Name
Every one of these cases runs on the same mistake about what the machine said. NIST's report is about thresholds and rates: given a gallery and an algorithm, how often does a stranger's face score high enough to count. The rate is never zero, and it is not the same for everyone. A search of a large gallery returns the closest faces in it whether or not the person being sought is in there at all.
That is why the Michigan report Williams's complaint quoted said, in capitals, that it was not an identification. The technology is honest about this in its own paperwork. The failure happens downstream, when a human being reads "most similar face in the database" as "the person who did it" and builds the rest of the case around a name instead of testing it.
And Then You're Dead
Your face is already in a database: a licence photo, a passport, a booking photo from an arrest that went nowhere. Each time someone runs a search, the software ranks you against a stranger's blurry still and reports how alike you are. Federal testing shows that the chance of you scoring as someone else depends on the vendor, the threshold, the photo, and your race, sex and age, and that it can differ between groups by a factor of a hundred. Most agencies running these searches could not say which systems their people were using. The warning is printed right on the report: this is a lead, not an identification. In the cases that reached a courtroom, the people who read it treated the lead as the answer anyway.
That's it. That's the whole thing.
Sources
- Definitions of false positives, false negatives and the false match rate; 1:1 verification compared with 1:N identification; 18.27 million images of 8.49 million people through 189 algorithms from 99 developers; false positive differentials of 10 to beyond 100 times; the regional, sex (2 to 5 times higher in women) and age findings; and the note that the most accurate algorithms show smaller differentials — Patrick Grother, Mei Ngan and Kayee Hanaoka, "Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects," NIST Interagency Report 8280, December 2019
- The survey of 42 federal agencies; the 20 that own or use face recognition; use on images from the 2020 protests and January 6, 2021; the 14 agencies using non-federal systems for criminal investigations and the one that tracked them; and the recommendations to 13 agencies — U.S. Government Accountability Office, "Facial Recognition Technology: Federal Law Enforcement Agencies Should Have Better Awareness of Systems Used By Employees," GAO-21-518, June 2021
- The allegations about the Shinola theft, the Investigative Lead Report and its warning, the photo array, the January 9, 2020 arrest, the 30-hour detention and the dismissal of charges — Complaint, Williams v. City of Detroit, No. 2:21-cv-10827 (E.D. Mich.), ECF No. 1, April 13, 2021
- The dismissal with prejudice and four-year retained jurisdiction — Stipulated Order of Voluntary Dismissal, ECF No. 73, June 28, 2024; the settlement terms, lineup restrictions, audit from February 22, 2017 and redacted release — Settlement Agreement, ECF No. 73-1; docket via CourtListener (RECAP archive)
- The allegations about Rite Aid's enrollment database, false-positive match alerts, the over 5,000 alerts more than 100 miles from the enrolling store, the 11-year-old girl and the groups harmed — Complaint, FTC v. Rite Aid Corp., No. 2:23-cv-05023 (E.D. Pa.), Dkt. 1, December 19, 2023; the five-year prohibition — Stipulated Order for Permanent Injunction and Other Relief, Dkt. 19, February 26, 2024; docket via CourtListener (RECAP archive)
- Photo: "Omotesando Hills dome camera" by Syced, CC BY-SA 3.0, via Wikimedia Commons
Comments
Loading…