Amazon’s face surveillance technology is the target of growing opposition nationwide, and today, there are 28 more causes for concern. In a test the ACLU recently conducted of the facial recognition tool, called “Rekognition,” the software incorrectly matched 28 members of Congress, identifying them as other people who have been arrested for a crime. 
 
 The members of Congress who were falsely matched with the mugshot database we used in the test include Republicans and Democrats, men and women, and legislators of all ages, from all across the country. 
 
 
 
 Our test used Amazon Rekognition to compare images of members of Congress with a database of mugshots. The results included 28 incorrect matches. 
 
 The false matches were disproportionately of people of color, including six members of the Congressional Black Caucus, among them civil rights legend Rep. John Lewis (D-Ga.). These results demonstrate why Congress should join the ACLU in calling for a moratorium on law enforcement use of face surveillance. 
 
 To conduct our test, we used the exact same facial recognition system that Amazon offers to the public, which anyone could use to scan for matches between images of faces. And running the entire test cost us $12.33 — less than a large pizza. 
 
 Tell Amazon to get out of the surveillance business 
 
 Using Rekognition, we built a face database and search tool using 25,000 publicly available arrest photos. Then we searched that database against public photos of every current member of the House and Senate. We used the default match settings that Amazon sets for Rekognition. 
 
 
 
 Rep. Sanford Bishop (D-Ga.) was falsely identified by Amazon Rekognition as someone who had been arrested for a crime. 
 
 In a recent letter to Amazon CEO Jeff Bezos, the Congressional Black Caucus expressed concern about the “profound negative unintended consequences” face surveillance could have for Black people, undocumented immigrants, and protesters. Academic research has also already shown that face recognition is less accurate for darker-skinned faces and women. Our results validate this concern: Nearly 40 percent of Rekognition’s false matches in our test were of people of color, even though they make up only 20 percent of Congress. 
 
 
 
 People of color were disproportionately falsely matched in our test. 
 
 If law enforcement is using Amazon Rekognition, it’s not hard to imagine a police officer getting a “match” indicating that a person has a previous concealed-weapon arrest, biasing the officer before an encounter even begins. Or an individual getting a knock on the door from law enforcement, and being questioned or having their home searched, based on a false identification. 
 
 An identification — whether accurate or not — could cost people their freedom or even their lives. People of color are already disproportionately harmed by police practices, and it’s easy to see how Rekognition could exacerbate that. A recent incident in San Francisco provides a disturbing illustration of that risk. Police stopped a car, handcuffed an elderly Black woman and forced her to kneel at gunpoint — all because an automatic license plate reader improperly identified her car as a stolen vehicle. 
 
 Matching people against arrest photos is not a hypothetical exercise. Amazon is aggressively marketing its face surveillance technology to police, boasting that its service can identify up to 100 faces in a single image, track people in real time through surveillance cameras, and scan footage from body cameras. A sheriff’s department in Oregon has already started using Amazon Rekognition to compare people’s faces against a mugshot database, without any public debate. 
 
 Face surveillance also threatens to chill First Amendment-protected activity like engaging in protest or practicing religion, and it can be used to subject immigrants to further abuse from the government. 
 
 These dangers are why Amazon employees, shareholders, a coalition of nearly 70 civil rights groups, over 400 members of the academic community, and more than 150,000 members of the public have already spoken up to demand that Amazon stop providing face surveillance to the government. 
 
 Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition. This technology shouldn’t be used until the harms are fully considered and all necessary steps are taken to prevent them from harming vulnerable communities. 
 
 List of Members of Congress Falsely Matched With Arrest Photos 
 
 Senate 
 
 John Isakson (R-Georgia) 
 
 Edward Markey (D-Massachusetts) 
 
 Pat Roberts (R-Kansas) 
 
 House ||||| FRVT 1:1 [Latest Report] [Agreement] [API] [Validation] [Encryption] [FAQs] 
 
 FRVT 1:N [Homepage] [API] 
 
 FRVT Status 
 
 The latest report adds results for algorithms submitted by 39 developers during August and September 2018. Updates to this report will be produced as remaining results for the recently submitted algorithms become available. We retired older algorithms from many developers, because we only report results for at most two per developer. FRVT Ongoing is currently closed while FRVT 2018 (1:N search algorithms) continues in Phase 3. FRVT Ongoing will resume in 2019 - algorithms should be submitted during January 28 - February 1, 2019. The next full report is scheduled for March 1, 2019. There are planned updates to the API document, header file, and validation package. Developers must download and adhere to the most current version of these files, which will be published ahead of the test reopening in January 2019. 
 
 Separately, we have completed Phase 2 of the FRVT 2018 evaluation of 1:N algorithms and will publish a public report soon. FRVT 1:N Phase 2 evaluated 65 algorithms from 38 developers. 
 
 To receive update notifications, subscribe by email via subscribe. 
 
 FRVT Leaderboard 
 
 The leaderboard shows the top performing algorithms measured on false non-match rate (FNMR) across several different datasets. FNMR is the proportion of mated comparisons below a threshold set to achieve the false match rate (FMR) specified. FMR is the proportion of impostor comparisons at or above that threshold. 
 
 # Developer VISA 
 
 Photos 
 
 FNMR@ 
 
 FMR ≤ 0.000001 VISA 
 
 Photos 
 
 FNMR@ 
 
 FMR ≤ 0.0001 MUGSHOT 
 
 Photos 
 
 FNMR@ 
 
 FMR ≤ 0.0001 WILD 
 
 Photos 
 
 FNMR@ 
 
 FMR ≤ 0.00001 CHILD EXP 
 
 Photos 
 
 FNMR@ 
 
 FMR ≤ 0.01 Submission 
 
 Date 1 yitu-002 0.0041 0.0011 0.0137 0.05213 2018_10_19 2 yitu-001 0.0072 0.0037 0.0138 0.05826 0.57913 2018_06_12 3 sensetime-001 0.0093 0.0036 0.01311 1.00076 2018_10_19 4 sensetime-002 0.0104 0.00310 0.01529 1.00077 2018_10_19 5 siat-002 0.0135 0.00415 0.01415 0.05520 0.4283 2018_06_13 6 ntechlab-004 0.0136 0.0034 0.01312 0.0466 0.4202 2018_06_14 7 ntechlab-005 0.0147 0.0022 0.01310 0.05010 2018_10_19 8 megvii-002 0.0148 0.00412 0.03063 0.07135 2018_10_19 9 vocord-005 0.0169 0.0033 0.01532 0.0489 2018_10_18 10 everai-001 0.01610 0.00414 0.0132 0.0312 2018_10_30 11 visionlabs-005 0.01611 0.0035 0.0131 0.0444 2018_10_19 12 tech5-000 0.01612 0.0039 0.01426 0.08942 2018_10_18 13 vocord-004 0.01713 0.0038 0.01534 0.05316 0.89048 2018_06_15 14 tech5-001 0.01814 0.00413 0.01421 0.09745 2018_10_19 15 cognitec-000 0.01815 0.00411 0.01425 0.10247 2018_10_19 16 isystems-002 0.01816 0.00723 0.01424 0.05518 2018_10_18 17 camvi-002 0.02217 0.00520 0.0135 0.0301 2018_10_19 18 toshiba-002 0.02218 0.00519 0.0136 0.11350 2018_10_19 19 everai-000 0.02319 0.00416 0.0133 0.0323 2018_10_30 20 anke-001 0.02420 0.01446 0.984100 0.99475 2018_06_26 
 
 Additional algorithms not listed in the leaderboard can be found in Table 2 of our latest FRVT report. The report includes results for fifty four algorithms applied to six datasets. It is a draft made available for comments. It will be updated on a monthly basis as algorithms and computations complete, as datasets are added, and as new analyses are developed. 
 
 Prior editions of the report: 2018-06-21 | 2018-06-18 | 2018-04-03 | 2018-02-15 | 2018-01-25 | 2017-12-14 | 2017-11-16 | 2017-10-03 | 2017-08-25 | 2017-08-07 | 2017-07-31 | 2017-05-15 | 2017-05-01 | 2017-04-12 | 2017-04-03 | 2017-03-23. 
 
 Developers may submit their algorithms at any time by following instructions below. Results will be posted as soon as algorithms successfully complete the tests. 
 
 Overview 
 
 NIST started a new evaluation of face recognition technologies in February 2017. Unlike previous evaluations, the activity is conducted on an ongoing basis in that the evaluation remains open indefinitely such that developers may submit their algorithms to NIST whenever they are ready, but no more frequently than three calendar months. The algorithms will be evaluated rapidly on a first-come-first-served basis, following our MINEX III evaluation of fingerprint recognition implementations. Performance results will be posted to the NIST website as soon as they are ready. This approach more closely aligns evaluation with development schedules; this improves over the two to four year interval between past FRVT tests. 
 
 Goals 
 
 The FRVT is aimed at measurement of the performance of automated face recognition technologies applied to a wide range of civil, law enforcement and homeland security applications including verification of visa images, de-duplication of passports, recognition across photojournalism images, and identification of child exploitation victims. In all cases the input image will contain one face only. Our performance reports will include measurements of accuracy, speed, storage and memory consumption, and resilience. NIST will report the dependence of performance on the properties of the images and the subjects. In its initial form, FRVT has one assessment track, for face verification. 
 
 1:1 Verification Track 
 
 Participation Agreement [last update: 2017-02-10]: FRVT is conducted by NIST, an agency of the United States Government. Participation is free of charge. FRVT is open to a global audience of face recognition developers. All organizations who seek to participate in FRVT must sign and submit all pages of this Participation Agreement. 
 
 [last update: 2017-02-10]: FRVT is conducted by NIST, an agency of the United States Government. Participation is free of charge. FRVT is open to a global audience of face recognition developers. All organizations who seek to participate in FRVT must sign and submit all pages of this Participation Agreement. API Document [last update: 2018-04-03]: An updated version of the 1:1 Verification API document has been published. Developers must ensure that their submission conforms to the document's specifications. 
 
 [last update: 2018-04-03]: An updated version of the 1:1 Verification API document has been published. Developers must ensure that their submission conforms to the document's specifications. Validation [last update: 2018-04-03]: An updated validation package has been published to reflect the API changes. All participants must run their software through the updated validation package prior to submission. The purpose of validation is to ensure consistent algorithm output between your execution and NIST's execution. 
 
 [last update: 2018-04-03]: An updated validation package has been published to reflect the API changes. All participants must run their software through the updated validation package prior to submission. The purpose of validation is to ensure consistent algorithm output between your execution and NIST's execution. Encryption Instructions [last update: 2017-09-11]: All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FRVT Ongoing public key linked from this page. Participants must email their public key to NIST. The participant's public key must correspond to the participant's public-key fingerprint provided on the signed Participation Application. 
 
 [last update: 2017-09-11]: All submissions must be properly encrypted and signed before transmission to NIST. This must be done according to these instructions using the FRVT Ongoing public key linked from this page. Participants must email their public key to NIST. The participant's public key must correspond to the participant's public-key fingerprint provided on the signed Participation Application. Submission Instructions [last update: 2017-11-09]: Encrypted files below 20MB can be emailed to NIST at frvt@nist.gov. Encrypted files above 20MB can be provided as a download link from a generic http webserver (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s) as soon as the signed participation agreement is sent to NIST. There is no need to wait for NIST confirmation of the received agreement. 
 
 [last update: 2017-11-09]: Encrypted files below 20MB can be emailed to NIST at frvt@nist.gov. Encrypted files above 20MB can be provided as a download link from a generic http webserver (e.g., Google Drive). We cannot accept Dropbox links. NIST will not register, or establish any kind of membership, on the provided website. Participants can submit their algorithm(s) as soon as the signed participation agreement is sent to NIST. There is no need to wait for NIST confirmation of the received agreement. FAQs [last update: 2017-01-30]: Ongoing responses to a number of questions regarding the evaluation are addressed in our FAQs document. 
 
 Important Dates [last update: 2018-11-16]: NIST will resume accepting submissions between January 28 - February 1, 2019. The next full report is scheduled for March 1, 2019. 
 
 Contact Information 
 
 Inquiries and comments may be submitted to frvt@nist.gov. 
 
 Subscribe to the FRVT mailing list to receive emails when announcements or updates are made. ||||| CLOSE Both tech giants have teamed up with government agencies to test it out. Time 
 
 An image from the product page of Amazon's Rekognition service, which provides image and video facial and item recognition and analysis. (Photo: Amazon) 
 
 SAN FRANCISCO — The Orlando Police Department in Florida is planning to continue its test of a facial recognition program from Amazon, despite outcry from civil rights and privacy groups that law enforcement and government agencies could abuse the technology. 
 
 OPD announced last month that the trial proof of concept run of the software had expired, but OPD public information officer, Sgt. Eduardo Bernal, said in a release Monday that the department will continue its testing of the program. 
 
 Two years ago, Amazon built the facial and product recognition tool, called Rekognition, as a way for customers to quickly search a database of images and look for matches. Rekognition uses Amazon's cloud computing network AWS to compare images to a database of images the customer has provided. 
 
 At least two police departments around the country have been testing the software, and privacy advocates have protested, fearing law enforcement might abuse the technology. 
 
 In May, civil rights and privacy groups, including the American Civil Liberties Union, demanded that Amazon stop selling Rekognition to law enforcement and other government entities because they fear it could be used to unfairly target protesters, immigrants and any person just going about their daily business. 
 
 “Amazon should be protecting customers and communities, it should not be in the business of powering dangerous surveillance,” said Matt Cagle, an ACLU attorney. 
 
 The tool is used by many companies and organizations beyond law enforcement. Amazon says it has been used to find abducted people and that amusement parks have used it to find lost children. 
 
 During the royal wedding of Prince Harry and Meghan Markle in May, British broadcaster Sky News used Rekognition to help it identify celebrities at they entered Windsor Castle. 
 
 OPD is testing Rekognition with images of seven Orlando officers, who volunteered for the trial. The city created a database with pictures of the officers, then compared those to a video stream from eight city-owned surveillance cameras to see if it could correctly identify the officers when in view of the cameras. 
 
 Orlando Police Chief John Mina said Monday the department has made good strides in the pilot program and will continue the testing to determine whether it will add "value in enhancing the City’s public safety mission in a manner that balances reasonable privacy concerns.” 
 
 No images of the public are used for testing, the facial recognition tool is not used in an investigative capacity and all elements of the program are in accordance with laws related to privacy and civil rights, Bernal said in the release. 
 
 More: Capital Gazette gunman was identified using facial recognition technology that's been controversial 
 
 More: Amazon should stop selling facial recognition software to police, ACLU and other rights groups say 
 
 More: Amazon's controversial facial recognition program dropped by city of Orlando 
 
 Thirty-four civil rights groups sent a letter on May 22 to Amazon CEO Jeff Bezos, saying people should be "free to walk down the street without being watched by the government. Facial recognition in American communities threatens this freedom." 
 
 They fear the real-time face recognition could allow police or government groups to watch crowds and pick out activists or undocumented residents. 
 
 "Amazon Rekognition is primed for abuse in the hands of governments. This product poses a grave threat to communities, including people of color and immigrants, and to the trust and respect Amazon has worked to build," the letter states. 
 
 Police and city officials provided examples in a memo last week of how the facial recognition tool could be used. They said police could locate a wanted fugitive before he or she committed another crime, locate a suspect who has made violent threats, find a missing child or identify a registered sex offender near schools 
 
 “If this technology works, the suspects in these examples would have had their images entered into the system and perhaps could have been spotted by one of the many cameras, and never allowed to get anywhere near the victims or a large gathering," officials said in the memo. 
 
 Read or Share this story: https://usat.ly/2u8BVgd
– Always thought politicians were crooks? Now Amazon's facial recognition system agrees with you, but not in a good way. A test by the ACLU matching all 535 Congress members to 25,000 mugshots found 28 false positives—with a notably high number of people of color, the Verge reports. "An identification—whether accurate or not—could cost people their freedom or even their lives," says the ACLU. "Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition." But Amazon defended its Rekognition API software, saying the ACLU used the default certainty threshold of 80%, while Amazon suggests cops crank it up to at least 95%. Yet there's nothing to make law enforcement use the higher setting. Rekognition drew attention in May when the ACLU revealed that several law enforcement agencies were using it. Privacy and civil rights groups cried foul, but police liked the price tag (around $12 a month), and at least one police force kept using it after a trial run, per USA Today. Some are concerned about racial bias: 39% of false positives in the ACLU test are people of color, including civil-rights leader John Lewis, while people of color comprise only 20% of Congress. Another test of facial-recognition software found higher false positives among blacks and women. Now three members of Congress have written Jeff Bezos expressing concerns about Rekognition. "Serious concerns have been raised about the dangers facial recognition can pose to privacy and civil rights," they write.