Face Recognition Technology: DOJ and FBI Have Taken Some Actions in Response to GAO Recommendations to Ensure Privacy and Accuracy, But Additional Work Remains
Released: 2019-06-04T00:00:00

Letter

Background

FBI’s Use of Face Recognition Technology

For decades, fingerprint analysis was the most widely used biometric technology for positively identifying arrestees and linking them with any previous criminal record. However, beginning in 2010, the FBI began incrementally replacing the Integrated Automated Fingerprint Identification System (IAFIS) with Next Generation Identification (NGI). NGI was not only to include fingerprint data from IAFIS and biographic data, but also to provide new functionality and improve existing capabilities by incorporating advancements in biometrics, such as face recognition technology. As part of the fourth of six NGI increments, the FBI updated the Interstate Photo System (IPS) to provide a face recognition service that allows law enforcement agencies to search a database of criminal photos that accompanied fingerprint submissions using a photo of an unknown person—called a probe photo. The FBI began a pilot of NGI-IPS in December 2011, and NGI-IPS became fully operational in April 2015.

NGI-IPS users include the FBI and selected state and local law enforcement agencies, which can submit search requests to help identify an unknown person using, for example, a photo from a surveillance camera. When a state or local agency submits such a photo, NGI-IPS uses an automated process to return a list of candidate photos from the database. The number of photos returned ranges from 2 to 50 possible candidate photos from the database, depending on the user’s specification. According to the FBI, in fiscal year 2018, NGI-IPS returned about 50,000 face recognition search results to law enforcement agency users, a decrease from about 90,000 search results in fiscal year 2017. Figure 1 describes the process for a search requested by state or local law enforcement.

In addition to the NGI-IPS, the FBI has an internal unit called Facial Analysis, Comparison and Evaluation (FACE) Services that provides face recognition capabilities, among other things, to support active FBI investigations. FACE Services has access to NGI-IPS, and can also search or request to search databases owned by the departments of State and Defense and 21 states, which use their own face recognition systems. Figure 2 shows which states partnered with the FBI for FACE Services requests, as of May 2019, according to the FBI.

Unlike NGI-IPS, which primarily contains photos obtained from criminal justice sources, these external systems primarily contain photos from state and federal government databases, such as driver’s license photos and visa applicant photos. According to the FBI, the total number of face photos available in all searchable repositories for FACE Services is over 641 million. Biometric images specialists for FACE Services manually review any photos received from their external partners before returning a photo as an investigative lead to the requesting FBI agents. No more than two photos are returned as a lead after the specialist for FACE Services completes the review. However, according to FACE Services officials we met with during our May 2016 review, if biometric images specialists determine that none of the databases returned a likely match, they do not return any photos to the agents. According to the FBI, from August 2011 (when searches began) through April 2019, FACE Services received 153,636 photos of unknown persons (often called probe photos) from FBI headquarters, field offices, and overseas offices, which resulted in 390,186 searches of various databases in attempt to find photo matches of known individuals in these databases.

Privacy Laws and Responsibilities at DOJ

Federal agency collection and use of personal information, including face images, is governed primarily by two laws: the Privacy Act of 1974 and the privacy provisions of the E-Government Act of 2002.

The Privacy Act places limitations on agencies’ collection, disclosure, and use of personal information maintained in systems of records. The Privacy Act requires that when agencies establish or make changes to a system of records, they must notify the public through a system of records notice (SORN) in the Federal Register. According to Office of Management and Budget (OMB) guidance, the purposes of the notice are to inform the public of the existence of systems of records; the kinds of information maintained; the kinds of individuals on whom information is maintained; the purposes for which they are used; and how individuals can exercise their rights under the Privacy Act.

The E-Government Act of 2002 requires that agencies conduct Privacy Impact Assessments (PIAs) before developing or procuring information technology (or initiating a new collection of information) that collects, maintains, or disseminates personal information. The assessment helps agencies examine the risks and effects on individual privacy and evaluate protections and alternative processes for handling information to mitigate potential privacy risks. OMB guidance also requires agencies to perform and update PIAs as necessary where a system change creates new privacy risks, for example, when the adoption or alteration of business processes results in personal information in government databases being merged, centralized, matched with other databases or otherwise significantly manipulated.

Within DOJ, preserving civil liberties and protecting privacy is a responsibility shared by departments and component agencies. As such, DOJ and the FBI have established oversight structures to help protect privacy and oversee compliance with statutory and policy requirements. For example, the FBI drafts privacy documentation for its face recognition capabilities, and DOJ offices review and approve key documents developed by the FBI—such as PIAs and SORNs.

DOJ and FBI Have Taken Steps Since May 2016 to Better Ensure Privacy but Work Remains to Fully Address Prior Recommendations

DOJ Has Taken Steps to More Quickly Publish Privacy Impact Assessments but Has Not Fully Implemented Its Revised Process

We reported in May 2016 that the FBI did not (1) update the NGI-IPS PIA in a timely manner when the system underwent significant changes, or (2) develop and publish a PIA for FACE Services before that unit began supporting FBI agents. However, DOJ and the FBI have since taken steps to review and publish PIAs more quickly.

As discussed in our 2016 report, consistent with the E-Government Act and OMB guidance, DOJ developed guidance that requires initial PIAs to be completed at the beginning of development of information systems and any time there is a significant change to the information system in order to determine whether there are any resulting privacy issues. In accordance with this guidance, FBI published a PIA at the beginning of the development of NGI-IPS in 2008, as required. However, the FBI did not publish a new PIA or update the 2008 PIA before beginning to pilot NGI-IPS in December 2011 or as significant changes were made to the system through September 2015. During the pilot, the FBI used NGI- IPS to conduct over 20,000 searches to assist in investigations.

Similarly, DOJ did not approve a PIA for FACE Services when it began supporting investigations in August 2011. As a new use of information technology involving the handling of personal information, it too required a PIA, according to the E-Government Act, as well as OMB and DOJ guidance. Figure 3 provides key dates in the implementation of these face recognition capabilities and the associated PIAs.

DOJ approved the NGI-IPS PIA in September 2015 and the FACE Services PIA in May 2015—over 3 years after the NGI-IPS pilot began and FACE Services began supporting FBI agents with face recognition services. Among other factors, implementation of the NGI-IPS pilot constituted a significant change in the FBI’s use of the technology that, consistent with the E-Government Act and OMB guidance required DOJ/FBI to update the PIA. Similarly, DOJ/FBI acknowledged that FACE Services began supporting FBI investigations in 2011, which involved storing photos in a new work log and also performing automated searches instead of manual searches. As a new use of information technology involving the handling of personal information, it too required a PIA. While DOJ and the FBI updated the internal drafts of these PIAs, the public remained unaware of the department’s consideration for how the FBI uses personal information in the face recognition search process. Given the issues we identified, we recommended that DOJ assess the PIA development process to determine why PIAs were not published prior to using or updating face recognition capabilities.

Although DOJ officials did not concur with this recommendation, they did agree that all DOJ processes may be reviewed for improvements and efficiencies. In November 2018, DOJ officials told us that they had reviewed the PIA development process and determined that one reason the FBI’s face recognition PIAs were not completed more quickly was because the FBI and DOJ engaged in an extensive PIA revision process. As a result, DOJ reported that it implemented a pilot in 2018 to expedite the PIA approval process, which included developing a PIA approval template, conducting DOJ’s review earlier in the process, and focusing the review solely on legal sufficiency instead of a more comprehensive review that included less significant editorial changes. According to DOJ, this new process has significantly reduced the time required between the completion of the PIA process by the FBI and the review by DOJ. Further, DOJ reported that it has applied the same process to other DOJ components since December 2018, and that the pilot is evolving into an operational process. We will continue to monitor DOJ’s implementation of its review process changes.

DOJ Did Not Complete a SORN Addressing FBI’s Face Recognition Capabilities in a Timely Manner and Has Not Implemented Corrective Actions

We reported in May 2016 that DOJ did not publish a SORN, as required by the Privacy Act, that addresses the collection and maintenance of photos accessed and used through the FBI’s face recognition capabilities, in a timely manner. The DOJ published the SORN on May 5, 2016—after completion of our review—even though those capabilities were in place since 2011. According to OMB guidance then in effect, the SORN “must appear in the Federal Register before the agency begins to operate the system, e.g., collect and use the information.” However, from 2011 through May 2016, the agency collected and maintained personal information for these capabilities without the required explanation of what information it was collecting or how it was used. For example, at the time of our review, the existing version of the SORN that covered FBI’s face recognition capabilities was dated September 1999. According to DOJ officials, it did not address the collection and maintenance of photos accessed and used through NGI for the FBI’s face recognition capabilities but rather discussed fingerprint searches. Given that DOJ did not publish the SORN in a timely manner, we recommended DOJ develop a process to determine why a SORN was not published for the FBI’s face recognition capabilities prior to using NGI-IPS, and implement corrective actions to ensure SORNs are published before systems become operational. DOJ agreed, in part, with our recommendation and submitted the SORN for publication after we provided our draft report for comment.

According to DOJ, it continues to review and update its pre-existing SORNs on an ongoing basis and is continually improving the scope and efficiency of its privacy processes. However, as of May 2019, DOJ had not taken actions to address our recommendation. Further, in April 2019, DOJ stated that with respect to transparency, a published PIA will provide much the same information that would be contained in a SORN and may provide it in a timelier manner. However, according to OMB guidance, the purpose of the SORN is to inform the public of the existence of systems of records; the kinds of information maintained; the kinds of individuals on whom information is maintained; the purposes for which they are used; and how individuals can exercise their rights under the Privacy Act. Further, PIAs and SORNs both contain information key to providing the public with information about the collection of their personal information, among other things. We continue to believe that by assessing the SORN development process and taking corrective actions to ensure timely development of future SORNs, DOJ would be better positioned to provide the public with a better understanding of how personal information is being used and protected by DOJ components.

FBI Has Conducted Audits to Oversee the Use of NGI-IPS and FACE Services

The Criminal Justice Information Services Division (CJIS), which operates FBI’s face recognition capabilities, has an audit program to evaluate compliance with restrictions on access to CJIS systems and information by its users, such as the use of fingerprint records. However, at the time of our May 2016 review, it had not completed audits of the use of NGI-IPS or FACE Services searches of external databases. We reported that state and local users had been accessing NGI-IPS since December 2011 and had generated IPS transaction records since then that would enable CJIS to assess user compliance. In addition, we found that the FACE Services Unit had used external databases that included primarily civil photos to support FBI investigations since August 2011, but the FBI had not audited its use of those databases. Standards for Internal Control in the Federal Government calls for federal agencies to design and implement control activities to enforce management’s directives and to monitor the effectiveness of those controls. In May 2016, we recommended that the FBI conduct audits to determine the extent to which users of NGI-IPS and biometric images specialists in FACE Services are conducting face image searches in accordance with CJIS policy requirements.

DOJ partially concurred with our recommendation. Specifically, DOJ concurred with the portion of our recommendation related to the use of NGI-IPS. In March 2017, DOJ reported that the FBI began assessing NGI-IPS requirements in participating states in conjunction with its triennial National Identity Services audit, and by February 2018 had conducted eight NGI-IPS audits, which found no significant findings of noncompliance. In February 2018, DOJ provided us with copies of the final audit results for one state and its NGI-IPS audit reference guide.

The FBI reported that it conducted an audit of FACE Services in September 2018. According to FBI documentation, the purpose of the audit was to determine the extent to which specialists in FACE Services conducted face image searches in accordance with FBI privacy laws and policies. The scope of the audit focused on determining adherence to policies which govern the appropriate use of NGI-IPS, including those for policy development as well as authorized requests and responses. The FBI reported that it finalized the audit report in April 2019, which concluded that the Face Services Unit is operating in accordance with privacy laws and policies. Further, the FBI stated in May 2019 that audits of FACE Services will continue on a triennial basis and that it conducts triennial audits of states that use NGI-IPS. As a result, DOJ has fully implemented our recommendation.

FBI Has Taken Limited Actions to Address Our Recommendations for Ensuring the Accuracy of Its Face Recognition Capabilities

FBI Has Conducted Limited Assessments of the Accuracy of NGI-IPS Face Recognition Searches

In May 2016, we reported that prior to accepting and deploying NGI-IPS, the FBI conducted testing to evaluate how accurately face recognition searches returned matches to persons in the database. However, we found that the tests were limited because they did not include all possible candidate list sizes and did not specify how often incorrect matches were returned. According to the National Science and Technology Council and the National Institute of Standards and Technology at the time, the detection rate (how often the technology generates a match when the person is in the database) and the false positive rate (how often the technology incorrectly generates a match to a person in the database) are both necessary to assess the accuracy of a face recognition system. The FBI’s detection rate requirement for face recognition searches at the time stated that when the person exists in the database, NGI-IPS shall return a match of this person at least 85 percent of the time. However, we found that the FBI only tested this requirement with a candidate list of 50 potential matches. In these tests, 86 percent of the time, a match to a person in the database was correctly returned. The FBI had not assessed accuracy when users requested a list of 2 to 49 matches.

According to FBI, a smaller list would likely lower the accuracy of the searches as the smaller list may not contain the likely match that would be present in the larger list. Further, FBI officials stated during our May 2016 review that they had not assessed how often NGI-IPS face recognition searches erroneously match a person to the database (the false positive rate). If false positives are returned at a higher than acceptable rate, law enforcement users may waste time and resources pursuing unnecessary investigative leads. In addition, we concluded that by conducting this assessment the FBI would help ensure that it is sufficiently protecting the privacy and civil liberties of U.S. citizens enrolled in the database. Therefore, we recommended that the FBI conduct tests of NGI-IPS to verify that the system is sufficiently accurate for all allowable candidate list sizes and ensure that both the detection rate and the false positive rate are identified for such tests.

In comments on our draft report in 2016, and reiterated during recommendation follow-up in May 2019, DOJ did not concur with this recommendation. DOJ officials stated that the FBI has performed accuracy testing to validate that the system meets the requirements for the detection rate, which fully satisfies requirements for the investigative lead service provided by NGI-IPS. As of May 2019, DOJ has not taken action to address the recommendation.

We continue to believe that the recommended action is needed. Such action would allow the FBI to have more reasonable assurance that NGI- IPS provides leads that help enhance, rather than hinder, criminal investigations and that helps protect the privacy of citizens. As noted above, a key focus of our recommendation is the need to ensure that NGI-IPS is sufficiently accurate for all allowable candidate list sizes. As we reported, although the FBI tested the detection rate for a candidate list of 50 photos, they did not do such tests when NGI-IPS users request smaller candidate lists—specifically between 2 and 50 photos. Further, according to the FBI Information Technology Life Cycle Management Directive, testing needs to confirm the system meets all user requirements. Because the accuracy of NGI-IPS’s face recognition searches when returning fewer than 50 photos in a candidate list is unknown, the FBI is limited in understanding whether the results are accurate enough to meet NGI-IPS users’ needs.

In comments on our May 2016 report, DOJ officials also stated that searches of NGI-IPS produce a gallery of likely candidates to be used as investigative leads, not for positive identification. As a result, according to DOJ officials, NGI-IPS cannot produce false positives and there is no false positive rate for the system. We disagree with DOJ. According to the National Institute of Standards and Technology, the detection rate and the false positive rate are both necessary to assess the accuracy of a face recognition system. Generally, face recognition systems can be configured to allow for a greater or lesser number of matches. A greater number of matches would generally increase the detection rate, but would also increase the false positive rate. Similarly, a lesser number of matches would decrease the false positive rate, but would also decrease the detection rate. Reporting a detection rate of 86 percent without reporting the accompanying false positive rate presents an incomplete view of the system’s accuracy.

FBI Agreed to Conduct Annual Operational Reviews of NGI-IPS but Implementation Is Incomplete

We reported in May 2016 that FBI, DOJ, and OMB guidance all required annual reviews of operational information technology systems to assess their abilities to continue to meet cost and performance goals. For example, the FBI’s Information Technology Life Cycle Management Directive required an annual operational review to ensure that the fielded system is continuing to support its intended mission, among other things.

In May 2016, we reported that the FBI had not assessed the accuracy of face recognition searches of NGI-IPS in its operational setting—the setting in which enrolled photos, rather than a test database of photos are used to conduct a search for investigative leads. According to FBI officials, at the time of our May 2016 review, the database of photos used in its tests was representative of the photos in NGI-IPS, and ongoing testing in a simulated environment was adequate. However, according to the National Institute of Standards and Technology, as the size of a photo database increases, the accuracy of face recognition searches performed on that database can decrease due to lookalike faces. At the time of our review, FBI’s test database contained 926,000 photos while NGI-IPS contained about 30 million photos. We concluded that by conducting an operational review of these systems, FBI officials would obtain information regarding what factors affect the accuracy of the face recognition searches, such as the quality of the photos in the database, and if NGI-IPS is meeting federal, state, and local law enforcement needs. As a result, we recommended the FBI conduct an operational review of NGI-IPS, at least annually, that includes an assessment of the accuracy of face recognition searches and take actions, as necessary, to improve the system.

In May 2016, DOJ concurred with this recommendation and has taken steps to seek input from its users. For example, the FBI submitted a staff paper through the fall 2016 Advisory Policy Board Process to solicit feedback from its users. Specifically, officials said the paper requested feedback on whether the face recognition searches of the NGI-IPS are meeting their needs, and input regarding search accuracy. According to FBI officials, no users expressed concern with any aspect of the NGI-IPS meeting their needs, including accuracy. DOJ reported that it repeated this process in the fall of 2017.

Although FBI’s action of providing working groups with a paper presenting our recommendation is a positive step, FBI’s actions do not fully meet the recommendation. FBI’s paper was presented as informational, and did not result in any formal responses from users. We disagree with FBI’s conclusion that receiving no responses on the informational paper fulfills the operational review recommendation, which includes determining that NGI-IPS is meeting user’s needs. In addition, in May 2019, the FBI stated that it will be working with the National Institute of Standards and Technology on annual operational testing and that such testing meets the intention of this recommendation. However, the proposed testing, while promising, will not occur in an operational environment. As such, we continue to believe the FBI should conduct an operational review of NGI- IPS at least annually, as we recommended.

FBI Has Not Assessed the Accuracy of External Partners’ Face Recognition Systems Used by FACE Services

In May 2016 we reported that FBI officials had not assessed the accuracy of face recognition systems operated by external partners. Specifically, before agreeing to conduct searches on, or receive search results from, these systems, the FBI did not ensure the accuracy of these systems was sufficient for use by FACE Services. Standards for Internal Control in the Federal Government calls for agencies to design and implement components of operations to ensure they meet the agencies mission, goals, and objectives, which, in this case, is to identify missing persons, wanted persons, suspects, or criminals for active FBI investigations. As a result, we recommended the FBI take steps to determine whether each external face recognition system used by FACE Services is sufficiently accurate for the FBI’s use and whether results from those systems should be used to support FBI investigations.

In comments on our draft report in 2016, and reiterated during subsequent recommendation follow-up, DOJ officials did not concur with this recommendation. DOJ officials stated that the FBI has no authority to set or enforce accuracy standards of face recognition technology operated by external agencies. In addition, DOJ officials stated that the FBI has implemented multiple layers of manual review that mitigate risks associated with the use of automated face recognition technology. Further, DOJ officials stated there is value in searching all available external databases, regardless of their level of accuracy.

We acknowledge that the FBI cannot and should not set accuracy standards for the face recognition systems used by external partners. We also agree that the use of external face recognition systems by the FACE Services Unit could add value to FBI investigations. However, we disagree with DOJ and continue to believe that the FBI should assess the quality of the data it is using from state and federal partners. We also disagree with the DOJ assertion that manual review of automated search results is sufficient. Even with a manual review process, the FBI could miss investigative leads if a partner does not have a sufficiently accurate system. The FBI has entered into agreements with state and federal partners to conduct face recognition searches using hundreds of millions of photos. Without assessments of the results from its state and federal partners, the FBI is making decisions to enter into agreements based on assumptions that the search results may provide valuable investigative leads. For example, the FBI’s accuracy requirements for criminal investigative purposes may be different than a state’s accuracy requirements for preventing driver’s license fraud. By relying on its external partners’ face recognition systems, the FBI is using these systems as a component of its routine operations and is therefore responsible for ensuring the systems will help meet the FBI’s mission, goals and objectives. Until FBI officials can assure themselves that the data they receive from external partners are reasonably accurate and reliable, it is unclear whether such agreements are beneficial to the FBI, whether the investment of public resources is justified, and whether photos of innocent people are unnecessarily included as investigative leads.

Chairman Cummings, Ranking Member Jordan, and Members of the Committee, this concludes my prepared statement. I would be happy to respond to any questions you may have at this time.

GAO Contact and Staff Acknowledgments

For questions about this statement, please contact Gretta Goodwin at (202) 512-8777 or GoodwinG@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include Dawn Locke (Assistant Director), Jason Jackson (Analyst-In- Charge), Jennifer Beddor, Ann Halbert-Brooks, Eric Hauswirth, Paul Hobart, Richard Hung, Susanna Kuebler, Kay Kuhlman, Tom Lombardi, and Dina Shorafa. Key contributors for the previous work that this testimony is based on are listed in the previously issued product.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.