2_use_case_name: Text Analytics Data Science Sentence Similarity Model
3_agency: Department of Homeland Security
3_abr: DHS
4_bureau: USCIS
8_topic_area: Law & Justice
10_commercial_ai: None of the above.
11_purpose_benefits: Text Analytics augments the tedious and time-consuming manual process to identify potential fraud, national security, and/or public safety concerns and enables the identification of such concerns across jurisdictional boundaries. It increases the integrity of immigration programs, strengthens officers’ confidence in their work, and contributes to the reduction in customer wait times.
12_outputs: Text Analytics does not make predictions, recommendations, or decisions. It is merely a research tool that identifies potential patterns, while remaining agnostic as to whether those patterns identify potential fraud, national security, and/or public safety concerns. Instead, trained staff evaluate the patterns to determine whether they identify potential concerns and then validate and/or invalidate those potential concerns through the course of their investigations or adjudications.
16_dev_stage: Operation and Maintenance
17_impact_type: Rights-Impacting

18_date_initiated: 2/1/2019
19_date_acq_dev_began: 2/1/2019
20_date_implemented: 11/1/2019
22_dev_method: Developed with contracting resources.
23_contract_piids: 70SBUR22F00000116
24_hisp_support: No
27_public_info: No
29_contains_pii: Yes
30_saop_review: Yes
31_data_catalog: Yes
33_agency_data: Text Analytics stores information extracted from benefit forms and supporting documents, focusing on the narrative portions of those documents.
34_data_docs: Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. 
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: Text Analytics
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.
51_extension_request: No – Agency did not request an extension for this use case.
52_impact_assessment: Yes
53_real_world_testing: Impact evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution and has utilized randomized experiments with a control group or other counterfactual, social systems analysis, or other rigorous research methodologies to evaluate impact and identify potential harm to users as well as broader groups of people.
54_key_risks: There is small risk of false positives or false negatives due to the model. The risk is mitigated through a manual review of any information produced from the tool. Text Analytics is a decision support tool. Text Analytics does not make recommendations of fraud or benefit / adjudication decisions, any decisions made from information stored in the tool is conducted through a manual review by a USCIS employee.
55_independent_eval: Yes – by the CAIO
56_monitor_postdeploy: Established Process of Machine Learning Operations: Alongside automated testing and drift detection, model re-training and re-deployments are supported by continuous integration pipelines that are managed by machine learning and data engineers on the platform, adapting work done by data science team into repeatable scripts for re-training and re-testing a model once deployed.
57_autonomous_impact: No - Some individual decisions or actions require direct human oversight.
59_ai_notice: N/A - individuals are not interacting with the AI for this use case
61_adverse_impact: No
63_stakeholder_consult: None of the above
65_appeal_process: Yes
67_opt_out: No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.