2_use_case_name: Elastic Machine Learning Threat Detection
3_agency: General Services Administration
3_abr: GSA
4_bureau: GSA
8_topic_area: Other
10_commercial_ai: None of the above.
11_purpose_benefits: Use AI to further analyze security logs and alert on anomalous data patterns for a human SOC analyst to review. The goal is for a human/AI partnership that can more effectively and quickly search through the data.
12_outputs: Write anomaly data to Security Information and Event Management (SIEM) tool and create alerts for Security Operations Center (SOC) team.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 4/1/2022
22_dev_method: Developed with both contracting and in-house resources.
24_hisp_support: No
25_hisp_name: Public Experience Portfolio – GSA 
26_public_service: Enable increased self-service through an expansion of digital costumer service tools
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
33_agency_data: Office of Leasing and IT Modernization partnered in a pilot program to test the feasibility of using artificial intelligence and machine learning using the Amazon Web Services (AWS) cloud service from the GSA Federal Acquisition Service (FAS) to classify pdf documents emailed into G-REX. The pilot built the pipeline to train individual document types that are typically uploaded in PDF format. The pilot was successful at illustrating how valuable time could be saved by completing this project in production, improving accuracy of document type assignment. The pilot utilized AWS FAS service, which would require additional funding. Appian announced they will provide the functionality as an out-of-the-box feature in late 2024, early 2025, to be able to scan pdf document types if G-REX can move to the cloud. The cloud assessment is ongoing as of submission of the need. This was an approved FY23 need.
34_data_docs: Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. 
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: I-SecTools
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: No
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.
51_extension_request: No – Agency did not request an extension for this use case.
52_impact_assessment: No
53_real_world_testing: Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. 
54_key_risks: There are no key risks currently as this system is designed to run beside a traditional human-operated SOC. All outputs are consumed by SOC operators and verified. No automated activity is taken by the AI beyond generating the initial alerts.
55_independent_eval: Planned or in-progress
56_monitor_postdeploy: Intermittent and Manually Updated: A plan for monitoring the AI use case is in place, and requires data science teams to work with DevOps engineers to manually update models at scheduled intervals, and create metrics to detect data distribution shifts between the operational environment and the training data for the model. 
57_autonomous_impact: No - Some individual decisions or actions require direct human oversight.
59_ai_notice: In-person
61_adverse_impact: Yes
62_disparity_mitigation: No demographic data is collected by the SIEM tool. No demographic data fed to AI system for training or inference.
63_stakeholder_consult: None of the above
65_appeal_process: No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.
66_no_appeal_reason: The AI system only generates alerts. All alerts are researched and validated by a human SOC analyst. Any action taken is at the discretion of the human, not the AI system.
67_opt_out: No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.
67_question_type: Human's review alerts generated and only human's make actionable decisions