2_use_case_name: Spoofing Detection AI/ML Project
3_agency: Commodity Futures Trading Commission
3_abr: CFTC
4_bureau: DOE
8_topic_area: Law & Justice
10_commercial_ai: None of the above.
11_purpose_benefits: Pilot project that applied AI/ML techniques to identify spoofing patterns in order message data. Project used a variety of supervised and unsupervised techniques trained with DOE's existing set of expert based spoofing detection algorithms.
12_outputs: Model provides probability of spoofing behavior for every individual trader in a specified market on a specified day
16_dev_stage: Retired
17_impact_type: Both
18_date_initiated: 8/1/2023
19_date_acq_dev_began: 9/1/2023
20_date_implemented: 12/1/2023
21_date_retired: 4/26/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: REDACTED
24_hisp_support: No
27_public_info: No
29_contains_pii: No
30_saop_review: Yes
31_data_catalog: Other
31_question_type: Acquisition of the data catalog was concurrent with development of the use case 
33_agency_data: Order message data
34_data_docs: Documentation is complete: Documentation exists regarding the maintenance, composition, quality, and intended use of the training and evaluation data, as well as any statistical bias across model features and protected groups. 
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: REDACTED
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
51_extension_request: No – Agency did not request an extension for this use case.
52_impact_assessment: No
53_real_world_testing: Performance evaluation in operational environment: The AI use case has been tested in an operational environment before being fully implemented as a solution, or has been tested in simulated or controlled environment using operational or synthetic data.
54_key_risks: Risks inherent to any analytical identification: risk of false positives; risks identified by discussion among developers and use case owners
55_independent_eval: Yes – by another appropriate agency office that was not directly involved in the system’s development
56_monitor_postdeploy: No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.
57_autonomous_impact: Other – Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. 
57_question_type: The model flags conduct for expert review. If the human reviewer were to see evidence of spoofing, the trader would be referred to the existing, human, pre-investigatory enforcement review process 
59_ai_notice: N/A - individuals are not interacting with the AI for this use case
61_adverse_impact: Yes
62_disparity_mitigation: We follow the same procedures as we do for all enforcement matters.
63_stakeholder_consult: None of the above
65_appeal_process: No – Law, operational limitations, or governmentwide guidance precludes an opportunity for an individual to appeal.
66_no_appeal_reason: Traders have rights in any enforcement process, including any initiated by AI output.
67_opt_out: No – This AI use case is not subject to the opt-out requirement because the AI functionality is solely used for the prevention, detection, and investigation of fraud or cybersecurity incidents, or the conduct of a criminal investigation.