2_use_case_name: Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) Clinician Training Simulator
3_agency: Department of Veterans Affairs
3_abr: VA
4_bureau: VHA: Veterans Health Administration
8_topic_area: Education & Workforce
10_commercial_ai: None of the above.
11_purpose_benefits: The CAPS-5 Clinician Training Simulator comprises three online courses, available in TMS and TRAIN, that provide instruction in the administration and scoring of the CAPS-5. The courses use voice recognition, 3-D technology, and artificial intelligence to create an immersive learning experience. The courses are intended for clinicians, trainees, researchers, and examiners in VA and in the community. They use AI in the following ways: 1. In the three current CAPS-5 virtual patient courses, learners have the option to speak the prompts, with their spoken prompts transformed to text using speech recognition. 2. The learners’ prompts, whether spoken or typed, are programmatically compared to the required prompts in the CAPS-5. The system analyzes whether the prompts were delivered exactly (“verbatim”), with slight variations (“paraphrased”), or with unacceptable variations (“off-script”). 3. One of the courses analyzes learner prompts in order to trigger comments –corrections or praise -- from a virtual coach. 4. We’re currently revising one of the courses to use generative AI. Here’s how the vendor described it in their response to our solicitation: “This new design will also leverage a separate backend to handle AI multi-agency and retrieval augmented generation (RAG) to ensure large language models (LLMs) used will always operate as designed.” This will be a closed system that does not pull information from or input information to publicly-available platforms (e.g., GPT-4, Bard). More information about the three courses and access to them is available at https://www.ptsd.va.gov/professional/continuing_ed/caps5_clinician_training.asp.


12_outputs: The AI system outputs, narrowly defined, are the virtual patients' responses to the learners' prompts. More broadly, the system is designed to help learners learn to more accurately administer and score the CAPS-5. Learners receive a summary document describing their performance at the conclusion of each course.
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 10/15/2018
19_date_acq_dev_began: 11/29/2018
20_date_implemented: 3/13/2020
22_dev_method: Developed with contracting resources.
23_contract_piids: 3610G8Q9155
24_hisp_support: No
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: No
43_infra_provisioned: No
45_compute_request: No
47_timely_resources: No
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.