2_use_case_name: USA Class
3_agency: Office of Personnel Management
3_abr: OPM
4_bureau: HRS/HRSES/Federal Classification Center
8_topic_area: Mission-Enabling
11_purpose_benefits: The purpose of this project is to build an application to help managers and HR specialists (classifiers) improve the quality and speed the development of position descriptions (PDs). Developing a PD can be a lengthy process that requires significant back-and-forth between managers and classifiers. The length of time required to develop a PD is often cited as a major pain point and bottleneck in federal hiring, workforce planning, organizational design, and performance management processes. To address the challenge today, managers will simply copy whole PDs or significant sections of PDs which weakens the accuracy of the new PD. Agencies have also turned to the use of “standard” PDs which tend to be too generic to be effective for use in recruitment and performance.

This project seeks to solve the “blank-page-syndrome” by suggesting draft major duties, factor evaluation statements, and other elements of the PD for managers and classifiers to use to develop PDs that meet classification standards.

This project is expected to significantly decrease the time required to develop PDs, leading to a decrease in time to hire and other related HR processes. A decrease in the time to hire means agencies can be more competitive in attracting top talent to their organizations, enhancing their ability to meet their missions. A decrease in the time required to develop position descriptions is also expected to lead to cost savings for agencies – freeing up manager and HR specialist time for other tasks. Additionally, the streamlined generation of PDs will enable agencies to move away from “standard” PD processes which will allow for better organizational design and workforce planning outcomes. The project is expected to also lead to higher-quality PDs that adhere to classification standards. This will improve position and performance management and talent acquisition in agencies, ensuring organizations are structured and staffed appropriately to meet their missions.

12_outputs: USA Class will provide draft PD content generated through large language models (LLMs) and informed by a PD library curated by OPM experts and user responses to prompt questions about the position’s purpose, regular and recurring work, and knowledge required. The questions are intended to be straightforward and easily answered by the manager in a few minutes. The application will then send this information in a prompt to the LLM and generates suggested major duty titles and work tasks. The user (manager or classifier) can edit the draft content suggested by the system and makes the decision whether to include it in the PD or not. Future iterations will include a certification by a classifier and others that they accept/approve the final content of the full PD.
16_dev_stage: Acquisition and/or Development
17_impact_type: Rights-Impacting

18_date_initiated: 3/1/2024
19_date_acq_dev_began: 3/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 24322624F0027
24_hisp_support: No
31_data_catalog: Yes
33_agency_data: The AI system uses a RAG (Retrieval-Augmented-Generation) architecture including GPT-4o (accessed through OPM’s Azure Open AI environment) that leverages manager-provided responses to a few brief questions as well as a library of curated and validated federal position descriptions. The Position Description library consists of approximately 2,800 PDs developed by OPM/HRS’s Federal Classification Center for internal and external customer agencies.
34_data_docs: Documentation is widely available: Documentation is not only complete, but is widely accessible within the agency, and has an owner and a regular update cadence.
40_has_ato: Yes
41_system_name: USAData Analytics
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
51_extension_request: No – Agency did not request an extension for this use case.
52_impact_assessment: Yes
53_real_world_testing: Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. 
54_key_risks: We are aware that AI models can perpetuate existing biases present in the training data, leading to unfair or discriminatory classification outcomes. 

It could be a risk to user adoption that Federal managers and HR professionals do not understand how USA Class uses generative AI and whether or not it is safe and effective for them to use.

USA Class could produce incomplete, inaccurate, or inconsistent suggestions due to limitations in training data or other gaps. 

USA Class users could rely too much on the suggested position description content and not review and edit or adjust their prompts enough to get as precise results as are possible leading to overly generic or incomplete PDs. 
55_independent_eval: Yes – by an agency AI oversight board not directly involved in the system’s development
56_monitor_postdeploy: No monitoring protocols have been established: Necessary infrastructure needed to perform monitoring of AI systems in production is not available and/or a plan to monitor models has not yet been established.
57_autonomous_impact: No - Some individual decisions or actions require direct human oversight.
59_ai_notice: Online - in the terms or instructions for the service.
61_adverse_impact: No
62_disparity_mitigation: These activities are expected as the current prototype matures in the pilot process.
63_stakeholder_consult: None of the above
65_appeal_process: Yes
67_opt_out: Yes