2_use_case_name: AI use for CPC classification
3_agency: Department of Commerce
3_abr: DOC
4_bureau: USPTO - United States Patent and Trade Office
8_topic_area: Mission-Enabling
10_commercial_ai: Searching for information using AI.
11_purpose_benefits: Automatically assign patent utility classification (CPC) symbols to patent utility documents to reduce classification time, provide agency cost savings, and improve consistency of classification practices.
12_outputs: Predictions of CPC symbols relevant for each utility patent document inputted
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 10/25/2017
19_date_acq_dev_began: 10/9/2019
22_dev_method: Developed with contracting resources.
23_contract_piids: 1333BJ21D00280001 (ID/IQ contract number)
24_hisp_support: Yes
25_hisp_name: United States Patent and Trademark Office
26_public_service: Applying for a patent
27_public_info: No
29_contains_pii: No
30_saop_review: Yes
31_data_catalog: Other
31_question_type: While we have access, the use case data used is not contained in those databases. The use case uses USPTO owned classification (CPC) data on patent utility documents since 1976; USPTO owned patent document text data; agency published public data of CPC scheme and definitions.
33_agency_data: The use case uses USPTO owned classification (CPC) data on patent utility documents since 1976; USPTO owned patent document text data; agency published public data of CPC scheme and definitions.
34_data_docs: Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.
35_demo_features: Other
35_question_type: Does not use any demographic variables.
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: OD/BD master system
42_dev_tools_wait: More than 12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
51_extension_request: No – Agency did not request an extension for this use case.
52_impact_assessment: No
53_real_world_testing: Benchmark evaluation: Testing of the AI model has involved the use of benchmarks (either publicly available or internally created) to estimate performance in real-world settings, but has not been tested in an operational environment. 
54_key_risks: unable to meet performance quality metrics- will be identified by benchmark evaluation and future operation-environment testing
55_independent_eval: Does not apply, use case is neither safety or rights impacting.
56_monitor_postdeploy: use case is not safety impacting or rights impacting; however, a monitoring process for AI performance is under development
57_autonomous_impact: use case does not impact safety or rights; however – Other,  Immediate human intervention is not practicable; however, an appropriate fail-safe has been implemented. 
57_question_type: AI decisions provide initial CPC symbols to patent documents, the AI decisions are reviewed by appropriate patent technical experts in the patent examination process
59_ai_notice: Other 
59_question_type: Use case is not safety or rights impacting; however Agency has notified the public that AI is being explored for classification; appropriate notice to public will be provided before full implementation.
61_adverse_impact: No
62_disparity_mitigation: Planned performance monitoring to evaluate performance across technologies and patent applications
63_stakeholder_consult: direct user testing
outreach to relevant labor organizations
other
63_question_type: Autoclass has regularly reported to and received feedback from public stakeholders thru Patent Public Advisory Committee (PPAC) and is working with internal stakeholders for direct quality assessment/testing to benchmark AI performance
65_appeal_process: No – it is not operationally practical to offer this.
66_no_appeal_reason: Use case is not safety impacting or rights impacting; however - Internal stakeholders do and will have the option to change the decision if they disagree; decision is not appeallable or contestable by external stakeholders as the decision.
67_opt_out: Other
67_question_type: No - the use case has be determined to not impact safety or rights; there is no planned opt-out mechanism by individual as the AI use case is for agency internal functioning