GreatNorthCollective/greatnorth-us-federal-ai-inventory-2024 huggingface dataset:
2_use_case_name: Information Visualization Platform (InfoViP) to Support Analysis of adverse event reports
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Information Visualization Platform (InfoVIP) supports post-market surveillance using AI to assist prioritization and analysis of adverse event reports by extracting data from unstructured case narratives and combining it with structured data to support analysis and review of adverse event reports. The tool assists with   case series analyses, creating data visualization, and classifying adverse event reports by information quality.

Improve the efficiency in preparing and analyzing data during the review and evaluation of adverse event reports.
12_outputs: Performs Natural Language Processing (NLP) and applying Machine Learning (ML) algorithm to extract data from unstructured case narratives and combine with structured data to support analysis and review of adverse event reports.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 9/1/2019
19_date_acq_dev_began: 9/1/2019
22_dev_method: Developed with contracting resources.
23_contract_piids: BAAFY23C2AWP1
31_data_catalog: Yes
33_agency_data: Adverse Event data in FAERS
34_data_docs: Documentation has been partially completed
35_demo_features: Sex/Gender
40_has_ato: No
42_dev_tools_wait: More than 12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: LLM-Assisted VAERS Analyses
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Health & Medical
10_commercial_ai: None of the above.
11_purpose_benefits: Build capacity for and assess the application of a LLM to VAERS (Vaccine Adverse Events Reporting System) to provide reviewers adhoc VAERS queries and efficiently generate customized query outputs

Efficiently generate customized query outputs of VAERS queries for reviewers.
12_outputs: To provide reviewers adhoc VAERS queries.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 12/1/2023
19_date_acq_dev_began: 12/1/2023
22_dev_method: Developed in-house.
31_data_catalog: Yes
33_agency_data: Adverse Events data in VAERS
34_data_docs: Documentation is widely available
35_demo_features: Sex/Gender, Age, Race/Ethnicity 
40_has_ato: Yes
41_system_name: CBER HIVE
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Pharmacovigilance machine learning to detect excess safety signal
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Government Services (includes Benefits and Service Delivery)
10_commercial_ai: None of the above.
11_purpose_benefits: Post-market adverse event reporting

Using a computer algorithm assists FDA in finding "needle in the haystack" events that are difficult to uncover with manual review of reports
12_outputs: Generate potential safety signals to investigate further using statistical based algorithms
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 1/1/2015
19_date_acq_dev_began: 1/1/2008
20_date_implemented: 1/1/2015
22_dev_method: Developed with contracting resources.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
37_custom_code: Yes
40_has_ato: Yes
41_system_name: PV Analyzer
43_infra_provisioned: No
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.
2_use_case_name: Module 3 Facilties
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Objective of this use case was to identify and extract all drug manufacturing facilities reported within unstructured module 3 submissions from marketing applications to build an comprehensive inventory of drug facilities

The extracted data are presented in a dashboard, along with additional data, for downstream human analysis.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 7/1/2023
19_date_acq_dev_began: 7/1/2023
20_date_implemented: 2/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: OPPQ Policy Bot (AKA Policy and Guidance Documents Search) 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Helps policy staff in searching documents containing regulations and responding to inquiries, and other policy related work. 

In developing policy and responding to inquiries, OPPQ staff often need to reference  previously published policy documents in order to ensure consistency. This AI-enabled tool will allow policy leads and other staff to quickly identify and query the appropriate documents.
12_outputs: Identifies applicable published MAPPS, guidances, and policy related documents based on queries from users. Includes link to the document for further human review.
16_dev_stage: Implementation and Assessment
17_impact_type: Neither
18_date_initiated: 9/1/2024
19_date_acq_dev_began: 9/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: CDER policy documents
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Packaging Materials and Suppliers
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Objective of this use case aims is to extract data from unstructured sources to build an inventory of drug packaging materials and their suppliers to support staff on conducting drug supply chain analysis

The extracted data are presented in a dashboard for downstream analysis.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 7/1/2023
19_date_acq_dev_began: 7/1/2023
20_date_implemented: 9/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: Applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Process Large Amount of Submitted Docket Comments
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: To enhance the automatic process of dockets, we have created an AI/ML tool in CBER/HIVE that automatically download dockets and process them to accelerate the review of docket comments, significantly improving the efficiency and accuracy of our regulatory processes

Efficiently generate customized query outputs of VAERS queries for reviewers.
12_outputs: To provide reviewers adhoc VAERS queries.
16_dev_stage: Implementation and Assessment
17_impact_type: Neither
18_date_initiated: 9/1/2021
19_date_acq_dev_began: 11/1/2021
22_dev_method: Developed in-house.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: Public comments for various dockets.
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CBER HIVE
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Real World Data/Evidence 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Identify industry unstructured submissions containing Real World Data/Evidence (RWD/E) by analyzing parsed content for likely indicators of Real World Data/Evidence RWD/E to support congressional reporting.

The extracted data are presented in a dashboard for downstream human analysis.
12_outputs: Supports extracting text from unstructured documents and tagging for documents containing Real World Evidence and Real World Data
16_dev_stage: Implementation and Assessment
17_impact_type: Neither
18_date_initiated: 2/1/2024
19_date_acq_dev_began: 2/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Regulatory Starting Material 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Extract RSMs and their suppliers from unstructured module 3 industry submissions to create an inventory that will illuminate the upstream supply chain.

The extracted data are presented in a dashboard for downstream analysis.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 9/1/2022
19_date_acq_dev_began: 9/1/2022
20_date_implemented: 8/1/2023
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Resource Capacity Planning
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Forecasting human drug review program submissions and corresponding FDA workload 

Improve the forecasting of human drug review program expected workload in order to adjust fee revenue and fees to fund resources needed to complete expected workload  
12_outputs: Forecasts workload submissions across major user fee programs to support fee setting for the human drug review programs
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 1/1/2016
19_date_acq_dev_began: 1/1/2017
20_date_implemented: 8/1/2020
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: HHSF223201510012B
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
33_agency_data: FDA systems including DARRTS and Panorama
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.
2_use_case_name: Supply Chain Resilience Program
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Emergency Management
10_commercial_ai: None of the above.
11_purpose_benefits: Forecasting demand of medical devices and supplies.

Forecast demand for critical devices during a variety of scenarios (e.g. natural disaster, PHE)  
12_outputs: Aids in forecasting demand for critical devices under a variety of scenarios.    
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 2/1/2022
19_date_acq_dev_began: 2/1/2022
20_date_implemented: 4/1/2023
22_dev_method: Developed with contracting resources.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: FiDLE; RSCP and is being moved to CEDh-OSCR
42_dev_tools_wait: 6-12 months
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.
2_use_case_name: AI Audit Resolution Assistant
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: HRSA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: The objective of this initiative is to reduce the time for HRSA auditors to review/resolve Single audits. HRSA has experienced an exponential increase in Single audits which is attributable in COVID funding attributable to the Provider Relief Fund programs. 

Success will be measured through the increase in the number of Single audits assigned to each auditor.  e.g., improved technological capabilities should allow HRSA Auditors to more efficiently review and closeout Single audits. 
12_outputs: HRSA will utilize generative AI to streamline the Single Audit resolution process and the creation of Management Decision Letters to formally close out and resolve Single audit findings.  The AI Audit Resolution Assistant (AIARA) includes a vector database comprised of Single audit documents assigned to HRSA, creates a retrieval augmented generation which integrates with a large language model to intelligently summarize audit findings and recommendations, and provides chatbot capability and reduce the cognitive load on HRSA auditors for any audit-specific questions. 
16_dev_stage: Implementation and Assessment
17_impact_type: Neither
18_date_initiated: 1/1/2024
19_date_acq_dev_began: 3/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75R60222F80097
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: HRSA did not use agency data to train or fine-tune the large language model. HRSA did use publicly-available data from the Federal Audit Clearinghouse to evaluate the accuracy and precision of LLM outputs. 
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: At the time of this response, HRSA is evaluating multiple HHS infrastructuure hosting services for Azure OpenAI platform with ATO in place.  A decision on a specific environment is forthcoming.
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.
2_use_case_name: Knowledge Navigator
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: HRSA
8_topic_area: Government Services (includes Benefits and Service Delivery)
10_commercial_ai: None of the above.
11_purpose_benefits: The objective is to develop an AI model that can answer detailed and complex questions about the key programmatic document, Application and Program Guidance (APG), that is issued annually before the application cycle opens. The PoC LLM has Applicant and Program Guidance (APG) documents for 10 loan repayments and scholarship programs.

This will allow loan repayment and scholarship analysts and call center agents to  better respond to public inquiries from applicants and participants.  If successful the Knowledge Navigator could be made available to applicants and participants and reduce the volume of calls to the BHW Call Center along with the number of call center agents.  
12_outputs: BHW has implemented a proof of concept Generative AI (GenAI) Large Language Model (LLM) Knowledge Navigator (KN) to support National Health Service Corps (NHSC) and Nurse Corps loan repayment and scholarship program analysts and call center agents respond to program applicants and participants. 
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 2/1/2024
19_date_acq_dev_began: 2/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75R60224F34005
31_data_catalog: No
33_agency_data: HRSA Application and Program Guidance (APG) documents for 10 HRSA loan repayment and scholarship programs.  All of which are public documents.  Here is an example: https://nhsc.hrsa.gov/sites/default/files/nhsc/loan-repayment/nhsc-students2service-lrp-application-program-guidance.pdf
34_data_docs: Documentation is widely available
40_has_ato: Yes
41_system_name: Bureau of Health Workforce Management Information System Solution (BMISS) Platform (UII: 009-000004158
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: No
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.