GreatNorthCollective/greatnorth-us-federal-ai-inventory-2024 huggingface dataset:
2_use_case_name: CLAT (Computerized Labeling Assessment Tool) 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: CLAT uses computer programming, image processing, and AI to assist the human review of drug labeling, including container labels, carton labeling, prescribing information, and Instructions for Use. CLAT assists the human visual comparison of drug labeling against a library of FDA regulations, guidances, standards, and best practices. CLAT identifies potential labeling deviations for human evaluation. 

CLAT is used to improve efficiencies in the manual labeling review process to help the reviewer mitigate medication errors and standardize labeling across therapeutic drug classes and product types. CLAT also assists the reviewer to catch deviations between label versions so no unintended errors have been introduced.
12_outputs: The tool processes images of carton and container labeling to identify minimum requirements on a label, identify availability of objects on an image, color differentiation of strength, missing barcode and orientation, incorrect or missing strength statement, error prone abbreviation, look-alike labels and text prominence. This will assist the reviewer by  improving the efficiency of drug labeling reviews, including container labels, carton labeling, prescribing information, and Instructions for Use. 
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 9/1/2020
19_date_acq_dev_began: 9/1/2020
22_dev_method: Developed with contracting resources.
31_data_catalog: Yes
33_agency_data: Label and Container cartons
34_data_docs: Documentation is complete
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.
2_use_case_name: Empirical evaluation of EHR-based signal detection approaches
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Health & Medical
10_commercial_ai: None of the above.
11_purpose_benefits: This supports Electronic Health Record (EHR) signal detection by extracting information using Natural Language Processing (NLP) and expanding the tree based scan statistic (TBSS) methods for EHR-based signal detection.


Extracts data to facilitate the identification of signals for outcomes, improving the manual identification of signals for outcomes derived from electronic health records.
12_outputs: The AI component in this study leverages an Natural Language Processing (NLP) approach to extract data from unstructured clinical notes.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 9/1/2023
19_date_acq_dev_began: 9/1/2023
22_dev_method: Developed with contracting resources.
23_contract_piids: 75F40119D10037 - 75F40123F19010
31_data_catalog: Yes
33_agency_data: Claims-Electronic Health Record (EHR)
34_data_docs: Documentation has been partially completed
40_has_ato: Yes
41_system_name: Sentinel 
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: No
45_compute_request: No
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.
2_use_case_name: FAR-based Facility Signal Detection Tool 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Identifies problem clusters for the flagged signals in an objective manner for triage/review. 

Assist with objectively identifying problem clusters for the flagged signals, which then go through a dedicated triage & review process for further actions, and if needed, CMS (inspection) work activities can be opened. 
12_outputs: Identifies problem clusters for the flagged signals in an objective manner for triage/review. 
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 6/1/2019
19_date_acq_dev_began: 6/1/2020
20_date_implemented: 6/1/2024
22_dev_method: Developed in-house.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
33_agency_data: Internal Field Alert Reports data from LSMV (FAERS tool)
34_data_docs: Documentation has been partially completed
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: No
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: MedWatch Dashboard 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Assist with consistent monitoring and identification of product risks from MedWatch reporting patterns and report content to support review staff.

Supports efficient, objective, risk-based product quality assessments from MedWatch reports for verification by review staff 
12_outputs: Identify product risk signals from MedWatch reports by using time series analysis to flag products and topic modeling to summarize the comments.
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 1/1/2021
19_date_acq_dev_began: 5/1/2021
20_date_implemented: 4/1/2023
22_dev_method: Developed in-house.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
33_agency_data: MedWatch data from LSMV and IQVIA data
34_data_docs: Documentation is complete
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: No
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Quality Surveillance Dashboard (QSD)
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Extracts unstructured text from documents and pools that with other available data to form a dashboard that enables consistent assessment of CDER regulated manufacturing facilities

Assists in the efficient, objective, risk-based assessments of CDER regulated manufacturing facilities.
12_outputs: Identifies and extracts keywords/phrases from unstructured documents and presents sentences containing keywords/phrases in context.
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 1/1/2019
19_date_acq_dev_began: 1/1/2019
20_date_implemented: 3/1/2023
22_dev_method: Developed in-house.
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Yes
33_agency_data: FDA EIR documents
34_data_docs: Documentation is complete
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: No
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Scalable automated NLP-assisted chart abstraction and feature extraction tool
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Examine the usability of electronic medical records in conducting pharmacoepidemiology studies through a case study of montelukast use among patients with asthma and neuropsychiatric events

The project will involve an “end-to-end” implementation of a full pharmacoepidemiologic study using one or more product-outcome pairs to demonstrate how NLP and other advanced analytic methods or tools can be used to accurately identify study outcomes and covariates from unstructured Electronic Health Record (EHR) data.
12_outputs: Natural language processing (NLP), a computational linguistics technology, will be used to read, interpret and organize health data that may be contained within unstructured data fields within EHRs. This activity intends to build a semi-automated system that uses advanced methods and tools, including NLP, to enable large-scale outcome identification, confounder feature extraction, and longitudinal contextualization of Electronic Health Record (EHR) data.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 10/1/2022
19_date_acq_dev_began: 9/1/2022
22_dev_method: Developed with contracting resources.
23_contract_piids: 75F40119D10037 - 7540119F19002
31_data_catalog: Yes
33_agency_data: Claims-Electronic Health Record (EHR) - (structured and unstructured)
34_data_docs: Documentation has been partially completed
40_has_ato: Yes
41_system_name: Sentinel 
42_dev_tools_wait: 6-12 months
43_infra_provisioned: No
45_compute_request: No
47_timely_resources: Yes
49_existing_reuse: None: This use case does not re-use any internally developed tooling or managed infrastructure from any other AI development efforts within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.
2_use_case_name: Annual Report CMC
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Objective of this use case was to extract CMC changes reported within unstructured annual report industry submission documents to build a complete repository for downstream analysis

The extracted data are presented in a dashboard (along with other data) for downstream human analysis.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 1/1/2024
19_date_acq_dev_began: 1/1/2024
20_date_implemented: 5/1/2024
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: Applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Application-DMF Reference
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Extracts references to DMFs from marketing application (ANDA/NDA/BLA) submissions. These submission documents may be structured (356H form) or unstructured (eCTD module 1-4). This pipeline parses content from these documents, extracts DMF references (e.g., ANDA123456 references DMF123456), and exposes the data in a structured format for analysis. 

The extracted data are presented in a dashboard, along with additional data, for downstream human analysis. The purpose of this use case is to build a comprehensive inventory of all application to DMF references to support OPQ in reviewing applications since many application to DMF references were previously missing from structured sources.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 5/1/2023
19_date_acq_dev_began: 5/1/2023
20_date_implemented: 12/1/2023
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Artificial Intelligence-based Deduplication Algorithm for Classification of Duplicate Reports in the FDA Adverse Event Reports (FAERS) 
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: The deduplication algorithm is applied to all historical and incoming case report submissions to the FDA Adverse Event Reporting System (FAERS) to identify duplicate individual case safety reports (ICSRs). Unstructured data that is in free text FAERS narratives is processed through a natural language processing system to extract relevant clinical features. Both structured and unstructured data are then used in a probabilistic record linkage approach to identify duplicates. Application of the deduplication algorithm is optimized for processing entire FAERS database to support the data mining analytics process. 

Enhance the efficiency of removing duplicate FAERS reports to support data mining.
12_outputs: Uses Natural Language Processing (NLP) to extract out clinical features from free text narratives in a case safety report and uses them in the identification of duplicative case safety reports in the FAERS database.
16_dev_stage: Implementation and Assessment
17_impact_type: Neither
18_date_initiated: 12/1/2023
19_date_acq_dev_began: 9/1/2024
22_dev_method: Developed with contracting resources.
23_contract_piids: BAAFY23C2AWP1
27_public_info: No
29_contains_pii: Yes
30_saop_review: No
31_data_catalog: Yes
33_agency_data: Adverse Event data in FAERS
34_data_docs: Documentation is complete
35_demo_features: Sex/Gender
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: No
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code from a different use-case: This use case re-uses production level code and/or internally developed code libraries for model development from existing AI systems or repositories inside the agency.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Development of automation tools and data warehouse to facilitate BE assessments
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Enhancing and streamlining the data entry associated with the review of bioequivalence data in Abbreviated New Drug Application (ANDA) submissions

Reduce manual effort by preparing data for analysis and exporting results as part of the bioequivalence review. This tool will facilitate data collection and Information retrieval from the firm-submitted ANDA packages but will not involve reviewers' evaluation or judgement.
12_outputs: Transform submitted PK data into a usable format so that the reviewer can perform the routine bioequivalence analysis using the authorized pre-written SAS codes. Then it exports results to a draft bioequivalence review  document with the firm-submitted eCTD tables.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 5/1/2018
19_date_acq_dev_began: 5/1/2018
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40120F80605
31_data_catalog: No
34_data_docs: Documentation is widely available
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: DMF (Drug Master File) Facilites
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Extracts facility references from Type II (Drug Substance) DMF manufacturing submissions (i.e., DMF123456 discloses that it uses Facility X for manufacturing and Facility Y for stability testing). These DMF submissions may include structured documents (3938 form) or unstructured documents (eCTD module 3). This pipeline parses content from these documents, extracts facility references and supply chain roles (i.e., FEI 6789 – manufacturing facility), and exposes the data in a structured format for analysis. This allows OPQ to identify discrepancies between facilities reported on marketing applications and those disclosed by DMFs referenced by those applications. 

The extracted data are presented in a dashboard, along with additional data, for downstream human analysis. The ultimate purpose of this use case is to build a comprehensive inventory of facilities reported within DMF submissions as well as the applications that reference those DMFs.
12_outputs: Support information extraction from unstructured documents
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 6/1/2022
19_date_acq_dev_began: 6/1/2022
20_date_implemented: 6/1/2023
22_dev_method: Developed with both contracting and in-house resources.
23_contract_piids: 75F40124A00003
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: No
33_agency_data: Applicant submissions
34_data_docs: Documentation is widely available
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: Yes
41_system_name: CDEROne Analytics: FISMA-high environment
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.
2_use_case_name: Four Part Harmony Identification
3_agency: Department of Health and Human Services
3_abr: HHS
4_bureau: FDA
8_topic_area: Mission-Enabling (internal agency support)
10_commercial_ai: None of the above.
11_purpose_benefits: Accuracy of language used to report back deficiencies to submitters.

This will reduce the amount of time it takes and reduce staff burden, thus allowing for MDUFA requirements to be met.
12_outputs: This will reduce the amount of time it takes to verify and identify issues with language used for deficiency letters and ensure the adherence to the language necessary.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 9/1/2023
19_date_acq_dev_began: 9/1/2023
22_dev_method: Developed in-house.
31_data_catalog: Yes
33_agency_data: Submission data and documents
34_data_docs: Documentation is widely available
40_has_ato: Yes
41_system_name: CEDh
42_dev_tools_wait: 6-12 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.
50_internal_review: Documentation has been developed: Complete documentation detailing model performance across a range of benchmarks, architecture, relevant features and information regarding the appropriate use of the model for predictive tasks has been created.