2_use_case_name: Deep-Learning approaches to develop candidate lists of terms for use in text searches 
3_agency: Department of Veterans Affairs
3_abr: VA
4_bureau: VHA: Veterans Health Administration
8_topic_area: Health & Medical
10_commercial_ai: None of the above.
11_purpose_benefits: The intended use is to increase the comprehensiveness of term lists to be used in text-matching approaches to identifying concepts of interest in free-text medical record notes. Text-matching approaches are an efficient method for identifying text of interest in medical notes.  The success of text-matching approaches depends on the quality of term lists used to search for a concept of interest.  While humans can generate word lists for text-matching approaches, the generated word lists are likely to be limited and biased by the vocabulary, language habits, population exposure, and local dialects of the human generating the list.  Deep learning methods can help to augment candidate term lists to help overcome some of these challenges.  Once generated these augmented candidate terms are reviewed by subject matter experts for appropriateness and accuracy of the suggested terms for the concept and use case of interest.  Here we (1) query pre-trained large language models (e.g., ChatGPT or Llama) through a user interface that allows prompting (such as the OpenAI API or CodeLlama) and/or (2) query word and phrase “embedding” models (e.g., Word2Vec and Phrase2Vec) to expand candidate term lists for review by subject matter experts.  Our team has found both approaches effective for expediting the generation of more comprehensive term to concept mappings for use in text-matching algorithms that increase the accuracy of clinical concept extraction.   We expect this use of AI to improve initial brainstorming of terms related to a given concept, resulting in a more sensitive approach to finding information of interest in free-text clinical notes. 
12_outputs: The outputs of this use of AI is a candidate list of terms related to a concept of interest.  This candidate list is combined with suggestions generated by human experts and then these are reviewed by a subject matter expert to generate a final list of terms for use in text-matching algorithms to extract medical record free text mentions of interest. For the pre-trained Large Language model approach, the input is a query prompting for synonyms of terms related to the concept of interest.  The output is the answer provided by the LLM, including the suggested candidate terms and the original terms that were stated in the query.  For the word and phrase embedding models, the input is an initial list of terms that are relevant to the concept of interest and the output are the suggested candidate terms, ranked by their statistical similarity to the original terms, based on the transformation of a clinical corpus into vector space or an “embedding model”.  Regardless of the type of model used, all suggested candidate terms are reviewed by subject matter experts to ensure quality term lists are used in all text searches.
16_dev_stage: Operation and Maintenance
17_impact_type: Neither
18_date_initiated: 10/1/2022
19_date_acq_dev_began: 10/1/2022
20_date_implemented: 10/1/2022
22_dev_method: Developed in-house.
24_hisp_support: No
27_public_info: No
29_contains_pii: No
30_saop_review: No
31_data_catalog: Other
31_question_type: The subject matter expert teams relevant to the concept of interest create the needed queries and initial term lists.
34_data_docs: Documentation is missing or not available: No documentation exists regarding maintenance, composition, quality, or intended use of the training and evaluation data.
37_custom_code: No
38_code_access: No – agency does not have access to source code.
40_has_ato: No
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Re-use production level code and/or data products: This use case re-uses production level code as well as data products that have been published or made widely available for data science teams for within the agency.
50_internal_review: No documentation is available: No documentation detailing model performance metrics, model architecture, features and intended use of the models have been created or are currently accessible to other data science teams within the agency.