bluuebunny/crossref_metadata_2025_split huggingface dataset:
Novel approach to implementing natural language processing for clinical staging of non-small-cell lung cancer.
Authors: Shah Manan P , Prichard Laura , Glaspy John A , Vu Vu , Wang Jason , Gelvezon Alex V , Cummings Amy Lauren 
Published: May 2024

e13624 Background: Cancer staging is instrumental in driving clinical management and trial enrollment, but staging data are generally unreliable and unstructured in the electronic health record (EHR). Advances in natural language processing (NLP) may facilitate clinical staging and documentation [1], but challenges to real-world implementation include (1) automatically identifying appropriate patients and reports from the EHR and (2) developing an unbiased dataset for training and validation [2]. We describe our institution’s novel approach to overcome these barriers while building an in-house NLP pipeline for clinical tumor staging of non-small-cell lung cancer (NSCLC). Methods: We identified patients by searching our EHR (Epic) for a molecular analysis test ordered specifically for pathological diagnoses of NSCLC at our institution. We used the test order date as the diagnosis proxy date (DPD). For each patient, we extracted imaging reports up to 16 weeks before and 6 weeks after the DPD. To derive primary tumor size, we analyzed the CT Chest or PET/CT report closest to the DPD using an oncology-trained NLP text extraction and labeling tool (John Snow Labs). We cleaned all extracted tumor size entities and identified the largest measurement linked to the lungs. We compared primary tumor measurements from the NLP pipeline to those in a preexisting, manually compiled cancer registry (CNEXT). We manually analyzed discrepancies through chart review. Results: 542 patients with a DPD between 11/2016 - 9/2023 were processed through the NLP pipeline. Of 443 patients with valid values in both the pipeline and CNEXT, 53% (234) were exact matches, and 20% (90) had a close match (within 0-5mm), yielding a 73% accuracy rate for values within 5mm. When mismatched values were manually reviewed, several cases in CNEXT were found to have a DPD differing by more than 3 months and tumor sizes derived from external reports. When these cases were excluded, 320 of the remaining 349 patients had valid values in both the pipeline and the updated manual review. In this refined population, 66% (213) were exact matches, and 15% (48) had a close match, yielding an 82% accuracy rate for values within 5mm. Conclusions: To our knowledge, this is the first report of a pathology-based method to automatically and reliably identify patients with NSCLC and their relevant imaging reports directly from the EHR. We used a prebuilt NLP tool to derive primary tumor sizes with relatively high accuracy and found that adding flags for timeline discrepancies and external reports can further improve validity. As we near completion of analogous pipelines for node and metastasis staging, we will develop methodology to identify subgroups of patients that can be clinically staged with near-perfect accuracy, ultimately aiming to substantially limit manual staging of uncomplicated cases. 1. Puts 2023. 2. Wang 2022.
Advancing precision medicine in cutaneous oncology through 3D high-frequency ultrasound imaging.
Authors: Henke Julian , Jeong Jiwoong Jason , Yang Xiaofeng , Khan Mohammad Khurram 
Published: May 2024

e13625 Background: Clinical assessment of tumor depth and shape often requires biopsy, especially when evaluating skin cancers. High Frequency Ultrasound (HFUS) is a non-invasive clinical tool that can quickly produce high resolution images ideally suited for evaluating topical tumors. In comparison to other imaging modalities, HFUS do not always provide the spatial data required to model 3-dimensional (3D) tumors. Currently, 3D-images are an amalgamation of many segmented image slices. However, in outpatient clinics it is unrealistic for HFUS devices and clinicians to gather large sets of image data. In our present original research, we aimed to overcome limited number of HFUS and dermatoscopic images by constructing a simple framework to process sparse and non-spatial data into working 3D skin tumor models. Methods: We analyzed two different skin tumors, the dermal duct tumor (DDT) and basal cell carcinoma (BCC), each with different number of HFUS images at 30Hz and 50Hz respectively. 1 set of 6 parallel images were obtained for the BCC, while 2 orthogonal sets of 6 parallel images were obtained for the DDT. For each tumor, a single dermatoscopic photo and sets of bitmap HFUS images with a resolution of 1024 x 512 were taken. Our study was IRB-approved. Mathematica and MATLAB, both with accessible documentation, were used to process images. Results: We developed a novel algorithmic approach to produce 3D tumor models with limited medical data. Both tumors appear corporally hypoechoic under HFUS. Mathematica was used to first reduce HFUS image noise and binarize the tumor region with a bilateral Gaussian filter. The user-friendly MATLAB Image Segmenter application manually filled the region of interest. Next, Mathematica was used to orient and plot tumor borders in 3D. Finally, we encompassed the borders with a 3D convex hull to conservatively predict tight margins around the tumor. Conclusions: Our simplified approach allows clinicians and scientists to produce 3D renderings of tumor margins. Margin control must always be balanced with a positive surgical outcome. By approaching our 3D-model with a convex hull around sampled border cross-sections, we chose a conservative approach that focuses skin tumor margin evaluation. Our results promote precision medicine and planning into the current standard of skin cancer treatment. Our findings can allow tracking tumor progression or treatment possible via non-invasive evaluation of tumor depth and contour. An aim of our present research is to lower the barrier of 3D construction in clinical settings. Simple algorithms, such as shown in our work, transform orthogonal and sparse data into usable data that can be implemented to train machine learning and artificial intelligent programs. Future research tailoring this approach may allow instant programming of HFUS images to non-invasively render cutaneous tumor borders and depth in real time during clinical examination.
Performance of three commercially available large language models and one locally fine-tuned model at preparing formal letters to appeal medical insurance denials of radiotherapy services.
Authors: Kiser Kendall , Waters Michael , Reckford Jocelyn , Lundeberg Christopher , Abraham Christopher 
Published: May 2024

e13630 Background: As many as 60% of prior authorization requests are denied, yet coverage approval occurs for more than 60% of appeals for some therapies. Appeal processes encumber providers and increase burnout, but large language models (LLMs) may aid providers by drafting appeal letters. We evaluated LLM performance at this task for radiotherapy denials. Methods: Three commercially accessible LLMs were evaluated: generative pre-trained transformer 3.5 (GPT3.5), GPT4, and GPT4+web with internet search capacity (OpenAI, Inc., San Francisco, CA). A fourth LLM, GPT3.5-FT, was developed by fine-tuning GPT3.5 in a HIPAA-complaint local environment. The fine-tuning training data comprised 53 insurance denial appeal letters prepared by radiation oncologists and paired prompts describing the clinical history and appeal intent. Training data were enriched in appeal letters for proton radiotherapy, stereotactic body radiotherapy, and image-guided radiotherapy for myriad clinical scenarios. Twenty prompts, each requesting a letter for a simulated patient history, were programmatically presented to the LLMs. Three radiation oncologists, who were blinded to the LLM source, scored letter outputs across four domains: language syntax and semantics, clinical detail inclusion, clinical reasoning validity, and overall readiness for insurer submission. Additionally, one radiation oncologist scored the authenticity and relevance of literature sources cited in output letters, which were requested by several test prompts. Interobserver agreement between radiation oncologist scores was determined by Cohen’s kappa coefficient. Scores were compared between LLMs with non-parametric statistical tests. Results: Agreement between radiation oncologists’ scores was moderate-to-excellent across all domains (median κ = 0.68, minimum κ = 0.41). GPT3.5, GPT4, and GPT4+web drafted letters that, by mode average, were semantically and syntactically clear, included all provided clinical history without confabulation, clinically reasoned with few necessary revisions, and overall were submissible to an insurer with minor revisions. GPT4 and GPT4+web clinically reasoned better than GPT3.5 (p values < 0.001). In contrast, GPT3.5-FT performance was inferior to other LLMs across all domains (p values < 0.001). LLMs were poor at identifying, citing, and summarizing relevant literature unless provided in the prompt. Conclusions: LLMs can draft insurance appeal letters for radiotherapy services that require few revisions yet are poor at referencing relevant literature. Contrary to our hypothesis, fine-tuning with data from our department compromised LLM performance.
A prospective study comparing AI-based clinical trial eligibility screening with traditional EMR-based screening.
Authors: Joseph-Thomas Jiby , Green Cristina , Tibrewal Abhishek , Dupont Frank , Flach Jen , Goldstein Amelia , Vaidya Vivek Prabhakar , Kulkarni Rajeev , Parikh Ravi Bharat 
Published: May 2024

e13636 Background: For clinical trials of novel oncology therapies, screening patients for eligibility relies on research staff manually abstracting complex eligibility criteria, which is time-consuming and subject to inconsistent interpretations. AI algorithms based on structured data fields may facilitate automated eligibility criteria extraction, assure more complete screening for the benefit of all patients, improve workflow for research staff, and assure no bias in who is reviewed. AI algorithms can support abstraction and imputation approaches to assure the greatest number of clinical records get reviewed. Prospective validation of such algorithms and approaches for sponsored registrational clinical trials is lacking from literature. Methods: We conducted a prospective controlled observational study of an AI model to prioritize potentially eligible patients, as part of a phase 3, randomized, multi-site therapeutic trial for Relapsed or Refractory Multiple Myeloma. In this observational study, we studied eligibility criteria focused on measurable disease and prior anti-myeloma treatments. The AI eligibility model extracted structured eligibility criteria to predict likelihood of meeting study eligibility. A single CRC (Clinical Research Coordinator) allocated 40 hours to screening in both control and intervention phases of the study. In the control phase, the CRC screened myeloma patients’ electronic health records in alphabetical order by last name. In the intervention phase, the CRC screened a different set of patients ranked by the eligibility likelihood model. Descriptive statistics were used to compare primary outcomes of efficiency (percentage of patients watch-listed) and speed (average time to screen a patient) between control and intervention phase. Patients marked as watch-listed meet eligibility criteria but need additional testing or disease progression before becoming eligible for the study. Results: The AI Eligibility method captured the same share (19%1) of screened patients as the EMR method, with a 3.3x (12.5 versus 41 minutes) improvement per screened patient. Conclusions: The AI eligibility method allows research staff to screen three times more patients with similar outcomes as compared to routine CRC EMR screening. Future work can leverage AI to identify and predict patient progression, further enhancing screening efficiency. [Table: see text]
Implementing a natural language processing pipeline to expedite clinical chart review to assess germline genetic testing in veterans with breast cancer.
Authors: Patil Vikas , Mcshinsky Richard , Halwani Ahmad Sami , Morreall Deborah , Li Chunyang , Rasmussen Kelli M , Yong Christina , Burningham Zachary R. , Menendez Carolyn S. , Chiba Akiko , Moss Haley , Colonna Sarah Violet , Rowe Kerry G. , Kelley Michael J. 
Published: May 2024

e13640 Background: The Making Advances in Mammography and Medical Options for Veterans Act of 2022 mandated that the Department of Veterans Affairs (VA) perform a study of germline genetic testing in Veterans with breast cancer (BC). Assessing whether a Veteran with BC was offered germline genetic testing based on electronic health records requires time-consuming review of unstructured clinical notes by clinical experts who are also extremely busy providing care. To expedite this review process, we implemented a productionized natural language processing (NLP) pipeline to identify relevant concepts and highlight them within a custom-made chart review interface. Methods: Clinical subject-matter experts (SMEs) were convened to identify keywords suggestive of genetic testing. These were categorized into 3 sets: gene-related concepts, VA-specific services, and exams and tests. We then used entity recognition and entity linking to link these concepts to a knowledge base. These were enriched with entities similarly extracted from an open-source reference clinical text on germline genetic testing in BC. The resulting concepts were visualized as a word cloud and reviewed by the SMEs. We implemented a Power BI and Power Apps interface to display clinical notes from 200 randomly selected Veterans, with relevant concepts highlighted. Each Veteran’s notes were reviewed independently by 2 of the 5 SMEs, who assessed whether the Veteran was offered testing, referred to a genetic counseling service, or otherwise had evidence of testing. Disagreements were adjudicated by the whole team. Results: 11622 notes (mean 58/Veteran, 3028 characters/note) were reviewed. Table shows the most prevalent entities among genes, mutations, and exam names. Of the 5 reviewers, 3 annotated 80 patients each in a single session, lasting less than 50 minutes on average. Reviewers had an inter-annotator agreement of 85.5%. Mention of target concepts was associated with germline genetic testing with an accuracy of 80.4%, precision of 76.4%, recall of 99.1%, and F1 of 86.3%. Germline genetic testing rates are detailed in a separate submission. Conclusions: Modern informatics tools such as productionized NLP, the Microsoft Power Platform, and high-performance computing environments allow for rapid implementation of health information technology tools that can enable clinicians to gain critical insights quickly and efficiently from electronic health record data. [Table: see text]
Exploring the literature on artificial intelligence use in oncology.
Authors: Phillips Ruth Anne , Jani Janvi , Bradley Sarah K. 
Published: May 2024

e13642 Background: Artificial Intelligence (AI) allows machines to analyze data in a way that mimics human intelligence. AI has been used in oncology diagnostics such as radiology and pathology. However, the specific processes and tasks that AI can support, especially in oncology, have not been fully explored. Methods: The electronic databases PubMed and MEDLINE were searched from September 1, 2018, to September 1, 2023, for publications related to the use of AI in oncology. Inclusion criteria consisted of English language, article type “study” and keywords Artificial Intelligence, Healthcare, Predictive/Prediction, Fever, Cytokine Release Syndrome, Nursing, Patient Care, Oncology, and Neutropenic Fever applied in a progressive manner to narrow the search. Results: The term Artificial Intelligence yielded over 124,385 studies. This number was reduced to 466 when adding “healthcare, predictive/prediction, and oncology.” Adding “patient care” further reduced the number to 102. Out of the 102, seven studies identified included the term “nursing” and focused on clinician decision support, readmissions, palliative care, and scheduling. One identified study discussed the potential benefits of AI-generated algorithms for risk stratification related to cytokine release syndrome. The keywords “neutropenic fever” did not generate any results. Six studies were identified by adding in the term “fever,” which focused on AI use in specialties outside of oncology. The literature reviewed shows that AI has had limited use in oncology outside of radiology and pathology. Only one study suggested the use of AI would be beneficial in predicting symptoms related to treatment. Small studies were identified showing the value of AI in increasing adherence to schedules, reducing readmissions, and providing precision medicine. Conclusions: AI is a promising tool for the future of oncology. The current availability of AI within oncology practices has been limited. Further studies should be undertaken to identify how AI can reduce provider burnout, care costs, and early identification of symptoms to prevent adverse events.
Detection of contralateral breast cancer in patients with primary breast cancer through AI-assisted mammography.
Authors: Jung Ji-Jung , Kang Eunhye , Byeon Jinyoung , Kim Hong-Kyu , Lee Han-Byoel , Moon Hyeong-Gon , Han Wonshik 
Published: May 2024

e13645 Background: Survivors of breast cancer are at an elevated risk for developing contralateral breast cancer (CBC), a risk that surpasses the incidence of initial primary breast cancer within the general population. Consequently, annual mammography screenings are advocated to monitor for ipsilateral recurrence and the emergence of CBC. However, the sensitivity of surveillance mammography exhibits considerable variation, ranging between 45% and 90%, with notably lower detection rates in younger individuals and those with dense breast tissue. This study explores the efficacy of an artificial intelligence (AI) detection algorithm in mammography for the early identification of CBC prior to the official diagnosis. Methods: A retrospective analysis was conducted on patients treated for primary breast cancer from 2000 to 2021 at Seoul National University Hospital. This involved reviewing serial mammograms post-surgery and adjuvant therapy from 460 patients who subsequently developed CBC (CBC group) and 469 patients who did not (control group). These mammograms were evaluated using the Lunit INSIGHT mammogram AI system, which provides a pixel-level abnormality score between 0 and 100, with higher scores indicating increased abnormality. Scores exceeding 10 were considered indicative of CBC, reflecting a 90% sensitivity based on the AI software's tuning dataset. The diagnostic accuracy of AI-assisted mammography was then juxtaposed with that of specialist radiologists' evaluations. Results: At the point of CBC diagnosis, the median AI abnormality score was markedly higher in the CBC group (22.9 [IQR, 2.1-81.1]) compared to the control group's final follow-up mammogram (0.4 [IQR, 0.1-1.5], p<0.001). The AI system's sensitivity for CBC detection paralleled that of radiologist assessments (59.6% for AI vs. 56.5% for radiologists out of 460 detected cancers), with the AI method achieving an AUROC of 0.836 and a specificity of 91.0%. Notably, AI detected 84 cases (30.7%) at least six months and 59 cases (21.5%) more than a year before the official CBC diagnosis. Conclusions: The application of AI in mammography screening has demonstrated significant potential for the early detection of CBC, identifying cases up to a year before formal diagnosis. This underscores the value of integrating AI tools in mammography to enhance early detection rates of CBC among primary breast cancer patients.
Treatment (Tx) inequities in patients (pts) with metastatic non-small cell lung cancer (mNSCLC) harboring actionable driver oncogenes (ADO) in the first-line (1L) setting.
Authors: Vidal Gregory A. , Stricker Thomas , Ma Esprit , Yu Elaine , Xia Zhiyu , Sheinson Daniel , Sincan Murat , Huang Richard S. P. , Lin Victor T. G. , Zuniga Richard Michael , Jain Neha M. , Martin Richard Lewis , Daniel Davey B. 
Published: May 2024

e13673 Background: Targeted treatments (TT) have been approved for pts with mNSCLC harboring ADO in the 1L setting in recent years. This study aimed to understand if subpopulations of mNSCLC pts harboring ADO do not receive equitable access to guideline recommended 1L TT in US clinical settings. Methods: This retrospective study used the Flatiron Health electronic health record-derived, US nationwide, de-identified database. Pts with de novo mNSCLC (stage IVA or IVB) diagnosed between 4/1/2018-6/30/2023 were included if they were ≥18 y, had ≥2 visits after advanced diagnosis (aDx) and ≥90 days follow-up, received any biomarker testing prior to 1L Tx initiation, and harbored ADO (ALK, BRAF, EGFR, MET, NTRK, RET and/or ROS1). Results: Overall 2165 pts met study criteria: mean age was 67 y, 65% female, 11% Medicare insured, 58% commercial insured, 76% treated in the community setting with 11% Medicare and 64% commercial insured (vs 12% and 41% treated in the academic setting with Medicare and commercial insurance), and 45% had smoking history; of non-missing data, 12% had lowest socioeconomic status (SES), 26% with highest SES, 95% had non-squamous NSCLC, and 83% had ECOG performance status ≤1. The median time from aDx to testing result among the 53% non-Latinx (nL) white, 8% nL Black, 7% Latinx, 13% nL Asian, 7% nL other and 13% missing race/ethnicity was 21, 23, 24, 21, 23 and 20 days, respectively; median time from aDx to TT initiation was 38, 38, 43, 36, 47 and 37 days, respectively. Pts significantly more likely to receive 1L TT were (Table): pts of color vs white pts; Medicare vs commercial insurance at community sites; without vs with smoking history. Pts less likely to receive 1L TT were (Table): community vs academic sites; Medicare vs commercial insurance at academic sites. Consistent findings were seen in the 1817 pts with established 1L ADO (ie. ALK, EGFR or ROS1) (Table). Conclusions: When pts with mNSCLC harboring ADO had biomarker testing before Tx initiation, contrary to prior observed racial inequity in NGS testing, there was no racial inequity in receiving 1L TT. However, inequity in receiving 1L TT was observed by insurance type, smoking history and site of care. Interventions are warranted to address these contributors to 1L TT inequities. [Table: see text]