tbuckley/cabot-search huggingface dataset:
Enhancing oncology research through data sharing and re-use: A five-year review of Vivli's global data platform.
Authors: Rebecca Li, Rebecca Mary Caroline Sudlow, Jennifer OCallaghan
Journal: Journal of Clinical Oncology (2024)

e13670 Background: The last decade has seen a paradigm shift in clinical research with the accessibility of trial data, facilitating secondary analyses and the development of novel predictive models. Vivli a non-profit entity established in 2018, epitomizes this movement with its global platform (vivli.org) dedicated to the facilitating access to and re-use of individual participant data from completed clinical trials for use by researchers worldwide. Methods: Commencing with approximately 2,500 trials from 15 members, Vivli has expanded over five years to encompass more than 7,000 trials involving 3.6 million participants across 50 members who contribute trial data from various sectors (industry, academic, government and non-profit foundation). Results: Oncology trials constitute 11% of the Vivli data pool of 7,000 trials, constituting 36% of Vivli’s research proposals due to the therapeutic area's high-impact potential. Researchers accessing data in the Vivli platform have produced a significant body of scientific work, with 74 oncology-focused publications emerging from their analyses. Notably, Roche's data has been a key resource, denoted by the 28% of all Vivli proposals requesting access to Roche data. Fifty-eight publications have resulted from secondary analysis of Roche oncology data. The motivation for secondary analyses in oncology have been predominantly for new research into treatment effectiveness or safety (39%). Other reasons include confirmation of existing research (15%), individual participant data (IPD) meta-analysis (14%), statistical method (10%), clinical trial design (5%), and summary-level meta-analysis (4%), training/testing (7%), other (5%). Conclusions: The Vivli platform has proven to be a key resource in recent years for the oncology community enabling the integration and publication of new insights. The sharing of IPD is in part motivated to ensure reproducibility of findings; our experience demonstrates that a minority of researchers are interested in reproducibility analysis but the majority of research is focused on new insight generation. The platform's success in facilitating the findability, accessibility and reuse of patient level data is signified by ever increasing numbers of scientific publications emerging from analyses on the Vivli platform and this underscores its role in advancing scientific discovery and development within the field of oncology.
Performance of three commercially available large language models and one locally fine-tuned model at preparing formal letters to appeal medical insurance denials of radiotherapy services.
Authors: Kendall Kiser, Michael Waters, Jocelyn Reckford, Christopher Lundeberg, Christopher Abraham
Journal: Journal of Clinical Oncology (2024)

e13630 Background: As many as 60% of prior authorization requests are denied, yet coverage approval occurs for more than 60% of appeals for some therapies. Appeal processes encumber providers and increase burnout, but large language models (LLMs) may aid providers by drafting appeal letters. We evaluated LLM performance at this task for radiotherapy denials. Methods: Three commercially accessible LLMs were evaluated: generative pre-trained transformer 3.5 (GPT3.5), GPT4, and GPT4+web with internet search capacity (OpenAI, Inc., San Francisco, CA). A fourth LLM, GPT3.5-FT, was developed by fine-tuning GPT3.5 in a HIPAA-complaint local environment. The fine-tuning training data comprised 53 insurance denial appeal letters prepared by radiation oncologists and paired prompts describing the clinical history and appeal intent. Training data were enriched in appeal letters for proton radiotherapy, stereotactic body radiotherapy, and image-guided radiotherapy for myriad clinical scenarios. Twenty prompts, each requesting a letter for a simulated patient history, were programmatically presented to the LLMs. Three radiation oncologists, who were blinded to the LLM source, scored letter outputs across four domains: language syntax and semantics, clinical detail inclusion, clinical reasoning validity, and overall readiness for insurer submission. Additionally, one radiation oncologist scored the authenticity and relevance of literature sources cited in output letters, which were requested by several test prompts. Interobserver agreement between radiation oncologist scores was determined by Cohen’s kappa coefficient. Scores were compared between LLMs with non-parametric statistical tests. Results: Agreement between radiation oncologists’ scores was moderate-to-excellent across all domains (median κ = 0.68, minimum κ = 0.41). GPT3.5, GPT4, and GPT4+web drafted letters that, by mode average, were semantically and syntactically clear, included all provided clinical history without confabulation, clinically reasoned with few necessary revisions, and overall were submissible to an insurer with minor revisions. GPT4 and GPT4+web clinically reasoned better than GPT3.5 (p values &lt; 0.001). In contrast, GPT3.5-FT performance was inferior to other LLMs across all domains (p values &lt; 0.001). LLMs were poor at identifying, citing, and summarizing relevant literature unless provided in the prompt. Conclusions: LLMs can draft insurance appeal letters for radiotherapy services that require few revisions yet are poor at referencing relevant literature. Contrary to our hypothesis, fine-tuning with data from our department compromised LLM performance.
Black patient representation in anti-cancer drug development.
Authors: Alexander B. Karol, Rodrigo Paredes, Yu Fujiwara, Anna Argulian, Himanshu Joshi, Matt D. Galsky
Journal: Journal of Clinical Oncology (2024)

e13680 Background: Black patient enrollment in cancer clinical trials is critical to improving the generalizability of trial results and advancing an understanding of potential differences in drug efficacy and safety in underrepresented populations. Prior research indicated that in 2018, Black patients comprised ~2.5% of accruals to phase III trials in oncology (1). Here, we probed whether the representation of Black patients has increased in contemporary trials. Methods: We systematically searched the clinicaltrials.gov database to review racial demographics for phase III clinical trials. These trials evaluated systemic anti-cancer therapies in adult patients between October 28 th 2013, and October 28 th 2023. Studies were excluded if they did not include racial demographic information. The demographics of the trials and cancer subtypes were recorded. 19-year domestic cancer specific prevalence data was obtained from United States Cancer Statistics as a reference to evaluate the representation of Black patients in the included clinical trials. P-values were calculated with a chi-squared test. Results: 436 trials met criteria for review and 238 trials, enrolling 115,969 patients, met our inclusion criteria. Among these patients, 72,471 (62.5%) were White, 1,895 (1.6%) were Black, 31,996 (35.9%) were from another race. Systemic therapies explored included: immunotherapy (99 trials, 41.6%), targeted therapy (110 trials, 47.9%), chemotherapy (17 trials, 7.6%), and other therapies (7 trials, 2.9%). Black patients were underrepresented in phase 3 trials across the vast majority of tumor types (Table). Sub-group analysis revealed significantly increased Black clinical trial representation in trials beginning enrollment in 2019-2022 versus 2013-2018 (2.3% vs 1.8%, p=0.02). Conclusions: Adult Black patients are significantly under-represented (&lt;2%) in modern oncology trials. However, there has been a significant increase in Black enrollment since 2019. Ongoing efforts are needed to ensure optimal representation of Black patients in oncology trials. 1. Tharakan, Cancer, 2021. [Table: see text]
Integrating electronic health records (EHRs) to facilitate cancer biomarker testing: Real-world implementation barriers and solutions.
Authors: Karen M. Huelsman, Caroline Offit, Wendi Waugh, Christopher McNair, Sandra E. Kurtin, Crystal Enstad, Courtney Rice, Travis Osterman, Peter Paul Yu, Gregg Christian Shepard, Nikki A. Martin, Joseph Kim, Molly Kisiel, Elana Plotkin
Journal: Journal of Clinical Oncology (2024)

e13649 Background: Effective integration between EHRs and cancer biomarker testing processes is an unmet need across academic and community settings. Despite the benefits of EHR-integrated workflows and the desire for streamlined test results, as supported by a recent ASCO Workforce Survey, many health systems do not currently have this capacity, especially when reference labs are external to the ordering institution. Methods: The Association of Cancer Care Centers (ACCC) convened a live summit on October 4, 2023 to understand facilitators and barriers to integrating biomarker orders and results into the EHR. To capture a comprehensive perspective, ACCC convened 37 oncologists, pathologists, nurse navigators, and administrators from diverse settings using a range of EHRs, patient advocacy groups, and EHR/testing companies. Thematic analysis of the summit transcript and a post-summit assessment was employed to identify common experiences among participants. Results: The key benefits of EHR integration include increased efficiency with test ordering/result retrieval, streamlined communication, and improved clinical decision-making. Factors associated with successful EHR integration include strong clinical champions, creation of a precision medicine oversight committee, buy-in from administrative leadership, IT support to build EHR customizations, support from EHR vendors/reference labs, and coordinated project management. Notable barriers include variations in EHR interoperability, utilization of multiple reference labs, lack of consistent nomenclature, resistance to change, allocation of time for post-integration updates/maintenance, lack of internal IT capabilities/resources, and a need to establish different clinical workflows depending on which clinician places the biomarker order. Conclusions: The rapid expansion of biomarker-driven cancer treatments requires oncologists to have timely access to guideline-concordant clinical information to inform complex clinical decision-making. While EHRs can streamline clinical workflows, these results underscore the need to address operational and human factors to facilitate successful EHR-laboratory integration. Strategies to address barriers and build successful EHR integrations across diverse clinical oncology settings are crucial for optimizing how cancer biomarker tests are ordered, reported and used in clinical practice.
Engagement of patients with cancer with digital health platforms: An age-based analysis.
Authors: Paris A. Kosmidis, Panagiotis Baxevanos, Thanos Kosmidis
Journal: Journal of Clinical Oncology (2024)

e13671 Background: Use of digital health platforms has been increasing, including those used by patients directly. In cancer, the nature of the condition, and patient age (higher vs average online users), are among the factors expected to reduce engagement. However, there is limited research to identify and quantify such correlations. We retrospectively analyze them for 4 cancer types. Methods: Patients engage directly with CareAcross platforms to receive information and support based on the data they anonymously provide. These include diagnosis, treatment, optionally year of birth; and side effects (repeatedly, with other inputs in between). On platforms used by breast, lung, prostate and colorectal cancer patients without clinician participation, the number of side effect reports submitted within 6 months of registration was analyzed as a patient engagement metric. This was correlated with patients' ages at registration, also yielding metrics for 10-year age groups around median age. Pearson correlation coefficient (r) was calculated. Results: 1010 breast, lung, prostate and colorectal cancer patients mainly from UK, Germany, France, Spain and Italy have reported birth years. 747 breast cancer patients (28-86yo; median: 53) submitted 2.34 reports on average. Patients above median age tended to submit more reports in general (2.56 vs 2.15 for younger than median); r = 0.055. 10-year age groups analysis: 34-43yo: 2.01 reports (N=109); 44-53yo: 2.19 (287); 54-63yo: 2.61 (236); 64-73yo: 2.45 (94); 74-83yo: 2.92 (12). 135 lung cancer patients (35-84yo; median: 62) submitted 3.27 reports on average. Older patients tended to submit more reports (r = 0.031) except the oldest 10-year age group analyzed (15 patients aged 73-82 reported 1.47 times). 99 prostate cancer patients (48-90yo; median: 68) submitted 2.87 reports on average. Overall correlation was positive: r = 0.039; patients above vs below median age reported 3.47 vs 2.45 times. Beyond these, age groups were not consistent: highest average number of reports was from 69-78yo (average 3.83; N=40) followed by 49-58yo (average 3.77; N=13). 29 colorectal cancer patients (38-81yo; median: 62) submitted 2.34 reports on average, showing negative correlation with age (r = -0.097). Age groups were small and their analysis inconclusive. Conclusions: Although evidence is not clear or uniform, the retrospective analysis of these self-directed digital health platforms shows that older patients appear to engage more than their younger counterparts. More, prospective, research can identify confounders and offer opportunities to improve engagement across all age groups. [Table: see text]
Bridging the gap: Leveraging AI to enhance access to clinical trials for underrepresented and underserved populations with cancer.
Authors: Avital Gaziel, Yelena Lapidot, Tzvia Bader
Journal: Journal of Clinical Oncology (2024)

e13701 Background: National guidelines emphasize the crucial role of clinical trials (CTs) participation for all cancer patients. Despite this, significant barriers hinder access, particularly for underrepresented populations. Patients encounter obstacles such as geographic disparities and limited knowledge of tumor genetics, which impact their access to innovative CTs in metropolitan areas and academic centers. Furthermore, the increasing requirement for specific genetic profiles poses an additional challenge. This study aims to assess the impact of digital health platforms on improving access to CTs, with a specific focus on barriers related to ethnicity, geographic location, and disease knowledge. Methods: Leal (formerly, Trialjectory) is an AI-based platform that matches cancer patients to CTs based on self-reported medical profiles. The study included a diverse cohort of 39,031 users who completed the questionnaire. The patients’ geographic distribution across the US was analyzed, considering proximity to medical/academic centers, willingness to travel, and disease knowledge. Additionally, we examined genetic testing and biomarker knowledge across various clinical and sociodemographic groups. Results: The digital platform significantly improved geographic and ethnic diversity among CT seekers. Compared to industry standards of as low as 3% CT diversity, nearly a quarter of platform users were non-whites. Users outside major metropolitan areas, where CT enrollment is traditionally lower, were also better represented. Markedly, non-whites reported a lower willingness to travel outside their residence for CT participation compared to their white counterparts (p &lt; 0.0001). A gap in disease knowledge was more evident in minority groups (p &lt; 0.0001), with African Americans having a lower probability of knowing their mutation status compared to other ethnic groups. Conclusions: Access to CTs is hindered by barriers such as ethnicity, location, and knowledge. AI-based platforms swiftly identify and resolve these issues, empowering healthcare and support organizations to address individual patient concerns and boost diversity in CTs. AI analysis pinpoints underrepresented populations and regions with limited trial access, paving the way for tailored education and support initiatives to encourage informed decision-making. Strategically establishing trial sites in relevant areas addresses limited travel willingness due to cultural or socioeconomic constraints. AI technologies offer a promising approach to mitigate disparities, improve disease knowledge, ensuring equitable CT access, and enhancing healthcare outcomes for underrepresented populations.
PENTAGON (Predicting Clinical Trials in Gynecologic Oncology): A retrospective study assessing study design factors that affect enrollment in gynecologic cancer trials.
Authors: Humza Pirzadah, Margaret Klein, Yasmeen Magharehabed, Andrew G Chapple, Tara Castellano, Amelia Jernigan
Journal: Journal of Clinical Oncology (2024)

e13647 Background: Obstacles to attaining a diverse clinical trial population contribute to poor representation in medicine, increasing health disparities by limiting generalizability and applicability of findings. Our objective is to evaluate trial enrollment trends according to patient and trial demographics for those who have sought care at our multi-center, mixed-setting Gynecologic Oncology practice. Additionally, we aim to determine specific trial criteria that promote or impede enrollment for patients and physicians. Methods: An IRB-approved, retrospective cohort study evaluating all patients who screened positive for a clinical trial through the Gyn Onc practice’s manual screening process was performed. Screening events were new neoplasms, recurrences or progressions, and changes in treatment. Data was collected from July 2022 to December 2023. Demographic information, trials for which each patient was considered, trials patients enrolled on, and trial characteristics were recorded. Results: After adjustment, there were no significant differences between patient factors (age, marital status, race, ethnicity, and number of screening events) and enrollment status. Of trials available, 9% require biopsy and 94.7% are in person. Most studies are interventional (78.9%) and pharmaceutical trials (73.7%). There was a drastic unadjusted estimated increase in enrollment for Medicaid patients compared to other insurances (55.2% vs 32%, p=.031). There was also an increase in enrollment for cervical cancer patients (18.4%, p=.106) compared to other cancer sites (56.5% vs 32.8%, p=.036). There was a positive relationship between distance traveled to the treatment site and clinical trial enrollment. Patients who lived within 20 miles of the clinical trial site had an enrollment percentage of 30.1% (n=93), compared to 46.55% enrollment for those who lived further (n=58). There was a significant increase in enrollment likelihood without an exclusion criterion of previous history of other cancer (50% vs 33.3%, p=.046). Trials that required no history of chemotherapy had an enrollment of 27.3% (compared to 44.1%). Conclusions: Unlike prior reports, we did not demonstrate differences in trial enrollment by race, socioeconomic or insurance status. This suggests that our objective manual screening process enhances equitable clinical trial offerings. It is possible that this objective manual screening process attenuates provider bias and offers clinical trials to historically marginalized and underrepresented groups.
Comparative analysis of ChatGPT-4.0’s management of six gastrointestinal cancers according to the NCCN guidelines.
Authors: Tamir E. Bresler, Tyler Wilson, Zin Htway, Samuel Slomowitz, Manabu Fujita
Journal: Journal of Clinical Oncology (2024)

e13654 Background: The integration of Natural Language Processing (NLP) into healthcare holds tremendous promise. ChatGPT-4.0 (OpenAI, San Francisco, CA) is a widely recognized large language model that can comprehend and generate text, answer questions, and perform other language-related tasks. However, pitfalls and errors have been described in its clinical application. We explored the ability of ChatGPT-4.0 (ChatGPT) to guide clinical decision-making in 6 gastrointestinal cancers using the National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines as a framework. Methods: We reviewed the NCCN Guidelines for Anal Squamous Cell Carcinoma (AN), Small Bowel Adenocarcinoma (SB), Ampullary Adenocarcinoma (AA), Biliary Tract Cancers (BT), Pancreatic Adenocarcinoma (PN), and Gastric Cancer (GA). Up to 2 clinical questions were designed for each decision-making page. Questions were categorized by type ( Wo rkup, Treatment, Surveillance, Diagnostics, or References). ChatGPT was queried in a reproducible fashion. To account for variable prompt engineering of our non-validated assessment tool, up to 3 follow-up questions were allowed. Responses were rated on a Likert scale: 5) Correct; 4) Correct, with missing information requiring clarification; 3) Correct, but unable to complete answer; 2) Partially incorrect; 1) Absolutely incorrect. Subgroup analysis was conducted on Correctness (defined as scores 1-2 vs 3-5) and Accuracy (scores 1-3 vs 4-5). Variance between ChatGPT responses to each cancer was analyzed. Descriptive statistics were used, and significance was tested with binary logistic regression. Results: A total of 270 questions were generated (range per cancer 32-68). The score frequency distribution was: 5) 45.2%; 4) 19.3%; 3) 13.3%; 2) 13.7%; and 1) 8.5%. On subgroup analysis, Correctness was seen in 210 (77.8%) of questions, and Accuracy with 174 (64.4%). The difference in Correctness scores between cancers was not statistically significant, and there was no statistically significant difference in scores by question type. There was a statistically significant difference in the Accuracy of ChatGPT between cancers (Table). Conclusions: ChatGPT was significantly more likely to provide accurate responses to questions regarding GA and PN versus AN or SB. It demonstrates a limited capacity to assist with complex clinical decision-making in 6 gastrointestinal cancers. However, the Accuracy level is below the acceptable threshold for implementation into clinical use. Further analysis of the expanding capabilities of ChatGPT and other NLP-based tools is warranted in this rapidly evolving domain. Future studies would benefit from a validating grading instrument. [Table: see text]