bluuebunny/crossref_metadata_2025_split huggingface dataset:
Comparative assessment of manual chart review and patient-level adverse drug event identification using artificial intelligence in evaluating immunotherapy-related adverse events (irAEs).
Authors: Zitu Md Muntasir , Li Lang , Elsaid Mohamed I. , Gatti-Mays Margaret Elena , Manne Ashish , Shendre Aditi 
Published: June 2023

e13583 Background: IrAEs monitoring in patients undergoing Immune checkpoint inhibitor (ICI) treatments relies on analyzing clinical notes and structure data from electronic health records. As IrAEs are not yet fully captured in the ICD claims, they could be under reported. On the other hand, IrAEs are usually recorded in the clinical notes. A natural language processing (NLP) tool is highly desirable to extract them from clinical notes. In this study, we aim to compare the performance of our machine learning (ML) NLP model in IrAE extraction against 1) manual chart reviews and 2) ICD claims data. Methods: The study validation set included 194 newly diagnosed stage I-VI gastrointestinal cancer patients treated with ICIs at the Ohio State University Comprehensive Cancer Center. An irAE was defined as the occurrence of gastrointestinal, pulmonary, dermatological, or endocrine adverse events post-ICI infusion. Our NLP ML model was trained on 1230 clinical notes from 20 positive (who had ADEs) and 10 negative (who did not have ADEs) patients who received ICIs. ICD irAE method is based on ICD-9 and ICD-10 codes, which identifies irAEs post-ICI infusions. All manual reviews of ICI induced irAEs were done by a medical oncologist specializing in gastrointestinal cancers and included patient charts, labs, and imaging post-ICI infusions. Results: In this 194 patient cohort, the median age was 61 years, 68% males, 89% White non-Hispanics, and 29% had colorectal cancer. The study included 8,500 medical notes with a median 29 notes per patient and 1,283 words per note. An estimated 4,726 (55.6%) were progress notes, 1,178 (13.9%) patient Instructions, and 430 (5.1%) consults. The incidences of irAEs were 13.9%, 25.8%, and 27.3%, respectively, using manual reviews, the ML model, and ICD claims, respectively. Compared to the manual reviews, the ML model had a 78% sensitivity, 83% specificity, 0.17 false positive rate (FPR), and 0.80 (95% CI, 0.72 to 0.89) area under the curve (AUC). In contrast, ICD claims had a poor agreement compared to manual reviews. 52% sensitivity, 77% specificity, 0.23 FPR, and AUC 0.64 (95% CI, 0.54 to 0.74). Conclusions: Using clinical notes, ML can identify patients with irAEs with a great potential to infer causality in real time clinical practice. Our irAE manual review analysis further indicates that ML model performance can be further improved if including lab and imaging data.
Deriving breast cancer chemotherapy patterns from real-world data.
Authors: London Jack W. , ORourke Julia , Warnick Jeff , Doole John , De Keyser Luc , Drebert Zuzanna , Wan Olivia , Thompson Courtney , Palchuk Matvey 
Published: June 2023

e13586 Background: Real World Data (RWD) collected during routine medical practice can help in clinical trial design, planning and execution. In addition, it can provide a compelling picture of safety of medical products, and account for all potential adverse events that can be encountered in routine clinical practice. However, raw drug information about chemotherapy in cancer care is not readily understandable in the format it is collected. Analytical techniques need to be applied to extract regimen information, which includes drugs, dosage, number of cycles and cycle length. Methods: A retrospective study was performed using data on 7,798 breast cancer patients from the TriNetX Network, a federated network of de-identified, HIPAA-compliant, health data from 21 healthcare organizations across North America as of May 2022. We investigated a method built on rule-based algorithm and clustering analysis to extract regimens and their patterns of administration and align them into lines of treatment (LOT). To derive drug administration patterns, we clustered administration time periods using three features: total number of drug administrations, median number of days between drug administrations, and standard deviation of the days between drug administrations. Results: The patterns of administration that correspond to the two most common regimens for Erb-B2 receptor tyrosine kinase 2 negative group (ERBB2-) stages 1, 2 and 3 are shown. We looked at patients with Hormone Receptor (HR+) positive and Triple Negative (TN) breast cancer. Results of our analysis were in a close agreement with NCCN Guidelines. However, regimen administration patterns varied. This information can be useful to characterize patients based on the adherence to the expected chemotherapy regimens. It can also provide meaningful insight into burden of illness, such that patients who have higher variability of the drug administration might be faced with clinical problems such as drug tolerability or side effects. The variability might also signify access to care challenges. Conclusions: Understanding regimen LOT and administration patterns is central to research based on RWD, but it can also be useful for clinical trial design, execution, as well as site selection and patient recruitment for clinical trials. [Table: see text]
Digital Trial Solutions eScreening: Software to rank patients by their predicted clinical trial eligibility using real-world data and machine learning.
Authors: Bhandari Sudip , Levonian Zachary , Annand Robert , Baxter Jo-Zahn , Cain Jericho , Flach Jen , Joseph-Thomas Jiby , Mueller Judith 
Published: June 2023

e13587 Background: The recruitment process for clinical trials is time-consuming and labor-intensive. Electronic screening tools aim to facilitate patient recruitment by focusing the recruiters’ efforts on patients most likely to be eligible. These tools are required to produce useful estimates of patient eligibility despite large amounts of missingness in Real World Data (RWD). Digital Trial Solutions (DTS) eScreening—unlike other proposed tools that use only top-level Inclusion/Exclusion (I/E) criteria—queries for all computable criteria and imputes missing data using Machine Learning (ML). Our ML models use the ConcertAI data from DTS clinical sites. Methods: DTS eScreening provides an eligibility score for each patient in a clinical trial cohort based on how many I/E criteria are fulfilled, weighted by criteria relevance as assessed by clinical experts. If information is missing in the patient record, DTS eScreening predicts the probability of passing the criterion using a binary classifier trained for each criterion. We evaluated classifier performance on a held-out test set. We evaluated the utility of DTS eScreening by collecting quantitative and qualitative data from users of DTS eScreening. Results: All evaluations were done on a multi-site clinical trial with 49 digitized criteria. The cohort was 10,156 patients with the appropriate cancer indication. Model performance: twenty-three criteria had complete data and so no ML models were built. To avoid biased model estimates, we avoided modeling ten criteria with greater than 99% missingness or 1000:1 imbalance. Seventeen LightGBM classifiers were built on the full cohort for criteria such as ECOG, stage, labs, and vitals. The model performance showed an average precision of >99% for most models, while the lowest precision was 83%. DTS eScreening utility: using our tool, clinical screeners reviewed 114 patients with the highest eligibility scores of the 10,156-patient cohort. Of the 114 patients, 38 were determined to meet all digitized eligibility criteria. The precision on reviewed patients was 33%, higher than other reports in the literature. Feedback from semi-structured interviews with 3 users indicated the tool was useful for reducing review time. A screener said: “It makes screening easier for me because the system already filters the most eligible patients at the top.”. Conclusions: DTS eScreening tool demonstrates the ability to conduct patient-trial matching using computable I/E criteria while addressing missingness in RWD. Positive initial feedback from users highlights improvements in the efficiency of patient recruitment in clinical trials. We believe our solution is scalable and can be applied beyond oncology. Future improvements include relaxing restrictions for modeling at low counts to include additional I/E criteria and incorporating unstructured data.
Machine learning prediction of progression events in oncology RECIST 1.1 clinical trials.
Authors: Bowman Erin , Banuprakash Shyam , Nguyen Kim , Marini Matthew 
Published: June 2023

e13589 Background: Oncology clinical trials often have an imaging based surrogate endpoint such as Progression Free Survival (PFS). Using PFS as a surrogate for survival has the advantage of achieving trial completion more quickly which reduces cost and risk to patients. These trials are frequently structured to require a certain number of events to achieve statistical significance. The use of machine learning models which can make predictions about progression events from data obtained after the initiation of therapy is a novel idea in this field. It is well understood that many factors play a role in whether a patient will ultimately progress on a certain therapy. While certain statistical predictions can be made given known factors such as tumor genetics and patient history, these are made prior to the patient initiating therapy and do not consider on-therapy response. Machine learning models can be used during trial conduct, incorporate patient response, and better predict progression outcomes. Methods: Machine learning models were trained on over 115,000 RECIST 1.1 datapoints. The iterative training process included inputs from the medical science team which included a lead radiologist with expertise in diagnostic radiology and molecular imaging; and a lead medical informaticist responsible for data quality. Models were trained using 56 raw and derived features. Examples of derived features include number of lesions, time between evaluations, and change in responses over time. The features were trained using Random Forest, AdaBoost, Gradient Boost, XGBoost model types and compared to determine which one performed best. These model types identified the top predictors and created tree-based visuals, which provided insights into the model’s decision-making process. Confusion matrices were reviewed by the team to iteratively address false positive and false negative records. Results: Initial results of the machine learning model demonstrated a 75.1% specificity and a 66.6% accuracy for prediction of progression events. With continuous improvement efforts, as of writing, the model demonstrated 94.4% specificity, and 82.2% accuracy. This process identified the top predictors of progression to be the primary RECIST 1.1 features. Given our scientific and AI expertise results are expected to improve over time. Conclusions: Outcomes have shown the ability to predict RECIST 1.1 progression. Such a model could successfully predict when the required number of progression events are present within a population, at which point enrollment could be ceased and decrease overall trial length. Given enough data and rigorous testing, this technology could predict complete response vs partial response, adverse events, and time to events. Data suggests that AI machine learning within Clinical Trials could have a significant impact and should be considered a priority for continued evaluation given the possible benefits to trials.
Different efficacy of pembrolizumab in overall survival between Asians and non-Asians with advanced cancers: A systematic review and meta-regression analysis.
Authors: Peng Shang-Hsuan , Lin Ching-Hung , Chen I-Chun , Shen Ying-Chun , Chang Dwan-Ying , Chen Tom Wei-Wu , Huang Shu-Min , Hu Fu-Chang , Lu Yen-Shen 
Published: June 2023

e13597 Background: Immune checkpoint inhibitors have revolutionized the treatment landscape of malignancy. However, disproportionate enrollment among different races/ethnicities in global trials placed the generalizability in doubt. This meta-analysis aimed to estimate the relative efficacy of pembrolizumab between Asians and non-Asians with advanced cancers. Methods: A systematic review was conducted in PubMed, Cochrane Library, Embase, and Web of Science. The meta-analysis included phase 3 randomized controlled trials investigating pembrolizumab in advanced cancers with available subgroup analysis for Asians and non-Asians. The experimental strategies involved monotherapy versus standard-of-care (SOC) or best supportive care, an add-on to SOC, and combined with multikinase inhibitor versus standard chemotherapy, regardless of the setting. The primary and secondary effect measures were the mean difference (MD) in the natural logarithm (i.e., log) of the hazard ratio (HR) for overall survival (OS) and progression-free survival (PFS) between the two subgroups [MD=log(HRAsians)−log(HRNon-Asians)=log(HRAsians/HRNon-Asians)]. We used the random-effects meta-analysis to calculate the pooled ratio of HRs (i.e., exp(MD)) and implemented a meta-regression analysis to identify significant covariates. The heterogeneity among the selected studies was evaluated using the Chi-squared Q test and the I2 statistic. Results: A total of 17 and 11 trials were selected in the meta-analysis for OS and PFS, respectively. These trials reported available subgroup results on Asians and non-Asians (patient numbers 2,732 [25.49%] and 7,000 [65.32%] for OS; 1,438 [22.50%] and 4,129 [64.61%] for PFS). The pooled ratio of HRs for OS was 0.87 (95% confidence interval [CI], 0.76−0.99; p = 0.0391; Chi-squared Q test Χ2 = 20.84, p = 0.29; I2 = 23.9%), favoring Asians, and no significant difference was found in PFS (the pooled ratio of HRs 0.93; 95% CI, 0.82−1.07; p = 0.2391; Chi-squared Q test Χ2 = 7.37, p = 0.77; I2 ≈ 0%). Both linear meta-regression analyses revealed an open-label design as an important covariate accounting for the observed difference, favoring non-Asians (estimated ratios of HR, 1.6318 for OS [95% CI, 1.1442-2.3273]; 1.3429 for PFS [95% CI, 1.0007−1.8020]). The sensitivity analysis showed that the percentage of subsequent treatment or post-progression programmed cell death 1 (PD-1)/ligand 1 (PD-L1) inhibitors had no significant impact. Conclusions: Asians receiving pembrolizumab for advanced cancers may have a superior OS advantage than non-Asians. Although the results warranted further exploration, this meta-analysis provided insight into clinical research design.
Accrual of adolescents and young adults (AYA) into cancer clinical trials in Canada.
Authors: Jafri Mariam , Willman Corey , Urton Alison May , Henning Jan-Willem , Abbott Lesleigh S. , Davison Kelly , Akra Mohamed A. , Seymour Lesley , Dancey Janet Ellen , Hay Annette E. 
Published: June 2023

e13598 Background: Historically, accrual rates of AYA (aged 16-39) to cancer clinical trials have been low. This is suggested to contribute to lesser improvements in their outcomes compared with older and younger populations. We sought to describe the impact of collaborative initiatives launched by the Canadian Cancer Trials Group (CCTG), NCTN and C17 to address this. Methods: All CCTG trials open to accrual from 01/01/2004 to 31/12/2022 were evaluated. Trials with eligibility criteria limited to age > 39, or for which CCTG did not have access to individual patient data on age, were excluded. Accrual rates were assessed by age (<30, 30-39, >40 yrs), cancer type, and compared with 2022 Canadian Cancer Society (CCS) prevalence data. Results: 379 studies were open to accrual. 21 trials were excluded as they had no patients aged 16-39 within their inclusion criteria. 2 trials were excluded as they were either COVID related or a trial sub-study. 230 trials were excluded owing to patient age data being unavailable to CCTG. 126 CCTG and NCTN led trials were included. They involved the following disease sites: brain (3), breast (13), GI (12), GU (11), gynecology (6), head & neck (3), hematology (10), lung (11), melanoma (4), sarcoma (2) novel investigational agents (45) and symptom control (6) – the latter two spanned multiple tumor types. Patients aged 16-18 were excluded from many trials limiting accrual opportunities in this subgroup. AYA patients were recruited into trials at a higher level than expected by prevalence data except for brain, lung and head & neck patients. This may be due to younger, fitter patients being recruited onto trials. GU trials, hematology and sarcoma trials recruited better than expected as there were trials geared to diseases common in younger individuals (e.g. germ cell cancers) and incorporating intensive therapy e.g. stem cell transplantation in lymphoma. 3.5% of patients accrued on trials were <40 in 2004-2013 c.f. 8.7% of patients 2014-2022. Conclusions: Limited data is available on AYA aged 16-18 due to trial entry criteria. Currently, most CCTG trials include patients aged ≥14. Trial accrual in the AYA population was higher than predicted by CCS data indicating that clinical trials are acceptable to AYA patients. This supports ongoing efforts to improve trial access to AYA. [Table: see text]
Phase I solid tumor trials (PHIstt) in the era of precision oncology (PO): A systematic survey.
Authors: Stockton Shannon , Ayers G. Dan , Lee Cody L. , Laferriere Heather E. , Berlin Jordan , Das Satya (Nanu) 
Published: June 2023

e13603 Background: In the era of PO, systemic therapies for patients (pts) with solid tumors have fundamentally changed from chemotherapy (CT) to targeted therapy (TT) and immunotherapy (IO). No study to the best of our knowledge has described PHIstt in this era. In this systematic survey, we describe the evolving features of PHIstt which began enrollment between 2010-2020, focusing on type of dose escalation scheme (DES) utilized, use of expansion cohorts (ECs) and nature of adverse events (AEs). Methods: The primary aim was to estimate the proportion of PHIstt which used rule-based DES. A literature search was performed by a biomedical librarian to identify PHIstt in adult patients published between January 1, 2000 – December 31, 2020 from a selection of 12 journals that publish PHIstt. However, we included studies enrolling between 2010-2020 to better capture agents in the PO era. We included trials with both single and multiple agents, and single and multiple modalities if the systemic therapy tested was being escalated. We excluded studies which only included intra-tumoral therapy, radiation therapy, or supportive care. Two reviewers assessed studies for inclusion and performed data abstraction; a 3rd reviewer established concordance. Results: Of 10,744 studies retrieved, 10,195 were non-topical or began enrollment prior to 2010; 437 studies in pts aged ≥ 18 years with advanced solid tumors were included. A median of 31 pts were enrolled per study (IQR 19-50), with median age 60 years in escalation cohorts. In studies reporting pt sex, 45.7% were female, and 53.9% were male. The most common classes of therapeutics were TT (47.6%), IO (22%), and CT (6.9%). Only 40.1% of studies reported ECs in the primary paper, and 46.6% of ECs used genomic selection. Rule-based designs (which assign patients to dose levels based upon prespecified rules of actual AE observations) were used in 89% of PHIstt; a 3+3 DES was used in 80.5% of these studies. With regards to grade 3/4 AE reporting, 56.6% of studies reported treatment-related AEs, 21.3% reported treatment-emergent AEs and 10.7% did not specify how they were reported. The most common class of grade 3/4 AEs was hematologic (37.1%). The most common dose-limiting toxicity classes were other (64.6%) and gastrointestinal (16.9%). Of all the drugs tested in PHIstt, 37.5% were tested subsequently in phase II while 10.3% garnered licensure. Conclusions: There is room for improvement in the trial design of PHIstt which may lead to more regulatory success. Some areas we identified include the more regular adoption of genomically selected ECs, more consistent reporting of AEs and diversification beyond rule-based designs to better assess safety/efficacy profiles of non-CT agents. Dose-Escalation Designs. [Table: see text]
Primary care provider perceptions of patient participation in a multi-cancer early detection trial.
Authors: Myers Ronald , Hallman Mie , Davis Kaitlyn , DiCarlo Melissa , Daskalakis Constantine , Shimada Ayako , Shenoda Botros , Danoff Robert , Dong Karen , Chambers Christopher 
Published: June 2023

e13607 Background: Randomized clinical trials of multi-cancer early detection (MCED) tests are being planned, and primary care providers (PCPs) are likely to encounter patients who are considering participation. Little information has been reported about provider support for patient participation in such trials. Methods: We surveyed PCPs from 4 practices in a large health system to learn about their views on patient participation in a hypothetical MCED test trial. The PCPs received a link to view a patient-oriented infographic that described a trial in which participants would undergo serial blood draws for MCED testing and would be randomly assigned either to standard care or to receive MCED test results and follow-up, as needed. After viewing the infographic, respondents were asked to complete a brief survey that included statements related to factors that would affect their receptivity to patient participation in the trial, using a 5-point Likert type response scale that ranged from 1 = Strongly Disagree to 5 = Strongly Agree. The survey also included a single item that asked respondents if they would support patient participation in the trial and provided an opportunity to report the main reason for their answer. Finally, the survey contained a background characteristics section. We computed frequency distributions of survey responses. Results: Twenty-seven of 37 (73%) invited primary care providers completed the survey. About half of the respondents were women and two thirds were white; 81% were physicians while 19% were nurse practitioners, 54% were in practice for less than 20 years, all were board certified, and 74% had a hospital affiliation. Receptivity scores regarding a future MCED trial were high, with a mean of 3.8 and standard deviation of 0.6. Overall, 25 providers (93%) reported that they would support patient participation in an MCED test trial. The most commonly reported reasons for supporting patient participation were the belief that early cancer detection is important and the belief that research to develop new cancer screening tests is needed. The most commonly cited reasons for not supporting participation were concern about managing patients diagnosed with cancer in the trial and worry that trial participants would require too many practice resources. Conclusions: The overwhelming majority of respondents supported patient participation in an MCED test trial, but many were concerned about trial impact on their capacity to meet patient needs. Further research is needed in a larger study to validate these results and learn more about factors likely to influence PCP support for patient participation in MCED test trials and actual participation.