Clinical trial NCT06561217: Assessing the Performance of Artificial Intelligence (AI)-Augmented Electronic Health Record (EHR) Data Abstraction for Clinical Trial Patient Screening
Official title: Assessing the Performance of Artificial Intelligence (AI)-Augmented Electronic Health Record (EHR) Data Abstraction for Clinical Trial Patient Screening
Study type: OBSERVATIONAL
Overall status: COMPLETED
Conditions: Cancer
Lead sponsor: University of Pennsylvania
Sponsor class: OTHER
Collaborators: name: Mendel AI
First posted: 2024-08-19
Enrollment: 355
Sex: ALL
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Observational model: OTHER
Time perspective: RETROSPECTIVE
Sampling method: PROBABILITY_SAMPLE
Study population: De-identified patient charts from community oncology practices, with a diagnosis of non-small cell lung cancer (NSCLC) or colorectal cancer (CRC).

Brief summary: Identifying eligible patients is a key process in the clinical trial enterprise. Currently, this process relies on time-intensive manual chart review, creating a rate-limiting step for trial participation. The integration of AI technology into the trial screening process has potential to improve participation rates. This study aims to assess the performance (accuracy, efficiency) of AI-augmented patient identification and inform optimal integration into clinical research screening processes.

Detailed description: The objective of this study is to assess and compare the accuracy and efficiency of three different approaches to abstracting clinical data used to identify oncology patients who meet the inclusion criteria for participation in clinical trials. The three approaches under evaluation include: (1) an autonomous AI algorithm (Mendel AI; developed by artificial intelligence startup company Mendel) which analyzes patient medical records to extract relevant clinical facts ("AI-alone"); (2) a human researcher who manually reviews patient charts as per the current norm/practice ("Human-alone"); and (3) a human researcher utilizing AI augmentation ("Human+AI"), where Mendel AI serves as a supportive tool in the decision-making process by providing the researcher a list of elements abstracted by the AI algorithm and a rank-order list of patients most likely to meet inclusion criteria for a trial.

The study primarily aims to compare (1) the chart-level accuracy of the Human+AI collaboration relative to Human-alone given the relevance of this comparison for real-world clinical workflows, defined by the percentage of pre-identified chart elements classified correctly compared against a predetermined "gold standard"; and (2) the efficiency of the Human+AI vs. Human-alone arms, defined by the time per chart review in minutes, measured for each chart.

Our hypotheses are (1) the Human+AI arm will be non-inferior in accuracy when compared to the Human-alone arm, in relation to a predetermined "gold standard", and (2) that a Human+AI arm will be superior in efficiency of abstraction when compared to Human-alone screening.

The identification of eligible patients for clinical trials is a critical component of clinical research, as it directly impacts patient recruitment, study enrollment, and the generalizability of research findings. Currently, the process of identifying eligible patients often relies on manual chart review by clinical research staff, which can be time-consuming, labor-intensive, and prone to human error. Consequently, eligible patients may be overlooked, and opportunities for trial participation may be missed. The integration of AI technology into the patient identification process has the potential to enhance the accuracy and efficiency of this critical task, leading to improved clinical trial recruitment and outcomes.

This study holds important implications for the field of clinical research by evaluating the effectiveness of AI-augmented patient identification compared to traditional manual methods and autonomous AI algorithms. By examining the strengths and limitations of each approach, the study will provide valuable insights into the optimal integration of AI technology in clinical research processes. Furthermore, the results of this study have the potential to benefit patients by improving their access to clinical trials and increasing awareness of available treatment options. For clinical research institutions, enhancing the efficiency of patient identification can lead to more effective use of research resources and the potential for accelerated clinical trial timelines. Ultimately, the findings of this study may contribute to advancements in clinical research practices, promoting more equitable access to trials and facilitating the development of innovative treatments for patients with cancer.

Eligibility criteria: Inclusion Criteria:

* Diagnosis of colorectal or non-small cell lung cancer.
* A minimum of 5 patient documents in the Mendel database.
* Most recent document was within 5 years from the time of data extraction.

Exclusion Criteria:

* None.

Primary outcomes:
measure: Mean proportion of criterion correctly abstracted by AI-augmented human reviewers
timeFrame: 1 year
measure: Mean proportion of criteria correctly abstracted by human-alone reviewers
timeFrame: 1 year
measure: Non-inferiority margin
timeFrame: 1 year
Secondary outcomes:
Other outcomes:
Locations:
city: Philadelphia
country: United States
facility: University of Pennsylvania
state: Pennsylvania
Study officials:
References: