EC - RAFT : Automated Generation of Clinical Trial Eligibility Criteria through Retrieval-Augmented Fine-Tuning (2025)

Abstract: Eligibility criteria (EC) are critical components of clinical trial design, defining the parameters for participant inclusion and exclusion. However, designing EC remains a complex, expertise-intensive process. Traditional approaches to EC generation may fail to produce comprehensive, contextually appropriate criteria. To address these challenges, we introduce EC-RAFT, a method that utilizes Retrieval-Augmented Fine-Tuning (RAFT) to generate structured and cohesive EC directly from clinical trial titles and descriptions. EC-RAFT integrates contextual retrieval, synthesized intermediate reasoning, and fine-tuned language models to produce comprehensive EC sets. To enhance clinical alignment evaluation with referenced criteria, we also propose an LLM-guided evaluation pipeline. Our results demonstrate that our solution, which uses Llama-3.1-8B-Instruct as a base model, achieves a BERTScore of 86.23 and an EC-matched LLM-as-a-Judge score of 1.66 out of 3, outperforming zero-shot Llama-3.1 and Gemini-1.5 by 0.41 and 0.11 points, respectively. On top of that, EC-RAFT also outperforms other fine-tuned versions of Llama-3.1. EC-RAFT was trained in a low-cost setup and, therefore, can be used as a practical solution for EC generation while ensuring quality and relevance in clinical trial design. We release our code on GitHub at https://github.com/biodatlab/ec-raft/

## introduction eligibility criteria (ec) are essential components of clinical trial design, specifying the parameters for participant inclusion and exclusion @xcite . these criteria ensure trials are scientifically valid, ethically sound, and capable of meeting their objectives. however, designing ec remains a laborintensive and expertise-driven process @xcite . tools that can suggest or generate relevant ec have the potential to significantly facilitate researchers' work in trial design @xcite . generating these criteria is inherently complex because consistency and clinical validity are needed throughout the criteria set. despite advances in using large language models (llms) for summarization or specialized tasks in the biomedical domain, several barriers remain to creating a fully automated, contextually accurate system that can generate comprehensive sets of ec directly from trial descriptions. recent developments in instruction fine-tuning for llms have shown promise in generating logical reasoning outputs through techniques like chain-of-thought prompting and rationale generation @xcite . retrieval-augmented generation (rag) @xcite has also emerged as an effective mechanism for grounding model outputs with external domain knowledge, thereby improving factual correctness @xcite . retrieval-augmented fine-tuning (raft) @xcite extends rag by incorporating instruction fine-tuning to improve both domain adaptation and retrieval robustness. these developments allow the development of an end-to-end system to generate a complete set of ec while preserving essential clinical context and domain relevance. to address these gaps, we propose ec-raft, a novel approach that leverages retrieval-augmented fine-tuning (raft) @xcite for automated ec generation. ec-raft aims to produce complete ec sets directly from trial titles and descriptions without requiring userprompted ec categories or a recommendation system. our key features include: our training setup was also optimized for cost efficiency using the parameter-efficient fine-tuning technique (peft) @xcite . specifically, training our best model required 380 gpu hours on nvidia a100 costing approximately 452.20 usd while achieving superior performance compared to the baseline. ## methods in this section, we introduce our approach, which leverages clinical trial data from clinicaltrials.gov and integrates state-of-the-art techniques in embedding, retrieval, and fine-tuning to automate the generation of ec (figure 1 ). we then describe the experiments designed to evaluate our system. ## evaluation due to the challenging nature of semi-structured eligibility criteria, we employ three metrics to compare our predicted output (ŷ) with the ground truth (y) to measure: 1) bertscore for overall semantic similarity, and 2) llm-guided evaluation which only evaluate the matched pair, pair-bertscore, identified by llms and utilize llm-as-a-judge to judge capability to assess clinical relevance for each matched pair. ## conclusion in this work, we introduced ec-raft, a framework that leverages retrieval-augmented fine-tuning and synthesized intermediate reasoning to automate the generation of clinical trial eligibility cri-teria. ec-raft generates structured, robust, and clinically relevant eligibility criteria directly from trial descriptions. our experiments on a largescale clinicaltrials.gov dataset demonstrate that ec-raft outperforms zero-shot baselines without retrieval, despite being much smaller in model size, and outperforms fine-tuned versions of llama, achieving higher bertscores and clinical alignment, as evidenced by bertscore and llmguided evaluations. notably, the llm-guided evaluations show high agreement with a licensed physician, further validating the clinical relevance of the generated criteria. while challenges remain, ec-raft represents a significant step towards automating the complex process of clinical trial design. future work will refine the intermediate steps generation process and scale up model size and compute for better performance.