## 2 Methodology
### Multi-Criteria Decision Analysis to Select Top Rank  
To determine the top-performing LLM for extracting patient characteristics from free-text eligibility criteria descriptions, we employ Multi-Criteria Decision Analysis (MCDA) using the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) [33].  
TOPSIS is a well-established Multi-Criteria Decision Analysis (MCDA) method that systematically evaluates alternatives based on multiple criteria [46; 47]. In our study, the alternatives are different large language models (LLMs) being evaluated, and the criteria include various evaluation metrics such as human validation scores, F1-scores, precision, recall, semantic textual similarity, factual consistency, coherence, consistency, fluency, and relevance. TOPSIS was chosen because it efficiently determines the relative performance of alternatives by measuring their closeness to an ideal solution. It provides a clear, objective, and mathematically grounded ranking of LLMs.  
The TOPSIS methodology involves the following steps [48; 49]:  
1. **Constructing a decision matrix**: The performance scores of each LLM across all evaluation metrics are organized into a decision matrix. This matrix serves as the foundation for the comparison of alternatives.  
2. **Normalizing the decision matrix**: The scores are normalized to ensure comparability across criteria with different scales. This step avoids the dominance of any particular metric due to its range or magnitude.
3. **Assigning weights to the criteria**: Each evaluation metric is assigned a weight based on its relative importance in assessing LLM performance. This ensures that critical metrics, such as human validation and F1-score, have greater influence.
4. **Identifying the Positive Ideal Solution (PIS) and Negative Ideal Solution (NIS)**: The PIS represents the best possible scores across all criteria, while the NIS represents the worst scores. These benchmarks serve as reference points for evaluating the alternatives.
5. **Calculating the Euclidean distance**: The distance of each LLM from the PIS (\(D_{i}^{+}\)) and NIS (\(D_{i}^{-}\)) is calculated using the Euclidean distance formula. This provides a robust measure of each model's closeness to the ideal and worst cases.
6. **Computing the Relative Closeness to the Ideal Solution (RCIS)**: The RCIS for each LLM is computed using the formula \[RCIS_{i}=\frac{D_{i}^{-}}{D_{i}^{+}+D_{i}^{-}}\] (8) where \(D_{i}^{+}\) is the distance of the \(i\)th LLM from the PIS, and \(D_{i}^{-}\) is the distance from the NIS. Higher RCIS values indicate better-performing LLMs.
7. **Ranking the LLMs**: The LLMs are ranked based on their RCIS values, with the highest value indicating the best-performing model.  
By applying the TOPSIS methodology, we achieve a comprehensive evaluation of the LLMs, considering multiple criteria and their relative importance [47]. TOPSIS ensures a transparent and objective ranking process, enabling the identification of the top-performing LLM that excels across various dimensions. This approach provides a reliable solution for extracting patient characteristics from free-text eligibility criteria descriptions [50].