## 4 Results
Turning to overall inter-model concordance, Table 4 (High-Level Inter-Model Reliability Summary, Debug Edition) lists [PERSON] kappa (\(\kappa\)) and the mean percent agreement for six fields: for **brain_stimulation_used**, the near-perfect \(\kappa\) (\(\sim\)0.94) and \(\sim\)99% mean agreement underscore how readily the models identify transcranial stimulation; for **primary_type**, there is a substantial \(\kappa\) (\(\sim\)0.71) and \(>\)93% mean agreement, reflecting minor confusion around acronyms (e.g., \"tCS,\" \"TPS\"); for **is_noninvasive**, despite the \(\sim\)98% raw agreement, \(\kappa\) (\(--\)0.003) hovers near zero due to class imbalance--all but a few interventions were classified as noninvasive; for **primary_target**, there is a moderate \(\kappa\) (\(\sim\)0.53) with \(\sim\)75% agreement, driven by the variety of anatomical terms and the partial mention of target sites; for **parameters_intensity** and **parameters_duration**, both hovered around \(\kappa\approx\) 0.50, with 75-78% agreement, reflecting variability in numeric references (\"2 mA,\" \"very weak current,\" \"20 to 60 min,\" etc.). In this analysis, we evaluated categorical and numeric fields extracted by five LLMs. We added two additional fields beyond the original set confidence_level (categorical: High, Medium, or Low) and frequency (numeric, e.g., 140 from \"140 Hz\" or 2 from \"2 Hz\"). Table 5 (Inter-model reliability and agreement statistics) provides a more detailed breakdown, splitting the reliability metrics into (A) Fleiss' kappa for categorical fields, with bootstrap confidence intervals, and (B) ICC (intraclass correlation coefficient) for numeric data. \"brain_stimulation_used\" achieves an Almost Perfect \(\kappa\) (0.90); \"primary_type\" (\(\kappa\approx\) 0.71) remains Substantial; and \"is_noninvasive\" (\(\kappa\approx\) 0.59) stands at Moderate. For numeric parameters (intensity, duration, frequency), ICC values range from 0.95 to 1.0 (Excellent), indicating strong consistency when the text clearly presents numerical values. At the same time, an ICC of 1.0 may reflect limited variability in how frequency is reported--many trials either did not provide frequency or gave a simple measure (e.g., \"2 Hz\"). Table 6 (Final \"Consensus\" Table) illustrates how each trial's fields look after majority voting for categorical variables and mean (or most common) values for numeric variables. Trials like NCT03814304 show a clear consensus (\"tDCS,\" 2.0 mA, 20.0 min), whereas others display blank or partial columns when protocols lack explicit numeric details. \begin{table}
\begin{tabular}{l c c c c} \hline \hline \multicolumn{1}{c}{**Field**} & \multicolumn{1}{c}{**FleissKappa**} & \multicolumn{1}{c}{**Mean \% Agreement**} & \multicolumn{1}{c}{**\# Trials**} & \multicolumn{1}{c}{**Categories**} \\ \hline brain\_stimulation\_used & 0.941 & 0.993 & 83 & yes, no \\  & & & & ctdcs, ddtms, dtds, hdt-tdcs, missing, non-invasive \\  & & & brain stimulation, tacs, tcs, tdcs and tacs, tdcs, \\ primary\_type & 0.709 & 0.938 & 78 & tacs, tps, tps (transcranial pulse stimulation), tps \\  & & & & (transcranial pulse stimulation, also known as \\  & & & low-intensity extracorporeal shock wave \\  & & & therapy (li-eswt)) \\  & & & true, false \\ \hline \hline \end{tabular}
\end{table} 
Table 4: This table compares how consistently five large language models (LLMs) label key fields in clinical trial text. We report field the attribute being extracted (e.g., whether or not brain stimulation is used, the primary stimulation type, etc.). Fleiss’ kappa: a statistical measure of how well the models agree beyond chance. Values typically range from \(-\)1.0 to 1.0. Mean % Agreement: The average fraction of model outputs that match the majority label in each trial. # Trials: The number of trials that had non-missing data for that field. Categories: The distinct labels (e.g., ’yes’, ’no’, ’tDCS’) that appeared in the model outputs for that field.