## 3 Results
### Evaluation of LLMs on USMLE Benchmarks
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model Configuration** & **Step 1 (\%)** & **Step 2 (\%)** & **Step 3 (\%)** & **Overall (\%)** \\ \hline
70B\_noquant\_00 (Base) & 47.06 & 55.00 & 54.01 & 52.13 \\ \hline
70B\_finetuned\_noquant\_dosample & 67.23 & 76.67 & 75.18 & 73.14 \\ \hline
70B\_noquant\_01 (Base) & 46.22 & 56.67 & 55.47 & 52.93 \\ \hline
70B\_finetuned\_noquant\_01 & 66.39 & 80.83 & 75.18 & 74.20 \\ \hline
70B\_noquant\_04 (Base) & 46.22 & 57.50 & 52.55 & 52.13 \\ \hline
70B\_finetuned\_noquant\_04 & 67.23 & 78.33 & 72.26 & 72.61 \\ \hline
70B\_noquant\_07 (Base) & 46.22 & 57.50 & 52.55 & 52.13 \\ \hline
70B\_finetuned\_noquant\_07 & 57.98 & 75.00 & 70.07 & 67.82 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Detailed USMLE performance of Llama-3.1 70B variants (base and fine-tuned) across all three steps and overall accuracy. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Configuration** & **Step 1 Accuracy (\%)** & **Step 2 Accuracy (\%)** \\ \hline Medical Students & 57.7 & 61.0 \\ \hline Llama 3.1-70B (1 Epoch, T 0.0) & 67.23 & 76.67 \\ \hline Llama 3.1-70B (1 Epoch, T 0.1) & 66.39 & 80.83 \\ \hline Llama 3.1-70B (1 Epoch, T 0.4) & 67.23 & 78.33 \\ \hline Llama 3.1-70B (1 Epoch, T 0.7) & 57.98 & 75.00 \\ \hline Llama 3.1-70B (3 Epoch, T 0.0) & 64.71 & 83.33 \\ \hline Llama 3.1-70B (3 Epoch, T 0.1) & 65.55 & 83.33 \\ \hline Llama 3.1-70B (3 Epoch, T 0.4) & 66.39 & 83.33 \\ \hline Llama 3.1-70B (3 Epoch, T 0.7) & 66.39 & 83.33 \\ \hline GPT-3.5 & 61.1 & 58.9 \\ \hline GPT-4 & 80.3 & 81.9 \\ \hline GPT-4 Omni & 89.9 & 90.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: USMLE Step-1 and Step-2 accuracy of fine-tuned Llama-3.1 70B models compared with medical-student averages and GPT baselines. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model Configuration** & **Step 1 (\%)** & **Step 2 (\%)** & **Step 3 (\%)** & **Overall (\%)** \\ \hline
70B\_noquant\_00 (Base) & 47.06 & 55.00 & 54.01 & 52.13 \\ \hline
70B\_finetuned\_noquant\_dosample & 67.23 & 76.67 & 75.18 & 73.14 \\ \hline
70B\_noquant\_01 (Base) & 46.22 & 56.67 & 55.47 & 52.93 \\ \hline
70B\_finetuned\_noquant\_01 & 66.39 & 80.83 & 75.18 & 74.20 \\ \hline
70B\_noquant\_04 (Base) & 46.22 & 57.50 & 52.55 & 52.13 \\ \hline
70B\_finetuned\_noquant\_04 & 67.23 & 78.33 & 72.26 & 72.61 \\ \hline
70B\_noquant\_07 (Base) & 46.22 & 57.50 & 52.55 & 52.13 \\ \hline
70B\_finetuned\_noquant\_07 & 57.98 & 75.00 & 70.07 & 67.82 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Detailed USMLE performance of Llama-3.1 70B variants (base and fine-tuned) across all three steps and overall accuracy. The best overall accuracy comes from 70B_finetuned_noquant_01 at temperature 0.1, which reaches 74.20%, about 22 percentage points higher than the untuned 70B baseline. Table 7 summarizes 8B model performance. The best 8-B variant--8B_finetuned_noquant_07 (T = 0.7)--tops out at 42% accuracy, more than 30 percentage points below the 70-B leader, underscoring the performance gap driven by model size. Table 8 details Nemotron model performance. The top Nemotron-70B run--nemo70b_finetuned_noquant_07 (T = 0.7)--scores 63.6%, still about 11 percentage points behind the leading Llama 3.1-70B model, underscoring the edge of the latter. **Evaluation Highlights: CDSS LLM Component**  
* **Fine-Tuning Impact. ** Fine-tuned models outperformed base models significantly, confirming the effectiveness of domain-specific adaptation. * **Model Size. ** 70B models consistently outperformed 8B models across all steps, especially in complex scenarios. * **Parameters. ** The three-epoch, temperature-0.1 checkpoint achieves an 83% Step-2 accuracy--within two points of GPT-4 and well above the medical-student mean. Step-1 gains are more modest, suggesting domain-specific fine-tuning benefits reasoning-heavy questions disproportionately. * **Temperature Sensitivity. ** Lower temperature values produced more deterministic and accurate results, while higher temperatures generated diverse but less reliable outputs.