## 4 Experimental Result
### Evaluation of the CRPT Model  
To assess the performance of our proposed Discharge Summary CRPT model, we utilized the BLUE benchmark dataset, which is designed for medical language understanding tasks across both biomedical and clinical domains. The BLUE benchmark evaluates models on various sentence-level and token-level tasks, such as inference, sentence similarity, named entity recognition (NER), and relation extraction. Table 3 provides a summary of the experimental datasets used for evaluation, including MedNLI for inference, BIOSSES for sentence similarity, ShARe/CLEFE for NER, and i2b2-2010 for relation extraction. Standard deviations from five independent runs are reported alongside median scores, underscoring the robustness of our models.  
Our evaluation compared the performance of BERT-Base, BioBERT, Discharge Summary BERT, Bio+Discharge Summary BERT, BlueBERT, and the Discharge Summary CRPT models on the BLUE benchmark tasks. Table 4 presents the performance results of the models across various NLP tasks. The Bio+Discharge Summary CRPT model achieved the highest performance across most tasks, outperforming other baseline models, especially in MedNLI and BIOSSES tasks.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **MedNLI (Acc)** & **BIOSSES (r)** & **ShARe/CLEFE (F1)** & **i2b2-2010 (F1)** \\ \hline BERT-Base & 0.757 \(\pm\) 0.011 & 0.800 \(\pm\) 0.089 & 0.798 \(\pm\) 0.006 & 0.728 \(\pm\) 0.006 \\ \hline BioBERT & 0.819 \(\pm\) 0.010 & 0.821 \(\pm\) 0.086 & 0.812 \(\pm\) 0.005 & 0.737 \(\pm\) 0.006 \\ \hline Discharge & & & & \\ Summary BERT & 0.785 \(\pm\) 0.011 & 0.718 \(\pm\) 0.101 & 0.814 \(\pm\) 0.005 & 0.736 \(\pm\) 0.006 \\ \hline Bio+Discharge & & & & \\ Summary BERT & 0.814 \(\pm\) 0.010 & 0.819 \(\pm\) 0.086 & 0.819 \(\pm\) 0.005 & 0.745 \(\pm\) 0.005 \\ \hline BlueBERT & 0.841 \(\pm\) 0.010 & 0.839 \(\pm\) 0.082 & **0.838**\(\pm\) 0.005 & 0.757 \(\pm\) 0.005 \\ \hline Discharge & & & & \\ Summary CRPT & 0.819 \(\pm\) 0.010 & 0.773 \(\pm\) 0.094 & 0.827 \(\pm\) 0.005 & 0.754 \(\pm\) 0.005 \\ \hline Bio+Discharge & & & & \\ Summary CRPT & **0.847**\(\pm\) 0.010 & **0.848**\(\pm\) 0.080 & 0.837 \(\pm\) 0.005 & **0.761**\(\pm\) 0.005 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance across BLUE clinical NLP tasks (Median \(\pm\) Standard Error).  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Corpus** & **Task** & **Metrics** & **Train** & **Dev** & **Test** \\ \hline MedNLI & Inference & Accuracy & 11,232 & 1395 & 1422 \\ BIOSSES & Sentence similarity & Pearson & 64 & 16 & 20 \\ ShARe/CLEFE & NER & F1 & 4628 & 1075 & 5195 \\ i2b2-2010 & Relation extraction & F1 & 3110 & 11 & 6293 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Summary of experimental dataset.  
In the second experiment, we analyzed the models' pre-training loss on the training set. In a comparison of training loss between the Discharge Summary BERT and Discharge Summary CRPT models over 150K training steps, both the Discharge Summary BERT and Discharge Summary CRPT models exhibited similar loss patterns at the beginning of the training. However, the Discharge Summary CRPT model demonstrated a more stable and consistent decrease in loss throughout the training process. The CRPT model maintained lower loss values as the training progressed, indicating a more efficient and stable pre-training process.  
These findings suggest that applying contrastive representation learning within the CRPT model enhances its ability to capture and refine semantic relationships in clinical texts, leading to improved performance. The Discharge Summary CRPT model, in particular, benefits from this learning method, showing more refined understanding capabilities for clinical domain tasks. This outcome provides essential insights for advancing NLP in the medical field, particularly in discharge summary analysis and other clinical NLP tasks.