## 3 Proposed Approach
### Contrastive Representation Learning (CRL)  
The conventional next-sentence prediction (NSP) objective in BERT captures only coarse sentence adjacency, which is often insufficient for the subtle discourse structure of long discharge summaries. We, therefore, replace NSP with a margin-based contrastive objective that pulls semantically coherent sentences together and pushes unrelated sentences apart in the latent space.  
Positive pairs were defined not only as immediately consecutive sentences but also as clinically linked sentences that share at least one UMLS clinical concept (CUI) within a two-sentence window; pilot studies showed that this broader criterion yields richer supervision than simple adjacency.  
For each anchor sentence, the immediately following sentence in the same summary is treated as a positive partner, whereas negative partners are selected in two ways: (i) a non-consecutive sentence drawn from a different section of the same summary (intra-document negative), and (ii) a sentence sampled from a different patient's summary (inter-document negative). This hybrid sampling scheme supplies the model with semantically unambiguous negative examples while preserving domain context.  
After encoding each sentence with BERT and projecting it through a tanh-activated dense layer, we obtain three vectors, \(z_{i}\), \(z_{i}^{+}\),and \(z_{i}^{-}\), that represent, respectively, the anchor, positive, and negative sentences in a mini-batch of size \(N\). Using the Euclidean distance \(D(u,v)=|u-v|_{2}\), we minimize the margin-based contrastive loss in Equation (1) [23].  
\[L_{contrast}=\frac{1}{N}{\sum_{i=1}^{N}}\Big{[}D\big{(}z_{i},z_{i}^{+}\big{)}^{ 2}+\alpha\ max\big{(}0,m-D\big{(}z_{i},z_{i}^{-}\big{)}\big{)}^{2}\Big{]} \tag{1}\]  
We evaluated margins m \(\in\){1.0,1.5,2.0.2.5] on the MedNLI development split and found that accuracy peaked at m = 2.0. Therefore, we fixed m = 2.0 for all experiments. We also set \(\alpha\) = 0.5 based on the same pilot study. Detailed numbers are available upon request.  
Figure 3 compares the traditional NSP approach with our CRL method, highlighting how CRL shifts from a soft-max next-sentence classifier to a distance-based objective for more robust semantic representation.  
Figure 4 illustrates how Equation (1) contracts distances between consecutive or contextually similar sentences (positive pairs) while expanding those between non-consecutive or unrelated sentences (negative pairs).  
To leverage unlabelled text more effectively, we combine Equation (1) with whole-word masked-language modeling (WWM-MLM) and optimize the joint objective:  
\[L_{total}=\lambda L_{MLM}+(1-\lambda)L_{contrast},\ \ \ \ \lambda=0.5 \tag{2}\]  
where \(\lambda\) balances MLM and contrastive loss (fixed to 0.5). Both objectives share the same masked tokens so that gradient signals are coherent. The margin mmm and coefficient \(\alpha\backslash\)alpha\(\alpha\) were tuned on the MedNLI development split and then held constant in every subsequent experiment. All remaining hyper-parameters--including the number of pre-training steps, learning rate, batch size, and sequence length--follow exactly the configuration reported in Section 4.1 and Table 2.  
Notation. z--sentence embedding after the projection head; \(D\)--Euclidean distance; \(N\)--mini-batch size; \(m\)--margin; \(\alpha\)--scaling coefficient; \(\lambda\)--MLM/contrastive mixing weight.