SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\tau$  controls the variances of negative samples. We experiment with an array of  $\tau$  values and track the corresponding variance of  $f_{\theta}(x,y)$  for negative samples and the mean value of  $f_{\theta}(x,y^{+})$  for positive samples in CIFAR10. As illustrated in Figure 2, we observe a decrease in the variance of negative samples and an increase in the mean value of positive samples as  $\tau$  diminishes. Although this observation might be challenging to interpret based on existing studies, it can be easily explained through Equation (8). Note that  $\tau$  governs the impact of the variance regularizer. A smaller  $\tau$  brings a larger variance penalty, resulting in a reduced variance of negative instances. Simultaneously, as the regularizer's contribution on optimization intensifies, it proportionately diminishes the impact of other loss component on enhancing the similarity of positive instances, thereby leading to a lower mean value of positive instances.
Mean-Variance objective achieves competitive performance. We replace the InfoNCE with the mean-variance objective (cf. Equation (8)) and evaluate the model performance. The results are shown in Table 2. Remarkably, such a simple loss can can deliver competitive results to InfoNCE, thereby affirming the correctness of the approximation in Theorem 3.5. Nevertheless, it marginally underperforms compared to InfoNCE, as it merely represents a second-order Taylor expansion of InfoNCE. However, it is worth highlighting that the mean-variance objective is more efficient.
Table 2: Mean-variance objective (MW) achieves comparable performance. We replace the loss function with variance penalty on negative samples.
Model	Cifar10	Stl10
Top-1	τ	Top-1	τ
SimCLR(τ0)	91.10	0.5	81.05	0.5
SimCLR(τ*)	92.19	0.3	87.91	0.2
MW	91.81	0.3	87.24	0.2
4 Relations among DRO, Constrastive Learning and Mutual Information
Through extensive theoretical analysis, we have far demonstrated that CL is fundamentally a CL-DRO objective that conduct DRO on negative samples. And it is well known that CL is also equivalent to mutual information estimation. A natural question arises: Is there a connection between CL-DRO and MI, as both are theoretical understanding of CL?
To make our conclusion more generalized, we would like to transcend the boundary of KL divergence, but exploring a broader family of  $\phi$ -divergence. First of all, we would like to extend the definition of MI to utilize  $\phi$ -divergence:
Definition 4.1 ( $\phi$ -MI). The  $\phi$ -divergence-based mutual information is defined as:
$$
I _ {\phi} (X; Y) = D _ {\phi} \left(P (X, Y) | | P (X) P (Y)\right) = \mathbb {E} _ {P _ {X}} \left[ D _ {\phi} \left(P _ {0} | | Q _ {0}\right) \right]. \tag {9}
$$
Comparing to KL-divergence,  $\phi$ -divergence showcases greater flexibility and may exhibits superior properties.  $\phi$ -MI, as a potential superior metric for distribution dependence, has been studied by numerous work [49; 50].
Theorem 4.2. For distributions  $P, Q$  such that  $P \ll Q$ , let  $\mathcal{F}$  be a set of bounded measurable functions. Let CL-DRO draw positive and negative instances from  $P$  and  $Q$ , marked as  $\mathcal{L}_{CL - DRO}^{\phi}(P, Q)$ . Then the CL-DRO objective is the tight variational estimation of  $\phi$ -divergence. In fact, we have:
$$
D _ {\phi} (P | | Q) = \max  _ {f \in \mathcal {F}} - \mathcal {L} _ {C L - D R O} ^ {\phi} (P, Q) = \max  _ {f \in \mathcal {F}} \mathbb {E} _ {P} [ f ] - \min  _ {\lambda \in \mathbb {R}} \left\{\lambda + \mathbb {E} _ {Q} \left[ \phi^ {*} (f - \lambda) \right] \right\}. \tag {10}
$$
Here, the choice of  $\phi$  in CL-DRO corresponds to the probability measures in  $D_{\phi}(P||Q)$ . And  $\phi^{*}$  denotes the convex conjugate.
The proof is presented in Appendix A.5. This theorem establishes the connection between CL-DRO and  $\phi$ -divergence. If we replace  $P$  with  $P_0$  and  $Q$  with  $Q_0$ , and simply hypothesis that the metric function  $f_{\theta}$  has a sufficiently large capacity, we have the following corollary:
Corollary 4.3.  $\mathbb{E}_{P_X}[D_\phi (P_0||Q_0)]$  is a tight variational estimation of  $I_{\phi}(X;Y)$
InfoNCE is a tighter MI estimation. The prevailing and widely used variational approximation of  $\phi$ -divergences is the Donsker-Varadhan target  $(I_{DV})$ , defined as  $D_{\phi}(P||Q) \coloneqq \max_{f \in \mathcal{F}} \{\mathbb{E}_P[f] - \mathbb{E}_Q[\phi^*(f)]\}$ . However, it has been observed by Ruderman et al. [51] that this expression is loose when applied to probability measures, as it fails to fully consider the fundamental nature of divergences being defined between probability distributions. To address this limitation, let us further assume, as we do for  $\mathbb{R}$ , that  $\mathbb{E}_P[1] = 1$ . Under this common assumption, we arrive at a more refined formulation:  $D_{\phi}(P||Q) \coloneqq \max_{f \in \mathcal{F}} \{\mathbb{E}_P[f] - \min_{\lambda \in \mathbb{R}} \{\lambda + \mathbb{E}_Q[\phi^*(f - \lambda)]\}$ \. Notably, this result, embracing the infimum over  $\lambda$  in the spirit of Ruderman et al. [51], appears to be consistent with our proof in Theorem 4.2. Hence, we argue that "InfoNCE is a tighter MI estimation."
Furthermore, this corollary provides a rigorous and succinct methodology for deriving InfoNCE from Variational MI. Specifically, given  $\phi(x) = x\log x - x + 1$  for the case of KL-divergence, its conjugate  $\phi^{*}(x) = e^{x} - 1$  can be computed. Subsequently, by resolving the simple convex problem  $\min_{\lambda \in \mathbb{R}}\{\lambda + \tau \mathbb{E}_{Q}[\phi^{*}(\frac{f(x,y) - \lambda}{\tau})]\}$ , we derive the optimal choice of  $\lambda$ , denoted as  $\lambda^{*} = \tau\log \mathbb{E}_{Q_{0}}[e^{f(x,y) / \tau}]$ . Inserting  $\lambda^{*}$  into Equation (10),  $D_{\phi}(P_0||Q_0)$  simplifies to  $\max_f\{\mathbb{E}_{P_0}[f(x,y^+)] - \tau\log \mathbb{E}_{Q_0}[e^{f(x,y) / \tau}]\}$ , which is the minus of the InfoNCE objective.
DRO bridges the gap between MI and InfoNCE. Although previous works such as MINE[24] and CPC[3] have demonstrated that InfoNCE is a lower bound for MI, they still have significant limitations [41]. For instance, MINE uses a critic in Donsker-Varadhan target  $(\mathcal{I}_{DV})$  to derive a bound that is neither an upper nor lower bound on MI, while CPC relies on unnecessary approximations in its proof, resulting in some redundant approximations. In contrast, Theorem 4.2 presents a rigorous and practical method for accurately estimating the tight lower bound of MI.
DRO provides general MI estimation. Although a strict variational bound for MI has been proposed in Poole et al. [41], their discussion is limited to the choice of KL-divergence. Theorem 4.2 is suitable for estimating various  $\phi$ -MI. For example, if we consider the case of  $\chi^2$  divergence, given by  $\phi(x) = \frac{1}{2\sqrt{2}} (x - 1)^2$ , we could obtain convex conjugate  $\phi^*(x) = x + x^2$ . The variational representation becomes  $I_{\chi^2}(X;Y) = D_{\chi^2}(P_0 \| Q_0) = \max_f \{\mathbb{E}_{P_0}[f(x,y^+)] - \mathbb{E}_{Q_0}[f(x,y)] - \mathbb{V}_{Q_0}[f(x,y)]\}$ , where  $\mathbb{V}_{Q_0}[f(x,y)]$  represents the variance of  $f$  on distribution  $Q$ . Our theoretical framework offers the opportunity to estimate flexible  $\phi$ -MI, which can be adapted to suit specific scenarios.
5.1 Shortcomings of InfoNCE
Based on the understanding of CL from DRO perspective, some weaknesses of InfoNCE is revealed:
- Too conservative. Distributionally Robust Optimization (DRO) focuses on the worst-case distribution, often leading to an overemphasis on the hardest negative samples by assigning them disproportionately large weights. More precisely, the weights are proportional to  $\exp [f_{\theta}(x,y) / \tau ]$  causing samples with the highest similarity to receive significantly larger weights. However, in practice, the most informative negative samples are often found in the "head" region (e.g., top-20% region with high similarity) rather than exclusively in the top-1% highest region [52]. Overemphasizing these top instances by assigning excessive weights is typically suboptimal.
- Sensitive to outliers. Existing literature [25] reveals that DRO is markedly sensitive to outliers, exhibiting significant fluctuations in training performance. This is because outliers, once they appear in regions of high similarity, can be assigned exceptionally high weights. Consequently, this phenomenon unavoidably influences the rate of model convergence during the training phase.
5.2 ADNCE
Our goal is to refine the worst-case distribution, aiming to assign more reasonable weights to negative instances. The aspiration is to create a flexible distribution that can be adjusted to concentrate on the informative region. This is accomplished by incorporating Gaussian-like weights, as defined below:
$$
w \left(f _ {\theta} (x, y), \mu , \sigma\right) \propto \frac {1}{\sigma \sqrt {2 \pi}} \exp \left[ - \frac {1}{2} \left(\frac {f _ {\theta} (x , y) - \mu}{\sigma}\right) ^ {2} \right], \tag {11}
$$
where  $\mu$  and  $\sigma$  are two hyper-parameter we could control. As is illustrated in Figure 3,  $\mu$  controls the central region of weight allocation, whereby samples closer to  $\mu$  have larger weights, while  $\sigma$  controls the height of the weight allocation in the central region. Intuitively, a smaller  $\sigma$  results in a more pronounced weight discrepancy among the samples.
Inserting the weights into InfoNCE formula, we obtain the novel ADjusted InfoNCE (ADNCE):
Figure 3: We visualize the training weight of negative samples w.r.t. similarity score. InfoNCE (in BLUE) over-emphasize hard negative samples, while ADNCE utilizes the weight  $w$  (in ORANGE, GREEN, BROWN) to adjust the distribution.
$$
\mathcal {L} _ {\mathrm {A D N C E}} = - \mathbb {E} _ {P} \left[ f _ {\theta} \left(x, y ^ {+}\right) / \tau \right] + \log \mathbb {E} _ {Q _ {0}} \left[ w \left(f _ {\theta} (x, y), \mu , \sigma\right) e ^ {f _ {\theta} (x, y) / \tau} / Z _ {\mu , \sigma} \right], \tag {12}
$$
where  $Z_{\mu ,\sigma}$  denotes the partition function of  $w(f(x,y),\mu ,\sigma)$  , i.e.,  $Z_{\mu ,\sigma} = \mathbb{E}_{Q_0}[w(f_\theta (x,y),\mu ,\sigma)]$
It is noteworthy that our proposed approach only involves two modified lines of code and introduce no additional computational overhead, as compared to InfoNCE [4]. Pytorch-style pseudocode for ADNCE is given in Appendix.
We empirically evaluate our adjusted InfoNCE method — ADNCE, and apply it as a modification to respective classical CL models on image, graph, and sentence data. For all experiments,  $\mu$  is treated as a hyper-parameter to adjust the weight allocation, where  $\sigma$  is set to 1 by default. For detailed experimental settings, please refer to Appendix.
6.1 Image Contrastive Learning
We begin by testing ADNCE on vision tasks using CIFAR10, CIFAR100, and STL10 datasets. We choose SimCLR [4] as the baseline model (termed as InfoNCE  $(\tau_0)$ ). We also include two variants
Table 3: Performance comparisons on multiple loss formulations (ResNet50 backbone, batchsize 256). Top-1 accuracy with linear evaluation protocol.  $\tau_0$  means  $\tau = 0.5$ ,  $\tau^*$  represents grid searching on  $\tau$  and  $\alpha$ -CL-direct is a similar work [16] on CL theory. Bold is highest performance. Each setting is repeated 5 times with different random seeds.
Model	CIFAR10	STL10	CIFAR100
100	200	300	400	100	200	300	400	100	200	300	400
InfoNCE (τ0)	85.70	89.21	90.33	91.01	75.95	78.47	80.39	81.67	59.10	63.96	66.03	66.53
InfoNCE (τ*)	86.54	89.82	91.18	91.64	81.20	84.77	86.27	87.69	62.32	66.85	68.31	69.03
α-CL-direct	87.65	90.11	90.88	91.24	80.91	84.71	87.01	87.96	62.75	66.27	67.35	68.54
ADNCE	87.67	90.65	91.42	91.88	81.77	85.10	87.01	88.00	62.79	66.89	68.65	69.35
Table 4: Sentence embedding performance on STS tasks (Spearman's correlation, "all" setting). Bold indicates the best performance while underline indicates the second best on each dataset. results from SimCSE [9]; all other results are reproduced or reevaluated by ourselves.
Model	STS12	STS13	STS14	STS15	STS16	STS-B	SICK-R	Avg.
GloVe embeddings (avg.)	55.14	70.66	59.73	68.25	63.66	58.02	53.76	61.32
BERTbase-flow	58.40	67.10	60.85	75.16	71.22	68.66	64.47	66.55
BERTbase-whitening	57.83	66.90	60.90	75.08	71.31	68.24	63.73	66.28
CT-BERTbase	61.63	76.80	68.47	77.50	76.48	74.31	69.19	72.05
SimCSE-BERTbase(τ0)	68.40	82.41	74.38	80.91	78.56	76.85	72.23	76.25
SimCSE-BERTbase(τ*)	71.37	81.18	74.41	82.51	79.24	78.26	70.65	76.81
ADNCE-BERTbase	71.38	81.58	74.43	82.37	79.31	78.45	71.69	77.03
SimCSE-RoBERTa(base(τ0))	70.16	81.77	73.24	81.36	80.65	80.22	68.56	76.57
SimCSE-RoBERTa(base(τ*))	68.20	81.95	73.63	81.83	81.55	80.96	69.56	76.81
ADNCE-RoBERTa(base)	69.22	81.86	73.75	82.88	81.88	81.13	69.57	77.10
of vanilla InfoNCE for comprehensive comparisons. One approach is applying a grid search for  $\tau$  (termed as InfoNCE  $(\tau^{*})$ ), which serves to confirm its importance as discussed in Section 3.4. The other is from Tian et al. [16], which proposes a novel approach for setting for the value of  $\tau$  (termed as  $\alpha$ -CL-direct).
As shown in Table 3, a grid search for  $\tau$  has a crucial impact on the model performance. This impact is significant throughout the entire training process, from the early stage (100 epochs) to the later stage (400 epochs). Additionally, ADNCE exhibits sustained improvement and notably enhances performance in the early stages of training. In contrast, while  $\alpha$ -CL-direct introduces a novel approach for setting the value of  $\tau$ , its essence remains weighted towards the most difficult negative samples, hence yielding similar performance compared to fine-tuning  $\tau$  using grid search. In Figure 4, we plot the training curve to further illustrate the stable superiority of ADNCE.
Table 5: Self-supervised representation learning on TUDataset: The baseline results are excerpted from the published papers.
Methods	RDT-B	NCI1	PROTEINS	DD
node2vec	-	54.9±1.6	57.5±3.6	-
sub2vec	71.5±0.4	52.8±1.5	53.0±5.6	-
graph2vec	75.8±1.0	73.2±1.8	73.3±2.1	-
InfoGraph	82.5±1.4	76.2±1.1	74.4±0.3	72.9±1.8
JOAO	85.3±1.4	78.1±0.5	74.6±0.4	77.3±0.5
JOAOv2	86.4±1.5	78.4±0.5	74.1±1.1	77.4±1.2
RINCE	90.9±0.6	78.6±0.4	74.7±0.8	78.7±0.4
GraphCL (τ0)	89.5±0.8	77.9±0.4	74.4±0.5	78.6±0.4
GraphCL (τ*)	90.7±0.6	79.2±0.3	74.7±0.6	78.5±1.0
ADNCE	91.4±0.3	79.3±0.7	75.1±0.6	79.2±0.6
Figure 4: Learning curve for Top-1 accuracy by linear evaluation on CIFAR10 and STL10.
6.2 Sentence Contrastive Learning
We follow the experimental setup in GraphCL [12] and conduct evaluations on 7 semantic textual similarity (STS) tasks in an unsupervised setting. Specifically, we evaluate the model on STS 2012-2016, STS-benchmark, and SICK-Relatedness. To ensure fair comparisons, we use pre-trained checkpoints of BERT and RoBERTa, and randomly sample sentences from the English Wikipedia.
As observed in Table 4, ADNCE consistently outperforms InfoNCE, achieving an average Spearman's correlation of  $77\%$ . The ease of replacing InfoNCE with ADNCE and the resulting substantial performance improvements observed in BERT and RoBERTa serve as testaments to the efficacy and broad applicability of ADNCE. Furthermore, the improvements of  $\tau^{*}$  over  $\tau_{0}$  emphasize the significance of selecting a proper robustness radius.
6.3 Graph Contrastive Learning
To study the modality-agnostic property of ADNCE beyond images and sentences, we conduct an evaluation of its performance on TUDataset [53]. We use a classical graph CL model GraphCL [12] as the baseline. To ensure a fair comparison, we adhered to the same protocol used in GraphCL [12], employing techniques such as node dropout, edge perturbation, attribute masking, and subgraph sampling for data augmentation. We replace the vanilla InfoNCE with two different variants, namely GraphCL  $(\tau^{*})$  obtained through grid search, and our proposed ADNCE.
Table 5 demonstrates that ADNCE outperforms all baselines with a significant margin on four datasets, especially when compared to three state-of-the-art InfoNCE-based contrastive methods, GraphCL, JOAO, and JOAOv2, thereby setting new records on all four datasets. The improvements observed in GraphCL  $(\tau^{*})$  relative to GraphCL  $(\tau_{0})$  align with our understanding of DRO.
We provide a novel perspective on contrastive learning (CL) via the lens of Distributionally Robust Optimization (DRO), and reveal several key insights about the tolerance to sampling bias, the role of  $\tau$ , and the theoretical connection between DRO and MI. Both theoretical analyses and empirical experiments confirm the above findings. Furthermore, we point out the potential shortcomings of CL from the perspective of DRO, such as over-conservatism and sensitivity to outliers. To address these issues, we propose a novel CL loss — ADNCE, and validate its effectiveness in various domains.
The limitations of this work mainly stem from two aspects: 1) Our DRO framework only provides a theoretical explanation for InfoNCE-based methods, leaving a crucial gap in the success of CL without negative samples [4; 26]. 2) ADNCE requires weight allocation to be adjusted through parameters and cannot adaptively learn the best reweighting scheme.
This work is supported by the National Key Research and Development Program of China (2021ZD0111802), the National Natural Science Foundation of China (62372399, 9227010114, 62302321), the Starry Night Science Fund of Zhejiang University Shanghai Institute for Advanced Study (SN-ZJU-SIAS-001), and supported by the University Synergy Innovation Program of Anhui Province (GXXT-2022-040).
[1] Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with application to face verification. In CVPR (1), pages 539-546. IEEE Computer Society, 2005.
[2] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR (2), pages 1735-1742. IEEE Computer Society, 2006.
[3] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018.
[60] Rui Zhu, Bingchen Zhao, Jingen Liu, Zhenglong Sun, and Chang Wen Chen. Improving contrastive learning by visualizing feature transformation. In ICCV, pages 10286-10295. IEEE, 2021.
[61] Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. CoRR, abs/2007.08663, 2020.
A Appendix of Proofs
$$
\begin{array}{l} \mathcal {L} _ {C L D R O} ^ {K L} = - \mathbb {E} _ {P _ {X}} \left[ \mathbb {E} _ {P _ {0}} [ f _ {\theta} (x, y ^ {+}) ] - \min  _ {\alpha \geq 0, \beta} \max  _ {Q \in \mathbb {Q}} \{\mathbb {E} _ {Q} [ f _ {\theta} (x, y) ] - \alpha [ D _ {K L} (Q | | Q _ {0}) - \eta ] + \beta (\mathbb {E} _ {Q _ {0}} [ \frac {Q}{Q _ {0}} ] - 1) \} \right] \\ = - \mathbb {E} _ {P _ {X}} \mathbb {E} _ {P _ {0}} \left[ \alpha^ {*} (\eta) \log \frac {e ^ {f _ {\theta} (x , y ^ {+}) / \alpha^ {*} (\eta)}}{\mathbb {E} _ {Q _ {0}} \left[ e ^ {f _ {\theta} (x , y) / \alpha^ {*} (\eta)} \right]} \right] + C o n s t a n t \\ = \alpha^ {*} (\eta) \mathcal {L} _ {\text {I n f o N C E}} + \text {C o n s t a n t}, \tag {4} \\ \end{array}
$$
Proof. In all the following appendix, for the sake of conserving space and clarity of expression, we have opted to simplify  $f_{\theta}(x,y)$  as  $f_{\theta}$  and omit  $\mathbb{E}_{P_X}$ .
To complete the proof, we start with giving some important notations and theorem.
Definition A.1 ( $\phi$ -divergence [54]). For any convex function  $\phi$  with  $\phi(1) = 0$ , the  $\phi$ -divergence between  $Q$  and  $Q_0$  is:
$$
D _ {\phi} (Q | | Q _ {0}) := \mathbb {E} _ {Q _ {0}} [ \phi (d Q / d Q _ {0}) ] \tag {13}
$$
where  $D_{\phi}(Q||Q_0) = \infty$  if  $Q$  is not absolutely continuous with respect to  $Q_0$ . Specially, when  $\phi (x) = x\log x - x + 1$ ,  $\phi$ -divergence degenerates to the well-known KL divergence.
Definition A.2 (Convex conjugate [55]). We consider a pair  $(A,B)$  of topological vector spaces and a bilinear form  $\langle \cdot ,\cdot \rangle \to \mathbb{R}$  such that  $(A,B,\langle \cdot ,\cdot \rangle)$  form a dual pair. For a convex function  $f:\mathbb{R}\rightarrow \mathbb{R}$ ,  $domf\coloneqq \{x\in \mathbb{R}:f(x) < \infty \}$  is the effective domain of  $f$ . The convex conjugate, also known as the Legendre-Fenchel transform, of  $f:A\to \mathbb{R}$  is the function  $f^{*}:B\to \mathbb{R}$  defined as
$$
f ^ {*} (b) = \sup  _ {a} \left\{a b - f (a) \right\}, \quad b \in B \tag {14}
$$
Theorem A.3 (Interchange of minimization and integration [56]). Let  $(\Omega, \mathcal{F})$  be a measurable space equipped with  $\sigma$ -algebra  $\mathcal{F}$ ,  $L^p(\Omega, \mathcal{F}, P)$  be the linear space of measurable real valued functions  $f: \Omega \to \mathbb{R}$  with  $||f||_p < \infty$ , and let  $\mathcal{X} := L^p(\Omega, \mathcal{F}, P)$ ,  $p \in [1, +\infty]$ . Let  $g: \mathbb{R} \times \Omega \to \mathbb{R}$  be a normal integrand, and define on  $\mathcal{X}$ . Then,
$$
\min  _ {x \in \mathcal {X}} \int_ {\Omega} g (x (\omega), \omega) d P (\omega) = \int_ {\Omega} \min  _ {s \in \mathbb {R}} g (s, \omega) d P (\omega) \tag {15}
$$
To ease the derivation, we denote the likelihood raito  $L(x,y) = Q(x,y) / Q_0(x,y)$ . Note that the  $\phi$ -divergence between  $Q$  and  $Q_0$  is constrained, and thus  $L(.)$  is well-defined. For brevity, we usually short  $L(x,y)$  as  $L$ . And in terms of Definition A.1 of  $\phi$ -divergence, the expression of CL-DRO becomes:
$$
\mathcal {L} _ {\mathrm {C L - D R O}} ^ {\phi} = - \mathbb {E} _ {P _ {0}} [ f _ {\theta} ] + \max  _ {L} \mathbb {E} _ {Q _ {0}} [ f _ {\theta} L ] \quad s. t. \mathbb {E} _ {Q _ {0}} [ \phi (L) ] \leq \eta \tag {16}
$$
Note that  $\mathbb{E}_{Q_0}[f_\theta L]$  and  $\mathbb{E}_{Q_0}[\phi (L)]$  are both convex in  $L$ . We use the Lagrangian function solver:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {C L - D R O}} ^ {\phi} = - \mathbb {E} _ {P _ {0}} [ f _ {\theta} ] + \min  _ {\alpha \geq 0, \beta} \max  _ {L} \left\{\mathbb {E} _ {Q _ {0}} [ f _ {\theta} L ] - \alpha \left[ \mathbb {E} _ {Q _ {0}} [ \phi (L) ] - \eta \right] + \beta \left(\mathbb {E} _ {Q _ {0}} [ L ] - 1\right) \right\} \\ = - \mathbb {E} _ {P _ {0}} [ f _ {\theta} ] + \min  _ {\alpha \geq 0, \beta} \left\{\alpha \eta - \beta + \alpha \max  _ {L} \left\{\mathbb {E} _ {Q _ {0}} [ \frac {f _ {\theta} + \beta}{\alpha} L - \phi (L) ] \right\} \right\} \\ = - \mathbb {E} _ {P _ {0}} [ f _ {\theta} ] + \min  _ {\alpha \geq 0, \beta} \left\{\alpha \eta - \beta + \alpha \mathbb {E} _ {Q _ {0}} \left[ \max  _ {L} \left\{\frac {f _ {\theta} + \beta}{\alpha} L - \phi (L) \right\} \right] \right\} \tag {17} \\ = - \mathbb {E} _ {P _ {0}} [ f _ {\theta} ] + \min  _ {\alpha \geq 0, \beta} \left\{\alpha \eta - \beta + \alpha \mathbb {E} _ {Q _ {0}} [ \phi^ {*} (\frac {f _ {\theta} + \beta}{\alpha}) ] \right\} \\ \end{array}
$$
The first equality holds due to the strong duality [57]. The second equality is a re-arrangement for optimizing  $L$ . The third equation follows by the Theorem A.3. The last equality is established based
on the definition of convex conjugate A.2. When we choose KL-divergence, we have  $\phi_{KL}(x) = x\log x - x + 1$ . It can be deduced that  $\phi_{KL}^{*}(x) = e^{x} - 1$ . Then, we have: