SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By strong convexity of  $\mathbf{P} \to \mathrm{KL}(\mathbf{P}|\mathbf{K}_{\sigma})$  and convexity of the constraints the problem (SEA-Proj) admits a unique solution. Moreover, the Lagrangian of this problem takes the following form, where  $\boldsymbol{\omega} \in \mathbb{R}_+^n$ ,  $\boldsymbol{\mu} \in \mathbb{R}^n$  and  $\Gamma \in \mathbb{R}^{n \times n}$ :
$$
\mathcal {L} (\mathbf {P}, \boldsymbol {\mu}, \omega , \Gamma) = \mathrm {K L} (\mathbf {P} | \mathbf {K} _ {\sigma}) + \langle \omega , (\log \xi + 1) \mathbf {1} - \mathrm {H} _ {r} (\mathbf {P}) \rangle + \langle \boldsymbol {\mu}, \mathbf {1} - \mathbf {P} \mathbf {1} \rangle + \langle \boldsymbol {\beta}, \mathbf {P} - \mathbf {P} ^ {\top} \rangle .
$$
Strong duality holds by Slater's conditions thus the KKT conditions are necessary and sufficient. In particular if  $\mathbf{P}^{\star}$  and  $(\omega^{\star},\mu^{\star},\beta^{\star})$  satisfy
$$
\nabla_ {\mathbf {P}} \mathcal {L} \left(\mathbf {P} ^ {\star}, \boldsymbol {\mu} ^ {\star}, \boldsymbol {\omega} ^ {\star}, \boldsymbol {\Gamma} ^ {\star}\right) = \log \left(\mathbf {P} ^ {\star} \oslash \mathbf {K}\right) + \operatorname {d i a g} \left(\boldsymbol {\omega} ^ {\star}\right) \log \mathbf {P} ^ {\star} - \boldsymbol {\mu} ^ {\star} \mathbf {1} ^ {\top} + \boldsymbol {\beta} ^ {\star} - \boldsymbol {\beta} ^ {\star \top} = \mathbf {0}
$$
$$
\begin{array}{l} \mathbf {P} ^ {\star} \mathbf {1} = \mathbf {1}, \mathrm {H} _ {r} (\mathbf {P} ^ {\star}) \geq (\log \xi + 1) \mathbf {1}, \mathbf {P} ^ {\star} = \mathbf {P} ^ {\star^ {\top}} \\ \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \end{array} \tag {KKT-Proj}
$$
$$
\omega^ {\star} \geq 0
$$
$$
\forall i, \omega_ {i} ^ {\star} (\mathrm {H} (\mathbf {P} _ {i:} ^ {\star}) - (\log \xi + 1)) = 0.
$$
then  $\mathbf{P}^{\star}$  is a solution to (SEA-Proj) and  $(\omega^{\star},\mu^{\star},\beta^{\star})$  are optimal dual variables. The first condition rewrites
$$
\forall (i, j), \log \left(P _ {i j} ^ {\star}\right) + \frac {1}{\sigma} C _ {i j} + \omega_ {i} ^ {\star} \log \left(P _ {i j} ^ {\star}\right) - \mu_ {i} ^ {\star} + \beta_ {i j} ^ {\star} - \beta_ {j i} ^ {\star} = 0, \tag {27}
$$
$$
\forall (i, j), \sigma \left(1 + \omega_ {i} ^ {\star}\right) \log \left(P _ {i j} ^ {\star}\right) + C _ {i j} - \sigma \mu_ {i} ^ {\star} + \sigma \left(\beta_ {i j} ^ {\star} - \beta_ {j i} ^ {\star}\right) = 0. \tag {28}
$$
Now take  $\mathbf{P}^{\mathrm{se}}$  the optimal solution of (SEA). As written in the proof Proposition 5 of  $\mathbf{P}^{\mathrm{se}}$  and the optimal dual variables  $(\gamma^{\star},\lambda^{\star},\Gamma^{\star})$  satisfy the KKT conditions:
$$
\forall (i, j), C _ {i j} + \gamma_ {i} ^ {\star} \log P _ {i j} ^ {\mathrm {s e}} - \lambda_ {i} ^ {\star} + \Gamma_ {i j} ^ {\star} - \Gamma_ {j i} ^ {\star} = 0
$$
$$
\mathbf {P} ^ {\mathrm {s e}} \mathbf {1} = \mathbf {1}, \mathrm {H} _ {r} \left(\mathbf {P} ^ {\mathrm {s e}}\right) \geq (\log \xi + 1) \mathbf {1}, \mathbf {P} ^ {\mathrm {s e}} = \left(\mathbf {P} ^ {\mathrm {s e}}\right) ^ {\top} \tag {KKT-SEA}
$$
$$
\gamma^ {\star} \geq 0
$$
$$
\forall i, \gamma_ {i} ^ {\star} (\mathrm {H} (\mathbf {P} _ {i:} ^ {\mathrm {s e}}) - (\log \xi + 1)) = 0.
$$
By hypothesis  $\gamma^{\star} > 0$  which gives  $\forall i, \mathrm{H}(\mathbf{P}_{i:}^{\mathrm{se}}) - (\log \xi + 1) = 0$ . Now take  $0 < \sigma \leq \min_{i} \gamma_{i}^{\star}$  and define  $\forall i, \omega_{i}^{\star} = \frac{\gamma_{i}^{\star}}{\sigma} - 1$ . Using the hypothesis on  $\sigma$  we have  $\forall i, \omega_{i}^{\star} \geq 0$  and  $\omega^{\star}$  satisfies  $\forall i$ ,  $\sigma(1 + \omega_{i}^{\star}) = \gamma_{i}^{\star}$ . Moreover for any  $i \in [[n]]$
$$
\omega_ {i} ^ {\star} \left(\mathrm {H} \left(\mathbf {P} _ {i:} ^ {\mathrm {s e}}\right) - (\log \xi + 1)\right) = 0. \tag {29}
$$
Define also  $\forall i, \mu_i^{\star} = \lambda_i^{\star} / \sigma$  and  $\forall (i,j), \beta_{ij}^{\star} = \Gamma_{ij}^{\star} / \sigma$ . Since  $\mathbf{P}^{\mathrm{se}}$ ,  $(\boldsymbol{\gamma}^{\star}, \boldsymbol{\lambda}^{\star}, \boldsymbol{\Gamma}^{\star})$  satisfies the KKT conditions (KKT-SEA) then by the previous reasoning  $\mathbf{P}^{\mathrm{se}}$ ,  $(\boldsymbol{\omega}^{\star}, \boldsymbol{\mu}^{\star}, \boldsymbol{\beta}^{\star})$  satisfy the KKT conditions (KKT-Proj) and in particular  $\mathbf{P}^{\mathrm{se}}$  is an optimal solution of (SEA-Proj) since KKT conditions are sufficient. Thus we have proven that  $\mathbf{P}^{\mathrm{se}} \in \arg \min_{\mathbf{P} \in \mathcal{H}_{\xi} \cap \mathcal{S}} \mathrm{KL}(\mathbf{P} | \mathbf{K}_{\sigma})$  and by the uniqueness of the solution this is in fact an equality.
B Sensitivity Analysis for Dimensionality Reduction Experiments
In Figure 8, we extend the sensitivity analysis performed for spectral clustering (Figure 5) to DR scores. One can notice that tSNEkhorn outperforms tSNE on a wide range of perplexity values.
Figure 8: Dimensionality reduction scores as a function of the perplexity parameter.