SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 2 Let  $(X,W,V,W^{\prime})\in \mathbb{R}^{|\mathcal{S}|\times n_{0}}\times \mathbb{R}^{n_{0}\times n_{1}}\times \mathbb{R}^{n_{1}\times n_{2}}\times \mathbb{R}^{n_{0}\times n_{1}}$ . Suppose  $\mathrm{rank}(\sigma (XW)) = |\mathcal{S}|$  and  $\mathrm{rank}(\sigma (XW^{\prime})) = |\mathcal{S}|$ . Then, under Assumption 2 and 3, there exists a continuous path map  $c:[0,1]\to \mathbb{R}^{n_0\times n_1}\times \mathbb{R}^{n_1\times n_2}$  with  $c(\lambda) = (W(\lambda),V(\lambda))$  such that
1)  $c(0) = (W, V)$ ,
2)  $\sigma(XW(\lambda))V(\lambda) = \sigma(XW)V, \forall \lambda \in [0,1]$ ,
3)  $W(1) = W'$ .
To prove Theorem 2, it suffices to show that for any  $\theta_{1} = (W_{1,l},b_{1,l})_{l = 1}^{L}\in \mathcal{U}_{\lambda ,r}^{\Omega}$  and  $\theta_{2} = (W_{2,l},b_{2,l})_{l = 1}^{L}\in \mathcal{U}_{\lambda ,r}^{\Omega}$  there exists a connected path that is completely within  $\mathcal{U}_{\lambda ,r}^{\Omega}$ .
Applying Lemma 1 with  $(X, W_{1,1}, b_{1,1}, W_{1,2})$  and  $(X, W_{2,1}, b_{2,1}, W_{2,2})$ , the problem simplifies to showing the existence of a continuous path within  $\mathcal{U}_{\lambda,r}^{\Omega}$  that connects
$$
\theta_ {1} ^ {\prime} = ((W _ {1, 1} ^ {\prime}, b _ {1, 1} ^ {\prime}), (W _ {1, 2} ^ {\prime}, b _ {1, 2}), (W _ {1, l}, b _ {1, l}) _ {l = 3} ^ {L})
$$
$$
\theta_ {2} ^ {\prime} = \left(\left(W _ {2, 1} ^ {\prime}, b _ {2, 1} ^ {\prime}\right), \left(W _ {2, 2} ^ {\prime}, b _ {1, 2}\right), \left(W _ {2, l}, b _ {2, l}\right) _ {l = 3} ^ {L}\right)
$$
$$
\operatorname {r a n k} \left(F _ {1} ^ {\theta_ {1} ^ {\prime}}\right) = \operatorname {r a n k} \left(F _ {1} ^ {\theta_ {2} ^ {\prime}}\right) = | \mathcal {S} |.
$$
Then, we can apply Lemma 2 with  $([X,1_{|S|}],[W_{1,1}^{\prime \top},b_{1,1}^{\prime}]^{\top},W_{1,2}^{\prime},[W_{2,1}^{\prime \top},b_{2,1}^{\prime}]^{\top})$  to show that there is a continuous path between  $\theta_1^\prime$  and  $\theta_1^{\prime \prime}$  with  $\theta_1^{\prime \prime} = ((W_{2,1}^{\prime},b_{2,1}^{\prime}),(W_{1,2}^{\prime \prime},b_{1,2}),(W_{1,l},b_{1,l})_{l = 3}^{L})$  such that
$$
\operatorname {r a n k} \left(F _ {1} ^ {\theta_ {1} ^ {\prime \prime}}\right) = \operatorname {r a n k} \left(F _ {1} ^ {\theta_ {1} ^ {\prime}}\right) = | \mathcal {S} |.
$$
As a consequence, now we simply have to show that  $\theta_1^{\prime \prime}$  and  $\theta_2^\prime$  is connected by a continuous path within  $\mathcal{U}_{\lambda ,r}^{\Omega}$ .
Note that  $\theta_1''$  and  $\theta_2'$  have identical first layer parameters and thus the same first layer output, which is full rank. This allows us to treat the layers from 2 to  $L$  as a new network and apply Proposition 1 (which requires the input to be full rank) to the new network to guarantee that there exists a continuous path map  $c: [0,1] \to \Omega_2 \times \ldots \times \Omega_k$  such that  $c(0) = ((W_{1,2}'', b_{1,2}), (W_{1,l}, b_{1,l})_{l=3}^L)$ ,  $c(1) = ((W_{2,2}', b_{1,2}), (W_{2,l}, b_{2,l})_{l=3}^L)$ , and
$$
\min \{J _ {r} (\pi_ {\theta_ {1}}), J _ {r} (\pi_ {\theta_ {2}}) \} \leq J _ {r} (\pi_ {((W _ {2, 1} ^ {\prime}, b _ {2, 1} ^ {\prime}), c (\alpha))}) \leq \max \{J _ {r} (\pi_ {\theta_ {1}}), J _ {r} (\pi_ {\theta_ {2}}) \}
$$
for all  $\alpha \in [0,1]$ . This implies that there is indeed a continuous path between  $\theta_1^{\prime \prime}$  and  $\theta_{2}^{\prime}$  within  $\mathcal{U}_{\lambda ,r}^{\Omega}$ . Similar to the proof of Theorem 1, the claim on the connectedness simply follows from the fact that the construction of the path map  $p$  does not depend on the reward function.
For each layer of the neural network  $k = 1, \dots, L$ , we define  $\Omega_k^* \subseteq \Omega_k$  to be the set of weights  $W_k$  and biases  $b_k$  of layer  $k$  such that  $W_k$  is full rank, i.e.
$$
\Omega_ {k} ^ {\star} = \left\{\left(W _ {k}, b _ {k}\right) \in \Omega_ {k}: W _ {k} \text {i s f u l l r a n k} \right\}. \tag {15}
$$
We denote  $\Omega^{\star} = \Omega_{1}^{\star}\times \Omega_{2}^{\star}\times \ldots \times \Omega_{L}^{\star}$ . Next, we introduce the following lemmas in aid of the analysis.
Condition 1 Given  $\theta = (W_l, b_l)_{l=2}^L$ ,  $W_l$  has full rank for every  $l \in [2, L]$ .
Lemma 3 Under Assumption 2, 3, and Condition 1, given any  $k \in [2, L]$  and matrix  $F \in \mathbb{R}^{|S| \times n_k}$ , there exists a continuous map  $h: \Omega_2^\star \times \ldots \times \Omega_k^\star \times \mathbb{R}^{|S| \times n_k} \to \Omega_1$  such that
1) Given  $((W_2, b_2), \ldots, (W_k, b_k), F) \in \Omega_2^\star \times \ldots \times \Omega_k^\star \times \mathbb{R}^{|S| \times n_k}$ , we have
$$
F _ {k} ^ {h ((W _ {l}, b _ {l}) _ {l = 2} ^ {k}, F), (W _ {l}, b _ {l}) _ {l = 2} ^ {k}} = F.
$$
2) For any  $\theta^{\star} = (W_{l}^{\star},b_{l}^{\star})_{l = 1}^{L}\in \Omega_{1}\times \Omega_{2}^{\star}\times \ldots \times \Omega_{L}^{\star}$ , there exists a continuous path map  $p:[0,1]\to \Omega_1\times \Omega_2^\star \times \ldots \times \Omega_L^\star$  such that  $p(0) = \theta^{\star}$ ,  $p(1) = \big(h((W_l^\star ,b_l^\star)_{l = 2}^k,F_k^{\theta^\star})$ ,  $(W_{l}^{\star},b_{l}^{\star})_{l = 2}^{L}$ ), and  $F_{L}^{p(\alpha)} = F_{L}^{\theta^{\star}}$  for all  $\alpha \in [0,1]$ .
Lemma 4 Given two connected sets  $\mathcal{A} \subseteq \mathbb{R}^{m_1 \times n}$  and  $\mathcal{B} \subseteq \mathbb{R}^{n \times m_2}$ , the set  $\{ab \mid a \in \mathcal{A}, b \in \mathcal{B}\}$  is connected. Given two connected sets  $\mathcal{A}, \mathcal{B} \subseteq \mathbb{R}^{m \times n}$ , the set  $\{a + b \mid a \in \mathcal{A}, b \in \mathcal{B}\}$  is connected.
Lemma 5 Under Assumption 2, for any  $\theta \in \Omega$ , there exist  $\theta^{\star} \in \Omega^{\star}$  and a continuous path map  $p:[0,1] \to \Omega$  such that  $p(0) = \theta$ ,  $p(1) = \theta^{\star}$ , and  $F_{L}^{p(\alpha)} = F_{L}^{\theta}$  for all  $\alpha \in [0,1]$ .
Lemma 6 If  $n < m$ , then the set  $\mathcal{F} = \{F \in \mathbb{R}^{m \times n} \mid \operatorname{rank}(F) = n\}$  is connected. In other words, given  $F_1, F_2 \in \mathcal{F}$ , there exists a continuous path map  $q: [0,1] \to \mathcal{F}$  such that  $q(0) = F_1$  and  $q(1) = F_2$ .
Fix a  $\lambda \in \mathbb{R}$ . To show the superlevel set  $\mathcal{U}_{\lambda,r}^{\Omega}$  is connected, it suffices to show that for any  $\theta_1, \theta_2 \in \mathcal{U}_{\lambda,r}^{\Omega}$ , there exists a continuous path between them that is completely in  $\mathcal{U}_{\lambda,r}^{\Omega}$ .
Without any loss of generality, we can safely assume that both  $\theta_{1} = (W_{1,l},b_{1,l})_{l = 1}^{L}$  and  $\theta_{2} = (W_{2,l},b_{2,l})_{l = 1}^{L}$  satisfy Condition 1, since otherwise by Lemma 5 we can find a continuous path from  $\theta_{1}$  and  $\theta_{2}$  that leads to one satisfying Condition 1. We denote the policies parameterized by  $\theta_{1},\theta_{2}$  as  $\pi_1,\pi_2$  , i.e.
$$
\pi_ {1} = F _ {L} ^ {\theta_ {1}}, \quad \pi_ {2} = F _ {L} ^ {\theta_ {2}}.
$$
By Lemma 3, there is a continuous path from  $\theta_1 / \theta_2$  to  $\theta_1' / \theta_2'$  where we define
$$
\theta_ {1} ^ {\prime} = \left(h \left(\left(W _ {1, l}, b _ {1, l}\right) _ {l = 2} ^ {L}, \pi_ {1}\right), \left(W _ {1, l}, b _ {1, l}\right) _ {l = 2} ^ {L}\right),
$$
and  $\theta_2^{\prime} = \left(h\left((W_{2,l},b_{2,l})_{l = 2}^{L},\pi_{2}\right),(W_{2,l},b_{2,l})_{l = 2}^{L}\right)$
Now, we just have to show that there exists a continuous path between  $\theta_1'$  and  $\theta_2'$  that is completely within  $\mathcal{U}_{\lambda,r}^{\Omega}$ . By Lemma 6, we know that for  $l = 2,..,L$ , there exists a continuous path map  $q_l:[0,1]\to \Omega_l^\star$  such that  $q_{l}(1) = W_{1,l}$  and  $q_{l}(0) = W_{2,l}$ . Then, we construct the map  $q:[0,1]\to \Omega$
$$
q (\alpha) = \left(h ((q _ {l} (\alpha), \alpha b _ {1, l} + (1 - \alpha) b _ {2, l}) _ {l = 2} ^ {L}, \pi_ {1}), (q _ {l} (\alpha), \alpha b _ {1, l} + (1 - \alpha) b _ {2, l}) _ {l = 2} ^ {L}\right) \quad \forall \alpha \in [ 0, 1 ].
$$
It is obvious that  $q$  is a continuous map as  $h, q_2, \ldots, q_L$  are continuous. In addition,  $F_L^{q(\alpha)} = \pi_1$  for all  $\alpha \in [0, 1]$ , and  $q(1) = \theta_1'$ . We define
$$
\theta_ {1} ^ {\prime \prime} = q (0) = \left(h \left(\left(W _ {2, l}, b _ {2, l}\right) _ {l = 2} ^ {L}, \pi_ {1}\right), \left(W _ {2, l}, b _ {2, l}\right) _ {l = 2} ^ {L}\right).
$$
Now our aim simplifies to finding a continuous path between  $\theta_1^{\prime \prime}$  and  $\theta_2^\prime$  that is completely in  $\mathcal{U}_{\lambda ,r}^{\Omega}$ . To show that this path exists, we construct a continuous map  $t:[0,1]\to \Omega$  as follows
$$
t (\alpha) = \left(h \left(\left(W _ {2, l}, b _ {2, l}\right) _ {l = 2} ^ {L}, \tilde {\pi} _ {\alpha}\right), \left(W _ {2, l}, b _ {2, l}\right) _ {l = 2} ^ {L}\right) \quad \forall \alpha \in [ 0, 1 ],
$$
where  $\tilde{\pi}$  is defined entry-wise
$$
\tilde {\pi} _ {\alpha} (a \mid s) = \frac {\alpha \mu_ {\pi_ {1}} (s) \pi_ {1} (a \mid s) + (1 - \alpha) \mu_ {\pi_ {2}} (s) \pi_ {2} (a \mid s)}{\alpha \mu_ {\pi_ {1}} (s) + (1 - \alpha) \mu_ {\pi_ {2}} (s)}.
$$
It can be seen that  $t$  is indeed continuous since  $\tilde{\pi}_{\alpha}$  is continuous in  $\alpha$ , and  $t(0) = \theta_2'$  and  $t(1) = \theta_1''$ . What remains to be shown is that  $F_{L}^{t(\alpha)} \in \mathcal{U}_{\lambda, r}^{\Omega}$ , i.e.  $J_{r}(F_{L}^{t(\alpha)}) \geq \lambda$ . By the definition of  $h$  in Lemma 3,  $F_{L}^{t(\alpha)} = \tilde{\pi}_{\alpha}$ . It has been shown in the proof of Theorem 1 that indeed  $J_{r}(\tilde{\pi}_{\alpha}) \geq \lambda$  provided that  $J_{r}(\pi_{1}) \geq \lambda$  and  $J_{r}(\pi_{2}) \geq \lambda$ . This concludes the proof of Proposition 1.
C Proof of Supporting Lemmas
This lemma is adapted from Lemma 5.2 of Nguyen [2019].
This lemma is adapted from Lemma 5.3 of Nguyen [2019].
C.3 Proof of Lemma 3
We provide a proof for the case  $k = L$ . For  $k \neq L$ , the proof can be found in Nguyen [2019][Lemma 3.3].
For  $((W_2, b_2), \dots, (W_L, b_L), \pi) \in \Omega_2^\star \times \dots \times \Omega_L^\star \times \Delta_A^S$ , we define the map  $h$  as follows
$$
h \left(\left(W _ {l}, b _ {l}\right) _ {l = 2} ^ {L}, \pi\right) = \left(\widehat {W} _ {1}, \widehat {b} _ {1}\right)
$$
where  $\widehat{W}_1$  and  $\widehat{b}_1$  is defined as
$$
\left\{ \begin{array}{l} {\left[ \begin{array}{c} W _ {1} \\ b _ {1} ^ {\top} \end{array} \right] = \left[ X, \mathbf {1} _ {| S |} \right] ^ {\dagger} \sigma^ {- 1} \left(B _ {1}\right),} \\ B _ {l} = \left(\sigma^ {- 1} \left(B _ {l + 1}\right) - \mathbf {1} _ {| S |} b _ {l + 1} ^ {\top}\right) W _ {l + 1} ^ {\dagger}, \forall l \in [ 1, k - 2 ] \\ B _ {k - 1} = \left(\psi_ {i n v} (\pi) - \mathbf {1} _ {| S |} b _ {L} ^ {\top}\right) W _ {L} ^ {\dagger} \end{array} \right. \tag {16}
$$
where we use  $A^\dagger$  to denote the Moore-Penrose inverse of a matrix  $A$ . If  $A$  has full column rank, then we have  $A^\dagger A = I$ . If  $A$  has full row rank, we have  $AA^\dagger = I$ . We can easily see that the defined  $h$  operator is continuous as it is a composition of continuous operators.
Assumption 3, and Condition 1 imply that the matrices  $W_{2}, \ldots, W_{L}$  all have full column rank, which means  $W_{l}^{\dagger}W_{l} = I$ . We also know that  $[X, \mathbf{1}_{|\mathcal{S}|}]$  has full row rank by our assumption that  $X$  has full row rank, which means  $[X, \mathbf{1}_{|\mathcal{S}|}][X, \mathbf{1}_{|\mathcal{S}|}]^{\dagger} = I$ . Therefore, we can layerwise invert (16) and verify that
$$
F _ {L} ^ {h ((W _ {l}, b _ {l}) _ {l = 2} ^ {L}, \pi), (W _ {l}, b _ {l}) _ {l = 2} ^ {L}} = \pi .
$$
For every layer  $l = 2,\dots,L$ , we define the operator  $G_{l}:\mathbb{R}^{|S|\times n_{l - 1}}\to \mathbb{R}^{|S|\times n_{l}}$
$$
G _ {l} (Y) = \left\{ \begin{array}{l l} \sigma \left(Y W _ {l} ^ {\star} + \mathbf {1} _ {| \mathcal {S} |} \left(b _ {l} ^ {\star}\right) ^ {\top}\right) & l \in [ 2, L - 1 ] \\ \psi \left(Y W _ {L} ^ {\star} + \mathbf {1} _ {| \mathcal {S} |} \left(b _ {L} ^ {\star}\right) ^ {\top}\right) & l = L \end{array} \right.
$$
We also define the operator  $H: \mathbb{R}^{(d + 1) \times n_1} \to \mathbb{R}^{|S| \times n_1}$
$$
H (Y) = \sigma \left([ X, \mathbf {1} _ {| S |} ] Y\right).
$$
To show the continuous path claimed in Lemma 3 exists, it suffices to show that the set  $\{(W_1,b_1): F_L^{(W_1,b_1),(W_l^\star,b_l^\star)_{l=2}^L} = F_L^{\theta^\star}\}$  is connected, which is equivalent to showing that the set  $f^{-1}(F_L^{\theta^\star})$  is connected where  $f$  is defined as
$$
f ([ W _ {1} ^ {\top}, b _ {1} ] ^ {\top}) = G _ {L} \circ \dots \circ G _ {2} \circ H ([ W _ {1} ^ {\top}, b _ {1} ] ^ {\top}).
$$
Note that the definition of  $f$  implies
$$
f ^ {- 1} (\pi) = H ^ {- 1} \circ G _ {2} ^ {- 1} \circ \dots \circ G _ {L} ^ {- 1} (\pi). \tag {17}
$$
Note that  $G_{l}^{-1}$  is
$$
G _ {l} ^ {- 1} (F) = \left\{ \begin{array}{l} \big (\psi_ {i n v} (F) + \{C \mid C _ {i, j} = C _ {i, j ^ {\prime}} \forall i, j \neq j ^ {\prime} \} - \mathbf {1} _ {N} b _ {L} ^ {T} \big) (W _ {L} ^ {\star}) ^ {\dagger} + \{B \mid B W _ {L} ^ {\star} = 0 \}, l = L \\ \big (\sigma^ {- 1} (F) - \mathbf {1} _ {N} b _ {l} ^ {\star} \big) (W _ {l} ^ {\star}) ^ {\dagger} + \{B \mid B W _ {l} ^ {\star} = 0 \}, l = 2,..., L - 1 \end{array} \right.
$$
It is easy to verify that  $\{C\mid C_{i,j} = C_{i,j^{\prime}}\forall i,j\neq j^{\prime}\}$  and  $\{B\mid BW_l^\star = 0\}$  for all  $l = 2,\ldots ,L$  . Then, Lemma 4 implies that  $G_{l}^{-1}(F)$  is a connected set for all  $F$
Similarly,  $H^{-1}(F) = \left[X,\mathbf{1}_{|S|}\right]^{\dagger}\sigma^{-1}(F) + \left\{B\mid \left[X,\mathbf{1}_{|S|}\right]B = 0\right\}$  is also a connected set for all  $F$ . Therefore, from (17) we know that  $f^{-1}(F)$  is a connected set for any  $F$ , which concludes the proof of Lemma 3.
C.4 Proof of Lemma 4
To show that the product of the two connected sets are connected, we consider any  $x,y\in \{ab\mid a\in \mathcal{A},b\in \mathcal{B}\}$ . Obviously, there exist  $a_{x},a_{y}\in \mathcal{A}$  and  $b_{x},b_{y}\in \mathcal{B}$  such that  $x = a_{x}b_{x},y = a_{y}b_{y}$ . The connectedness of  $\mathcal{A}$  and  $\mathcal{B}$  implies that there exists continuous path maps  $p_{\mathcal{A}}:[0,1]\to \mathcal{A}$  and  $p_{\mathcal{B}}:[0,1]\to \mathcal{B}$  such that  $p_{\mathcal{A}}(0) = a_x,p_{\mathcal{A}}(1) = a_y,p_{\mathcal{B}}(0) = b_x,p_{\mathcal{B}}(1) = b_y$ . Define  $p(\alpha) = p_{\mathcal{A}}p_{\mathcal{B}}$  for  $\alpha \in [0,1]$ . It is obvious that  $p(\alpha)\in \{ab\mid a\in \mathcal{A},b\in \mathcal{B}\}$  for all  $\alpha$ . Since the product of continuous maps is still continuous,  $p:[0,1]\rightarrow \{ab\mid a\in \mathcal{A},b\in \mathcal{B}\}$  is a continuous path map satisfying  $p(0) = x$  and  $p(1) = y$ . This implies that the set  $\{ab\mid a\in \mathcal{A},b\in \mathcal{B}\}$  is a connected set.
A similar argument can be used to show that the sum of two connected sets is connected.
C.5 Proof of Lemma 5
Define  $\tilde{F}_L((W_l, b_l)_{l=1}^L)$  as the output of the final layer before the softmax activation
$$
\tilde {F} _ {L} ^ {(W _ {l}, b _ {l}) _ {l = 1} ^ {L}} = F _ {L - 1} W _ {L} + \mathbf {1} _ {| S |} b _ {L} ^ {\top}.
$$
Then, existing results in the literature (such as Lemma 3.4 of Nguyen [2019]) show that for any  $\theta \in \Omega$ , there exists a continuous path map  $p:[0,1]\to \Omega$  such that  $p(0) = \theta$ ,  $p(1) = \theta^{\star}\in \Omega^{\star}$ , and  $\tilde{F}_L(p(\alpha)) = \tilde{F}_L(\theta)$  for all  $\alpha \in [0,1]$ . This leads to our claim.
C.6 Proof of Lemma 6
This lemma is adapted from Theorem 4 of Evard and Jafari [1994].
D Convexity of Optimization Program (11)
In this section, we show that (11) is a convex optimization program. First, we note that
$$
J _ {r ^ {\prime}} (\pi) = \sum_ {s, a} r ^ {\prime} (s, a) \widehat {\mu} _ {\pi} = \widehat {\mu} _ {\pi} ^ {\top} r ^ {\prime},
$$
which means that the objective function is linear in the reward.
The constraint set is obvious closed. It is also bounded as the reward  $r(s, a) \in [0, U_r]$ . To prove the constraint set is convex, we need to show that for any  $r_1, r_2$  such that  $\text{Attack}(r_1, \pi_{\dagger}, \epsilon_{\dagger}) = r_{\dagger}$  and  $\text{Attack}(r_2, \pi_{\dagger}, \epsilon_{\dagger}) = r_{\dagger}$ , we have
$$
\operatorname {A t t a c k} \left(\alpha r _ {1} + (1 - \alpha) r _ {2}, \pi_ {\dagger}, \epsilon_ {\dagger}\right) = r _ {\dagger}, \quad \forall \alpha \in [ 0, 1 ]. \tag {18}
$$
By the optimality condition of (9),  $r_{\dagger}$  being the optimal poisoned reward for true reward  $r_1$  and  $r_2$  is equivalent to
$$
\langle r - r _ {\dagger}, r _ {1} - r _ {\dagger} \rangle \leq 0 \quad \text {a n d} \quad \langle r - r _ {\dagger}, r _ {2} - r _ {\dagger} \rangle \leq 0
$$
for all  $r$  such that  $J_{r}(\pi_{\dagger}) \geq J_{r}(\pi) + \epsilon_{\dagger}$ ,  $\forall \pi \in \Pi^{\mathrm{det}} \backslash \pi_{\dagger}$ . By taking the convex combination of these two inequalities, we have for any  $\alpha \in [0,1]$
$$
\langle r - r _ {\dagger}, \alpha r _ {1} + (1 - \alpha) r _ {2} - r _ {\dagger} \rangle \leq 0 \tag {19}
$$
for all  $r$  such that  $J_{r}(\pi_{\dagger}) \geq J_{r}(\pi) + \epsilon_{\dagger}$ ,  $\forall \pi \in \Pi^{\mathrm{det}} \backslash \pi_{\dagger}$ . Again by the optimality condition of (9), (19) is equivalent to (18).
At this point, we have shown that (11) has a linear objective function and a convex (and also compact) constraint set. As a result, the optimization program is convex.