SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By composition with affine functions, both  $g(x) = f(\langle u,x\rangle ,\langle v,x\rangle)$  and  $h(x) = f(1 - \langle u,x\rangle ,\langle v,x\rangle)$  are convex.
Theorem 3.2 (Strongly observable graphs). Suppose that the feedback graph  $G_{t}$  is deterministic and strongly observable with independence number no more than  $\alpha$ . Then Algorithm 1 guarantees that
$$
\overline {{\mathsf {d e c}}} _ {\gamma} (p _ {t}; \widehat {f} _ {t}, x _ {t}, G _ {t}) \leq \mathcal {O} \bigg (\frac {\alpha \log (K \gamma)}{\gamma} \bigg).
$$
Proof. For conciseness, we omit the subscript  $t$ . Direct calculation shows that for all  $a^{\star} \in [K]$ ,
$$
\begin{array}{l} \mathbb {E} _ {a \sim p} \left[ f ^ {\star} (x, a) - f ^ {\star} (x, a ^ {\star}) - \frac {\gamma}{4} \sum_ {a ^ {\prime} \in N ^ {\mathrm {i n}} (G, a)} (\widehat {f} (x, a ^ {\prime}) - f ^ {\star} (x, a ^ {\prime})) ^ {2} \right] \\ = \sum_ {a = 1} ^ {K} p _ {a} f ^ {\star} (x, a) - f ^ {\star} (x, a ^ {\star}) - \frac {\gamma}{4} \sum_ {a = 1} ^ {K} W _ {a} \left(\widehat {f} (x, a) - f ^ {\star} (x, a)\right) ^ {2}, \\ \end{array}
$$
where  $W_{a} = \sum_{a^{\prime}\in N^{\mathrm{in}}(G,a)}p_{a^{\prime}}$ . Therefore, taking the gradient over  $f^{*}(x,\cdot)$  and we know that
$$
\begin{array}{l} \sup  _ {f ^ {\star} \in (\mathcal {X} \times [ K ] \mapsto \mathbb {R})} \left[ \sum_ {a = 1} ^ {K} p _ {a} f ^ {\star} (x, a) - f ^ {\star} (x, a ^ {\star}) - \frac {\gamma}{4} \sum_ {a = 1} ^ {K} W _ {a} \left(\widehat {f} (x, a) - f ^ {\star} (x, a)\right) ^ {2} \right] \\ = \sum_ {a = 1} ^ {K} p _ {a} \widehat {f} (x, a) - \widehat {f} (x, a ^ {\star}) + \frac {1}{\gamma} \| p - e _ {a ^ {\star}} \| _ {\operatorname {d i a g} (W) ^ {- 1}} ^ {2}. \\ \end{array}
$$
Then, denote  $\widehat{f} \in \mathbb{R}^K$  to be  $\widehat{f}(x, \cdot)$  and consider the following minimax form:
$$
\begin{array}{l} \inf  _ {p \in \Delta (K)} \sup  _ {a ^ {\star} \in \mathcal {A}} \left\{\sum_ {a = 1} ^ {K} p _ {a} \widehat {f} (x, a) - \widehat {f} (x, a ^ {\star}) + \frac {1}{\gamma} \| p - e _ {a ^ {\star}} \| _ {\operatorname {d i a g} (W) ^ {- 1}} ^ {2} \right\} \\ = \min  _ {p \in \Delta (K)} \max  _ {a ^ {\star} \in \mathcal {A}} \left\{\sum_ {a = 1} ^ {K} p _ {a} \widehat {f} (x, a) - \widehat {f} (x, a ^ {\star}) + \frac {1}{\gamma} \sum_ {a \neq a ^ {\star}} \frac {p _ {a} ^ {2}}{W _ {a}} + \frac {1}{\gamma} \frac {(1 - p _ {a ^ {\star}}) ^ {2}}{W _ {a ^ {\star}}} \right\} (10) \\ = \min  _ {p \in \Delta_ {K}} \max  _ {q \in \Delta_ {K}} \left\{\sum_ {a = 1} ^ {K} \left(p _ {a} - q _ {a}\right) \widehat {f} _ {a} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} (1 - q _ {a})}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - p _ {a}) ^ {2}}{\gamma W _ {a}} \right\} (11) \\ = \max  _ {q \in \Delta_ {K}} \min  _ {p \in \Delta_ {K}} \left\{\sum_ {a = 1} ^ {K} \left(p _ {a} - q _ {a}\right) \widehat {f} _ {a} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} (1 - q _ {a})}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - p _ {a}) ^ {2}}{\gamma W _ {a}} \right\}, (12) \\ \end{array}
$$
where the last equality is due to Sion's minimax theorem and the fact that Eq. (10) is convex in  $p \in \Delta(K)$  by applying Lemma A.1 with  $u = e_a$  and  $v = g_a$  for each  $a \in [K]$ , where  $g_a \in \{0, 1\}^K$  is defined as  $g_{a,i} = \mathbb{1}\{(i,a) \in E\}$ ,  $G = ([K], E), \forall i \in [K]$ .
Choose  $p_a = (1 - \frac{1}{\gamma})q_a + \frac{1}{\gamma K}$  for all  $a \in [K]$ . Let  $S$  be the set of nodes in  $[K]$  that have a self-loop. Then we can upper bound the value above as follows:
$$
\max  _ {q \in \Delta (K)} \min  _ {p \in \Delta (K)} \left\{\sum_ {a = 1} ^ {K} \left(p _ {a} - q _ {a}\right) \widehat {f} _ {a} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} \left(1 - q _ {a}\right)}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} \left(1 - p _ {a}\right) ^ {2}}{\gamma W _ {a}} \right\}
$$
$$
\begin{array}{l} \leq \max  _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {\left((1 - \frac {1}{\gamma}) q _ {a} + \frac {1}{\gamma K}\right) ^ {2} (1 - q _ {a}) + q _ {a} \left(1 - (1 - \frac {1}{\gamma}) q _ {a} - \frac {1}{\gamma K}\right) ^ {2}}{W _ {a}} \right\} \\ \leq \max  _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {2 \left((1 - \frac {1}{\gamma}) ^ {2} q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}}\right) (1 - q _ {a}) + q _ {a} \left(1 - (1 - \frac {1}{\gamma}) q _ {a}\right) ^ {2}}{W _ {a}} \right\} \\ \leq \max _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {2}{\gamma^ {2}} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {2 q _ {a} ^ {2} (1 - q _ {a}) + 2 q _ {a} (1 - q _ {a}) ^ {2} + \frac {2 q _ {a} ^ {3}}{\gamma^ {2}}}{W _ {a}} \right\} \\ \left(W _ {a} = \sum_ {j \in N ^ {\text {i n}} (G, a)} p _ {j} \geq \frac {1}{\gamma K} \text {f o r a l l} a \in [ K ]\right) \\ \leq \max  _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {2}{\gamma^ {2}} + \frac {2}{\gamma} \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} + \frac {2}{\gamma^ {3}} \sum_ {a = 1} ^ {K} \frac {q _ {a} ^ {3}}{W _ {a}} \right\} \\ = \max  _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {2}{\gamma^ {2}} + \frac {2}{\gamma} \sum_ {a = 1} ^ {K} \frac {q _ {a} \left(1 - q _ {a}\right)}{W _ {a}} + \frac {2}{\gamma^ {3}} \sum_ {a \in S} \frac {q _ {a} ^ {3}}{W _ {a}} + \frac {2}{\gamma^ {3}} \sum_ {a \notin S} \frac {q _ {a} ^ {3}}{W _ {a}} \right\} (13) \\ \leq \max  _ {q \in \Delta (K)} \left\{\frac {2}{\gamma} + \frac {2}{\gamma^ {2}} + \frac {2}{\gamma} \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} + \frac {2}{\gamma^ {3}} \sum_ {a \in S} q _ {a} ^ {2} + \frac {2}{\gamma^ {3}} \sum_ {a \notin S} \frac {q _ {a} ^ {3}}{\frac {K - 1}{\gamma K}} \right\} \\ (\text {i f} a \notin S, W _ {a} = 1 - p _ {a} \geq \frac {K - 1}{\gamma K}) \\ \leq \max  _ {q \in \Delta (K)} \left\{\frac {8}{\gamma} + \frac {2}{\gamma} \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} \right\}. (K≥2) \\ \end{array}
$$
Next we bound  $\frac{2q_a(1 - q_a)}{W_a}$  for each  $a\in [K]$ . If  $a\in [K]\backslash S$ , we have  $W_{a} = 1 - p_{a}$  and
$$
\frac {2 q _ {a} (1 - q _ {a})}{W _ {a}} \leq \frac {2 q _ {a} (1 - q _ {a})}{1 - (1 - \frac {1}{\gamma}) q _ {a} - \frac {1}{\gamma K}} \leq \frac {2 q _ {a} (1 - q _ {a})}{(1 - \frac {1}{\gamma}) (1 - q _ {a}) + \frac {K - 1}{\gamma K}} \leq \frac {2}{1 - \frac {1}{\gamma}} q _ {a} \leq 4 q _ {a}. \tag {14}
$$
If  $a \in S$ , we know that
$$
\begin{array}{l} \sum_ {a \in S} \frac {2 q _ {a} (1 - q _ {a})}{W _ {a}} \leq \sum_ {a \in S} \frac {2 q _ {a} (1 - q _ {a})}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} ((1 - \frac {1}{\gamma}) q _ {j} + \frac {1}{\gamma K})} \\ \leq \frac {\gamma}{\gamma - 1} \sum_ {a \in S} \frac {2 ((1 - \frac {1}{\gamma}) q _ {a} + \frac {1}{\gamma K}) (1 - q _ {a})}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} ((1 - \frac {1}{\gamma}) q _ {j} + \frac {1}{\gamma K})} \\ \leq 4 \sum_ {a \in S} \frac {\left((1 - \frac {1}{\gamma}) q _ {a} + \frac {1}{\gamma K}\right)}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} \left((1 - \frac {1}{\gamma}) q _ {j} + \frac {1}{\gamma K}\right)} \leq \mathcal {O} (\alpha \log (K \gamma)), \tag {15} \\ \end{array}
$$
where the last inequality is due to Lemma 5 in Alon et al. [2015]. We include this lemma (Lemma E.1) for completeness. Combining all the above inequalities, we obtain that
$$
\begin{array}{l} \inf  _ {p \in \Delta (K)} \sup  _ {a ^ {\star} \in \mathcal {A}} \left\{\sum_ {a = 1} ^ {K} p _ {a} \widehat {f} (x, a) - \widehat {f} (x, a ^ {\star}) + \frac {1}{\gamma} \| p - e _ {a ^ {\star}} \| _ {\operatorname {d i a g} (W) ^ {- 1}} ^ {2} \right\} \\ = \max  _ {q \in \Delta (K)} \min  _ {p \in \Delta (K)} \left\{\sum_ {a = 1} ^ {K} \left(p _ {a} - q _ {a}\right) \widehat {f} _ {a} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} (1 - q _ {a})}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - p _ {a}) ^ {2}}{\gamma W _ {a}} \right\} \\ \leq \max  _ {q \in \Delta (K)} \left\{\frac {8}{\gamma} + \frac {2}{\gamma} \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} \right\} \leq \mathcal {O} \left(\frac {\alpha \log (K \gamma)}{\gamma}\right). \\ \end{array}
$$
Proof. Similar to the strongly observable graphs setting, for weakly observable graphs, we know that
$$
\begin{array}{l} \overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \\ = \max  _ {q \in \Delta_ {K}} \min  _ {p \in \Delta_ {K}} \left\{\sum_ {a = 1} ^ {K} \left(p _ {a} - q _ {a}\right) \widehat {f} _ {a} + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} (1 - q _ {a})}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - p _ {a}) ^ {2}}{\gamma W _ {a}} \right\}. \tag {16} \\ \end{array}
$$
Choose  $p_a = (1 - \frac{1}{\gamma} - \eta d) q_a + \frac{1}{\gamma K} + \eta \mathbb{1}\{a \in D\}$  where  $D$  with  $|D| = d$  is the minimum weak dominating set of  $G$  and  $0 < \eta \leq \frac{1}{4d}$  is some parameter to be chosen later. Substituting the form of  $p$  to Eq. (16) and using the fact that  $|\widehat{f_a}| \leq 1$  for all  $a \in [K]$ , we can obtain that
$$
\begin{array}{l} \overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \\ \leq \max _ {q \in \Delta_ {K}} \left\{\frac {2}{\gamma} + \eta d + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2} (1 - q _ {a})}{W _ {a}} + \sum_ {a = 1} ^ {K} \frac {q _ {a} (1 - p _ {a}) ^ {2}}{\gamma W _ {a}} \right\}. \\ \end{array}
$$
Then we can upper bound the value above as follows:
$$
\begin{array}{l} \overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \\ \leq \max _ {q \in \Delta_ {K}} \left\{\frac {2}{\gamma} + \eta d + \frac {1}{\gamma} \sum_ {a = 1} ^ {K} \frac {\left((1 - \frac {1}{\gamma} - \eta d) q _ {a} + \frac {1}{\gamma K} + \eta \mathbb {1} \{a \in D \}\right) ^ {2} (1 - q _ {a})}{W _ {a}} \right. \\ \left. + \sum_ {a = 1} ^ {K} \frac {q _ {a} \left(1 - \left(1 - \frac {1}{\gamma} - \eta d\right) q _ {a}\right) ^ {2}}{W _ {a}} \right\} \\ \leq \max  _ {q \in \Delta_ {K}} \left\{\frac {2}{\gamma} + \eta d + \frac {1}{\gamma} \sum_ {a \notin D} \frac {\left(q _ {a} + \frac {1}{\gamma K}\right) ^ {2} (1 - q _ {a}) + q _ {a} \left((1 - q _ {a}) + \frac {1}{\gamma} q _ {a} + \eta d q _ {a}\right) ^ {2}}{W _ {a}} \right. \\ \left. + \frac {1}{\gamma} \sum_ {a \in D} \frac {\left(q _ {a} + \frac {1}{\gamma K} + \eta\right) ^ {2} (1 - q _ {a}) + q _ {a} \left(\left(1 - q _ {a}\right) + \frac {1}{\gamma} q _ {a} + \eta d q _ {a}\right) ^ {2}}{W _ {a}} \right\} \\ \leq \max  _ {q \in \Delta_ {K}} \left\{\frac {2}{\gamma} + \eta d + \frac {1}{\gamma} \sum_ {a \notin D} \frac {2 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}} \right. \\ \left. + \frac {1}{\gamma} \sum_ {a \in D} \frac {3 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}} + \eta^ {2}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}} \right\}. \tag {17} \\ \end{array}
$$
Now consider  $a \in D$ . If  $a \in S$ , then we know that  $W_{a} \geq \eta$ ; Otherwise, we know that this node can be observed by at least one node in  $D$ , meaning that  $W_{a} \geq \eta$ . Combining the two cases above, we know that
$$
\frac {1}{\gamma} \sum_ {a \in D} \frac {3 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}} + \eta^ {2}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {1}{\gamma^ {2}} q _ {a} ^ {2} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}}
$$
$$
\begin{array}{l} \leq \frac {3}{\eta \gamma} \sum_ {a \in D} \left[ \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}} + \eta^ {2}\right) (1 - q _ {a}) + q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {1}{\gamma^ {2}} q _ {a} ^ {2} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right) \right] \\ \leq \frac {3}{\eta \gamma} \sum_ {a \in D} \left[ q _ {a} - q _ {a} ^ {2} + \frac {1}{\gamma^ {2}} q _ {a} ^ {3} + \eta^ {2} d ^ {2} q _ {a} ^ {3} + \frac {1}{\gamma^ {2} K ^ {2}} + \eta^ {2} \right] \\ \leq \mathcal {O} \left(\frac {1}{\eta \gamma} + \frac {d \eta}{\gamma} + \frac {1}{\eta \gamma^ {3} K}\right) \quad (\eta \leq \frac {1}{4 d} \text {a n d} \gamma \geq 1 6 d) \\ \leq \mathcal {O} \left(\frac {1}{\eta \gamma}\right), \tag {18} \\ \end{array}
$$
where the last inequality is because  $\eta \leq \frac{1}{4d}$  and  $\gamma \geq 16d$ . Consider  $a \notin D$ . Let  $S_0$  be the set of nodes which either have a self-loop or can be observed by all the other node. Recall that  $S$  represents the set of nodes with a self-loop. Then similar to the derivation of Eq. (13), we know that for  $a \in S_0$ ,
$$
\begin{array}{l} \frac {1}{\gamma} \sum_ {a \notin D, a \in S _ {0}} \frac {2 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}} \\ \leq \frac {1}{\gamma} \sum_ {a \notin D, a \in S _ {0}} \frac {2 q _ {a} ^ {2} (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}} + \mathcal {O} \left(\frac {1}{\gamma^ {2}} + \frac {1}{\eta \gamma^ {3} K}\right) \\ \left(W _ {a} \geq \frac {1}{\gamma K} \text {i f} a \in S \text {a n d} W _ {a} \geq \eta \text {i f} a \in [ K ] \backslash S\right) \\ \leq \mathcal {O} \left(\frac {1}{\gamma} \sum_ {a \in S _ {0}, a \notin D} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} + \frac {1}{\gamma^ {3}} \sum_ {a \in S, a \notin D} q _ {a} ^ {2} + \frac {1}{\gamma^ {3}} \sum_ {a \in S _ {0}, a \notin D \cup S} \frac {q _ {a} ^ {3}}{\frac {K - 1}{\gamma K}} + \frac {1}{\gamma^ {2}} + \frac {1}{\eta \gamma^ {3} K}\right) \\ + \mathcal {O} \left(\frac {1}{\gamma} \sum_ {a \in S, a \notin D} \eta^ {2} d ^ {2} q _ {a} ^ {2} + \frac {1}{\gamma} \sum_ {a \in S _ {0}, a \notin D \cup S} \frac {\eta^ {2} d ^ {2} q _ {a} ^ {3}}{\eta}\right) \\ \left(\text {f o r} a \in S _ {0}, a \notin S, W _ {a} \geq \max  \left\{\frac {K - 1}{\gamma K}, \eta \right\}\right) \\ \leq \mathcal {O} \left(\frac {1}{\gamma} \sum_ {a \in S _ {0}, a \notin D} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} + \frac {1}{\eta \gamma}\right). \tag {19} \\ \end{array}
$$
For  $a \notin S_0$ , we know that  $W_a \geq \eta$ . Therefore,
$$
\begin{array}{l} \frac {1}{\gamma} \sum_ {a \notin D \cup S _ {0}} \frac {2 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \eta^ {2} d ^ {2} q _ {a} ^ {2}\right)}{W _ {a}} \\ \leq \frac {1}{\gamma \eta} \sum_ {a \notin D \cup S _ {0}} \left(2 \left(q _ {a} ^ {2} + \frac {1}{\gamma^ {2} K ^ {2}}\right) (1 - q _ {a}) + 3 q _ {a} \left((1 - q _ {a}) ^ {2} + \frac {q _ {a} ^ {2}}{\gamma^ {2}} + \frac {1}{1 6} q _ {a} ^ {2}\right)\right) \\ \leq \frac {1}{\gamma \eta} \sum_ {a \notin D \cup S _ {0}} \left(2 q _ {a} (1 - q _ {a}) + \frac {1}{\gamma^ {2} K ^ {2}} + \frac {2 q _ {a} ^ {3}}{\gamma^ {2}} + \frac {3}{1 6} q _ {a} ^ {3}\right) \\ \leq \mathcal {O} \left(\frac {1}{\gamma \eta}\right). \tag {20} \\ \end{array}
$$
Plugging Eq. (18), Eq. (19), and Eq. (20) to Eq. (17), we obtain that
$$
\overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \leq \mathcal {O} \left(\frac {1}{\gamma} + \eta d + \frac {1}{\gamma \eta} + \frac {1}{\gamma} \sum_ {a \in S _ {0}, a \notin D} \frac {q _ {a} \left(1 - q _ {a}\right)}{W _ {a}}\right) \tag {21}
$$
Consider the last term. If  $a \in S_0 \setminus S$ , similar to Eq. (14), we know that
$$
\frac {q _ {a} (1 - q _ {a})}{W _ {a}} \leq \frac {q _ {a} (1 - q _ {a})}{1 - (1 - \frac {1}{\gamma} - d \eta) q _ {a} - \frac {1}{\gamma K}} \leq \frac {q _ {a} (1 - q _ {a})}{(1 - \frac {1}{\gamma} - \eta d) (1 - q _ {a})} \leq \frac {1}{1 - \frac {1}{\gamma} - \eta d} q _ {a} \leq \mathcal {O} (q _ {a}),
$$
where the last inequality is due to  $\gamma \geq 16d$  and  $\eta \leq \frac{1}{4d}$ . If  $a \in S$ , similar to Eq. (15), we know that
$$
\begin{array}{l} \sum_ {a \in S} \frac {q _ {a} (1 - q _ {a})}{W _ {a}} \leq \sum_ {a \in S} \frac {q _ {a} (1 - q _ {a})}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} ((1 - \frac {1}{\gamma} - \eta d) q _ {j} + \frac {1}{\gamma K})} \\ \leq \frac {\gamma}{\gamma - 1 - \gamma \eta d} \sum_ {a \in S} \frac {((1 - \frac {1}{\gamma} - \eta d) q _ {a} + \frac {1}{\gamma K}) (1 - q _ {a})}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} ((1 - \frac {1}{\gamma} - \eta d) q _ {j} + \frac {1}{\gamma K})} \\ \leq 2 \sum_ {a \in S} \frac {\left((1 - \frac {1}{\gamma} - \eta d) q _ {a} + \frac {1}{\gamma K}\right)}{\sum_ {j \in N ^ {\mathrm {i n}} (G , a)} \left((1 - \frac {1}{\gamma} - \eta d) q _ {j} + \frac {1}{\gamma K}\right)} \quad (\gamma \geq 4, \eta \leq \frac {1}{4 d}) \\ \leq \mathcal {O} (\widetilde {\alpha} \log (K \gamma)), \tag {22} \\ \end{array}
$$
where the last inequality is again due to Lemma 5 in [Alon et al., 2015] and  $\widetilde{\alpha}$  is the independence number of the subgraph induced by nodes with self-loops in  $G$ . Plugging Eq. (22) to Eq. (21) gives
$$
\overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \leq \mathcal {O} \left(\eta d + \frac {1}{\gamma \eta} + \frac {\widetilde {\alpha} \log (K \gamma)}{\gamma}\right).
$$
Picking  $\eta = \sqrt{\frac{1}{\gamma d}} \leq \frac{1}{4d}$  proves the result.
Next, we prove Corollary 3.5 by combining Theorem 3.4 and Theorem 3.1.
Corollary 3.5. Suppose that  $G_{t}$  is deterministic, weakly observable, and has weak domination number no more than  $d$  for all  $t \in [T]$ . In addition, suppose that the independence number of the subgraph induced by nodes with self-loops in  $G_{t}$  is no more than  $\widetilde{\alpha}$  for all  $t \in [T]$ . Then, Algorithm 1 with  $\gamma = \max \{16d, \sqrt{\widetilde{\alpha} T / \mathbf{Reg}_{\mathbf{Sq}}}, d^{\frac{1}{3}} T^{\frac{2}{3}} \mathbf{Reg}_{\mathbf{Sq}}^{-\frac{2}{3}}\}$  guarantees that
$$
\mathbb {E} \left[ \mathbf {R e g} _ {\mathrm {C B}} \right] \leq \widetilde {\mathcal {O}} \left(d ^ {\frac {1}{3}} T ^ {\frac {2}{3}} \mathbf {R e g} _ {\mathrm {S q}} ^ {\frac {1}{3}} + \sqrt {\widetilde {\alpha} T \mathbf {R e g} _ {\mathrm {S q}}}\right).
$$
Proof. Combining Eq. (8), Eq. (9) and Theorem 3.4, we can bound  $\mathbf{Reg}_{\mathbb{C}\mathbb{B}}$  as follows:
$$
\mathbb {E} [ \mathbf {R e g} _ {\mathrm {C B}} ] \leq \mathcal {O} \left(\sqrt {\frac {d}{\gamma}} T + \frac {\widetilde {\alpha} T \log (K \gamma)}{\gamma} + \gamma \mathbf {R e g} _ {\mathrm {C B}}\right).
$$
Picking  $\gamma = \max \left\{16d, \sqrt{\widetilde{\alpha}T / \mathbf{Reg}_{\mathbf{Sq}}}, d^{\frac{1}{3}}T^{\frac{2}{3}}\mathbf{Reg}_{\mathbf{Sq}}^{-\frac{2}{3}}\right\}$  finishes the proof.
A.3 Python Solution to Eq. (5)
This particular formulation multiplies both sides of the constraint in Eq. (5) by  $\sqrt{\gamma}$  while scaling the objective by  $\sqrt{\gamma}$ . While mathematically equivalent to Eq. (5), empirically it has superior numerical stability for large  $\gamma$ . For additional stability, when using this routine we recommend subtracting off the minimum value from  $\widehat{f}$ , which is equivalent to making the substitutions  $\sqrt{\gamma} \widehat{f} \gets \sqrt{\gamma} \widehat{f} - \sqrt{\gamma} \min_{a} \widehat{f}_{a}$  and  $z \gets z + \sqrt{\gamma} \min_{a} \widehat{f}_{a}$  and then exploiting the  $1^{\top} p = 1$  constraint.
Proof. Denote  $f^{\star} = f^{\star}(x,\cdot)\in \mathbb{R}^{K}$ . Note that according to the definition of  $G$ , we know that  $(G^{\top}p)_{i}$  denotes the probability that action  $i$ 's loss is revealed when the selected action  $a$  is sampled from distribution  $p$ . Then, we know that
$$
\begin{array}{l} \overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) \\ = \sup_{\substack{a^{\star}\in [K]\\ f^{\star}\in \mathbb{R}^{K}}}\mathbb{E}_{a_{t}\sim p}\left[f_{a_{t}}^{\star} - f_{a^{\star}}^{\star} - \frac{\gamma}{4}\mathbb{E}_{A\sim G(\cdot |a_{t})}\left[\sum_{a\in A}\left(\widehat{f}_{a} - f_{a}^{\star}\right)^{2}\right]\right] \\ = \sup_{\substack{a^{\star}\in [K]\\ f^{\star}\in \mathbb{R}^{K}}}(p - e_{a^{\star}})^{\top}f^{\star} - \frac{\gamma}{4}\sum_{a\in [K]}\| \widehat{f} -f^{\star}\|_{\mathrm{diag}(G^{\top}p)}^{2} \\ = \sup  _ {a ^ {\star} \in [ K ]} (p - e _ {a ^ {\star}}) ^ {\top} \widehat {f} + \frac {1}{\gamma} \| p - e _ {a ^ {\star}} \| _ {\operatorname {d i a g} (G ^ {\top} p) ^ {- 1}} ^ {2} \quad \left(G ^ {\top} p \succ 0\right) \\ = p ^ {\top} \widehat {f} + \max  _ {a ^ {\star} \in [ K ]} \left\{\frac {1}{\gamma} \| p - e _ {a ^ {\star}} \| _ {\operatorname {d i a g} (G ^ {\top} p) ^ {- 1}} ^ {2} - e _ {a ^ {\star}} ^ {\top} \widehat {f} \right\}, \\ \end{array}
$$
where the third equality is by picking  $f^{\star}$  to be the maximizer and introduces a constraint. Therefore, the minimization problem  $\min_{p\in \Delta (K)}\overline{\mathrm{dec}}_{\gamma}(p;\widehat{f},x,G)$  can be written as the following constrained optimization by variable substitution:
$$
\min  _ {p \in \Delta (K), z} \qquad p ^ {\top} \widehat {f} + z
$$
$$
\text {s u b j e c t} \quad \forall a \in [ K ]: \frac {1}{\gamma} \| p - e _ {a} \| _ {\operatorname {d i a g} (G ^ {\top} p) ^ {- 1}} ^ {2} \leq e _ {a} ^ {\top} \widehat {f} + z,
$$
$$
G ^ {\top} p \succ 0.
$$
The convexity of the constraints follows from Lemma A.1.
B Omitted Details in Section 4
In this section, we provide proofs for Section 4. We define  $W_{a} \coloneqq \sum_{a^{\prime} \in N^{\mathrm{in}}(G,a)} p_{a^{\prime}}$  to be the probability that the loss of action  $a$  is revealed when selecting an action from distribution  $p$ . Let  $\widehat{f} = \widehat{f}(x,\cdot) \in \mathbb{R}^{K}$  and  $f = f(x,\cdot) \in \mathbb{R}^{K}$ . Direct calculation shows that for any  $a^{\star} \in [K]$ ,
$$
f ^ {\star} = \operatorname * {a r g m a x} _ {f \in \mathbb {R} ^ {K}} \mathbb {E} _ {a \sim p} \left[ f (x, a) - f (x, a ^ {\star}) - \frac {\gamma}{4} \cdot \sum_ {a ^ {\prime} \in N ^ {\mathrm {i n}} (G, a)} (\widehat {f} _ {t} (x, a ^ {\prime}) - f (x, a ^ {\prime})) ^ {2} \right]
$$
$$
= \frac {2}{\gamma} \operatorname {d i a g} (W) ^ {- 1} \left(p - e _ {a ^ {*}}\right) + \widehat {f}.
$$
Therefore, substituting  $f^{\star}$  into Eq. (4), we obtain that
$$
\overline {{\operatorname {d e c}}} _ {\gamma} (p; \widehat {f}, x, G) = \max  _ {a ^ {\star} \in [ K ]} \left\{\frac {1}{\gamma} \left(\sum_ {a = 1} ^ {K} \frac {p _ {a} ^ {2}}{W _ {a}} + \frac {1 - 2 p _ {a ^ {\star}}}{W _ {a ^ {\star}}}\right) + \left\langle p - e _ {a ^ {\star}}, \widehat {f} \right\rangle \right\}. \tag {23}
$$
Without loss of generality, we assume the  $\min_{i\in [K]}\widehat{f}_i = 0$ . This is because shifting  $\widehat{f}$  by  $\min_{i\in [K]}\widehat{f}_i$  does not change the value of  $\left\langle p - e_{a^{\star}},\widehat{f}\right\rangle$ . In the following sections, we provide proofs showing that a certain closed-form of  $p$  leads to optimal  $\overline{\mathrm{dec}}_{\gamma}(p;\widehat{f},x,G)$  up to constant factors for several specific types of feedback graphs, respectively.
B.1 Cops-and-Robbers Graph
Proof. We use the following notation for convenience:  $p_1 \coloneqq p_{a_1}$ ,  $p_2 \coloneqq p_{a_2}$ ,  $\widehat{f}_1 \coloneqq \widehat{f}_{a_1} = 0$ ,  $\widehat{f}_2 \coloneqq \widehat{f}_{a_2}$ . For the cops-and-robbers graph and closed-form solution  $p$  in Eq. (6), Eq. (23) becomes:
$$
\overline {{\mathsf {d e c}}} _ {\gamma} (p; \widehat {f}, x, G _ {\mathrm {C R}}) = \max _ {a ^ {\star} \in [ K ]} \left\{\frac {1}{\gamma} \left(\frac {p _ {1} ^ {2}}{1 - p _ {1}} + \frac {(1 - p _ {1}) ^ {2}}{p _ {1}} + \frac {1 - 2 p _ {a ^ {\star}}}{W _ {a ^ {\star}}}\right) + \left\langle p - e _ {a ^ {\star}}, \widehat {f} \right\rangle \right\}.
$$
If  $a^{\star}\neq a_{1}$  and  $a^\star \neq a_2$  , we know that