SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\Delta_ {t} := - \sum_ {v \in \mathcal {B} ^ {\prime}} [   [ \mathcal {Q} (y, v) \neq \emptyset ]   ] \ln (w _ {t} (v, \mathcal {Q} (y, v)))
$$
Choose some arbitrary trial  $t \in [T]$ . From here until we say otherwise all probabilities and expectations (i.e. whenever we use  $\mathbb{P}[\cdot]$  or  $\mathbb{E}[\cdot]$ ) are implicitly conditional on the state of the algorithm at the start of trial  $t$ . Note first that we have:
$$
\Delta_ {t} - \Delta_ {t + 1} = \sum_ {v \in \mathcal {B} ^ {\prime}} [   [ \mathcal {Q} (y, v) \neq \emptyset ]   ] \ln \left(\frac {w _ {t + 1} (v , \mathcal {Q} (y , v))}{w _ {t} (v , \mathcal {Q} (y , v))}\right) \tag {17}
$$
For all  $j \in [\log(K)] \cup \{0\}$  let  $\gamma_{t,j}$  be the ancestor (in  $\mathcal{B}$ ) of  $y(x_t)$  at depth  $j$ . Note that for all  $v \in \mathcal{X} \setminus \{\gamma_{t,j} \mid j \in [\log(K)] \cup \{0\}\}$  we have  $y(x_t) \notin \Downarrow(v)$  so that  $x_t \notin \mathcal{Q}(y, v)$  and hence, directly from the CANPROP algorithm, we have  $w_{t+1}(v, \mathcal{Q}(y, v)) = w_t(v, \mathcal{Q}(y, v))$ . By Equation (17) and the fact that  $\mathcal{Q}(y, v) \neq \emptyset$  for all ancestors  $v$  of  $y(x_t)$  this implies that:
$$
\Delta_ {t} - \Delta_ {t + 1} = \sum_ {j \in [ \log (K) ]} \ln \left(\frac {w _ {t + 1} \left(\gamma_ {t , j} , \mathcal {Q} \left(y , \gamma_ {t , j}\right)\right)}{w _ {t} \left(\gamma_ {t , j} , \mathcal {Q} \left(y , \gamma_ {t , j}\right)\right)}\right) \tag {18}
$$
For all  $j \in [\log(K)]$  define:
$$
\lambda_ {t, j} := \ln \left(\frac {w _ {t + 1} (\gamma_ {t , j} , \mathcal {Q} (y , \gamma_ {t , j}))}{w _ {t} (\gamma_ {t , j} , \mathcal {Q} (y , \gamma_ {t , j}))}\right)
$$
$$
\epsilon_ {t, j} := \mathbb {E} \left[ \ln \left(\psi_ {t, j}\right) \mid \gamma_ {t, j} \in \mathcal {P} _ {t} \right]
$$
Now choose some arbitrary  $j \in [\log(K)]$ . If  $\gamma_{t,(j-1)} \in \mathcal{P}_t$  then  $\gamma_{t,(j-1)} = v_{t,(j-1)}$  so  $\uparrow(\gamma_{t,j}) = v_{t,(j-1)}$  and hence, since  $x_t \in \mathcal{Q}(y, \gamma_{t,j})$ , we have  $\lambda_{t,j} = \ln(\beta_t(\gamma_{t,j}))$ . By definition of  $\beta_t(\gamma_{t,j})$  this means that:
$$
\mathbb {E} [ \lambda_ {t, j} \mid \gamma_ {t, j} \in \mathcal {P} _ {t}, \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} ] = \epsilon_ {t, j} - \mathbb {E} [ \ln (\psi_ {t, (j - 1)}) \mid \gamma_ {t, j} \in \mathcal {P} _ {t}, \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} ]
$$
and that:
$$
\mathbb {E} \left[ \lambda_ {t, j} \mid \gamma_ {t, j} \notin \mathcal {P} _ {t}, \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} \right] = - \mathbb {E} \left[ \ln \left(\psi_ {t, (j - 1)}\right) \mid \gamma_ {t, j} \notin \mathcal {P} _ {t}, \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} \right]
$$
Multiplying these two equations by  $\mathbb{P}[\gamma_{t,j}\in \mathcal{P}_t\mid \gamma_{t,(j - 1)}\in \mathcal{P}_t]$  and  $\mathbb{P}[\gamma_{t,j}\notin \mathcal{P}_t\mid \gamma_{t,(j - 1)}\in \mathcal{P}_t]$  respectively, and summing them together, then gives us:
$$
\mathbb {E} [ \lambda_ {t, j} \mid \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} ] = \mathbb {P} [ \gamma_ {t, j} \in \mathcal {P} _ {t} \mid \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} ] \epsilon_ {t, j} - \mathbb {E} [ \ln (\psi_ {t, (j - 1)} \mid \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} ]
$$
Since  $\mathbb{P}[\gamma_{t,j}\in \mathcal{P}_t\mid \gamma_{t,(j - 1)}\in \mathcal{P}_t] = \pi_t(\gamma_{t,j})$  we then have:
$$
\mathbb {E} \left[ \lambda_ {t, j} \mid \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} \right] = \pi_ {t} \left(\gamma_ {t, j}\right) \epsilon_ {t, j} - \epsilon_ {t, (j - 1)} \tag {19}
$$
If, on the other hand,  $\gamma_{t,(j - 1)}\notin \mathcal{P}_t$  then  $\uparrow (\gamma_{t,j})\notin \mathcal{P}_t$  so  $\lambda_{t,j} = 0$ . This means that:
$$
\mathbb {E} \left[ \lambda_ {t, j} \right] = \mathbb {P} \left[ \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} \right] \mathbb {E} \left[ \lambda_ {t, j} \mid \gamma_ {t, (j - 1)} \in \mathcal {P} _ {t} \right] \tag {20}
$$
Since the probability that  $\gamma_{t,(j - 1)}\in \mathcal{P}_t$  is equal to  $\prod_{j^{\prime}\in [j - 1]}\pi_t(\gamma_{t,j'})$  we then have, by combining equations (19) and (20), that:
$$
\mathbb {E} [ \lambda_ {t, j} ] = \epsilon_ {t, j} \prod_ {j ^ {\prime} \in [ j ]} \pi_ {t} (\gamma_ {t, j ^ {\prime}}) - \epsilon_ {t, (j - 1)} \prod_ {j ^ {\prime} \in [ j - 1 ]} \pi_ {t} (\gamma_ {t, j ^ {\prime}})
$$
By substituting into Equation (18) (after taking expectations) we then have that:
$$
\begin{array}{l} \mathbb {E} \left[ \Delta_ {t} - \Delta_ {t + 1} \right] = - \epsilon_ {t, 0} + \epsilon_ {t, \log (K)} \prod_ {j \in [ \log (K) ]} \pi_ {t} (\gamma_ {t, j}) \\ = - \mathbb {E} [ \ln (\psi_ {t, 0}) ] + \mathbb {E} [ \ln (\psi_ {t, \log (K)}) \mid a _ {t} = \gamma_ {t, \log (K)} ] \prod_ {j \in [ \log (K) ]} \pi_ {t} (\gamma_ {t, j}) \tag {21} \\ \end{array}
$$
Note that if  $a_{t} = \gamma_{t,\log (K)}$  then  $\gamma_{t,j} = v_{t,j}$  for all  $j\in [\log (K)]$ . By definition of  $\psi_{t,\log (K)}$  and the fact that  $\gamma_{t,\log (K)} = y(x_{t})$ , Equation (21) then gives us:
$$
\mathbb {E} \left[ \Delta_ {t} - \Delta_ {t + 1} \right] = - \mathbb {E} \left[ \ln \left(\psi_ {t, 0}\right) \right] - \eta \ell_ {t, y \left(x _ {t}\right)} \tag {22}
$$
For all  $(v,a)\in \mathcal{B}\times [K]$  define:
$$
p _ {t, a} (v) = \mathbb {P} [ a _ {t} = a \mid v \in \mathcal {P} _ {t} ]
$$
noting that this is non-zero only when  $a \in \Downarrow(v)$ . Suppose we have some  $v \in \mathcal{B} \setminus \{r(\mathcal{B})\}$  and some  $a \in \Downarrow(v) \cap [K]$ . Then, since  $\mathbb{P}[a_t = a \mid v \notin \mathcal{P}_t] = 0$ , we have:
$$
p _ {t, a} (\uparrow (v)) = \mathbb {P} [ a _ {t} = a \mid \uparrow (v) \in \mathcal {P} _ {t} ] = \mathbb {P} [ a _ {t} = a \mid v \in \mathcal {P} _ {t} ] \mathbb {P} [ v \in \mathcal {P} _ {t} \mid \uparrow (v) \in \mathcal {P} _ {t} ] = \pi_ {t} (v) p _ {t, a} (v)
$$
Since  $p_{t,a}(v) = 0$  whenever  $a \notin \Downarrow(v)$ , this implies that for all  $(v, a) \in \mathcal{B}^{\dagger} \times [K]$  we have:
$$
p _ {t, a} (v) = \pi_ {t} (\triangleleft (v)) p _ {t, a} (\triangleleft (v)) + \pi_ {t} (\triangleright (v)) p _ {t, a} (\triangleright (v)) \tag {23}
$$
For all  $a\in [K]$  define:
$$
\hat {\ell} _ {t, a} = \frac {\llbracket a _ {t} = a \rrbracket \ell_ {t , a}}{\mathbb {P} [ a _ {t} = a ]}
$$
We now take the inductive hypothesis that for all  $j \in [\log(K)] \cup \{0\}$  we have:
$$
\psi_ {t, j} = \sum_ {a \in [ K ]} p _ {t, a} (v _ {t, j}) \exp (- \eta \hat {\ell} _ {t, a})
$$
and prove this via reverse induction (i.e. from  $j = \log(K)$  to  $j = 0$ ). Note that given  $a' := a_t$  we have  $\mathbb{P}[a_t = a'] = \prod_{j \in [\log(K)]} \pi_t(v_{t,j})$  and hence:
$$
\psi_ {t, \log (K)} = \exp (- \eta \hat {\ell} _ {t, a _ {t}})
$$
so the inductive hypothesis holds for  $j = \log(K)$ . Now suppose that we have some  $j' \in [\log(K)]$  and that the inductive hypothesis holds for  $j = j'$ . We shall now show that it holds also for  $j = j' - 1$ . Let  $v'$  be the child of  $v_{t,(j' - 1)}$  that is not equal to  $v_{t,j'}$ . Note that  $a_t \notin \Downarrow(v')$  and hence  $\exp(-\eta \hat{\ell}_{t,a}) = 1$  for all  $a \in \Downarrow(v')$  (i.e. whenever  $p_{t,a}(v') \neq 0$ ) which implies:
$$
\sum_ {a \in [ K ]} p _ {t, a} \left(v ^ {\prime}\right) \exp \left(- \eta \hat {\ell} _ {t, a}\right) = 1 \tag {24}
$$
For all  $a\in [K]$  ,Equation (23) gives us:
$$
p _ {t, a} \left(v _ {t, \left(j ^ {\prime} - 1\right)}\right) \exp \left(- \eta \hat {\ell} _ {t, a}\right) = \pi_ {t} \left(v ^ {\prime}\right) p _ {t, a} \left(v ^ {\prime}\right) \exp \left(- \eta \hat {\ell} _ {t, a}\right) + \pi_ {t} \left(v _ {t, j ^ {\prime}}\right) p _ {t, a} \left(v _ {t, j ^ {\prime}}\right) \exp \left(- \eta \hat {\ell} _ {t, a}\right)
$$
Substituting Equation (24) and the inductive hypothesis into this equation (when summed over all  $a \in [K]$ ) then gives us:
$$
\sum_ {a \in [ K ]} p _ {t, a} (v _ {t, (j ^ {\prime} - 1)}) \exp (- \eta \hat {\ell} _ {t, a}) = \pi_ {t} (v ^ {\prime}) + \pi_ {t} (v _ {t, j ^ {\prime}}) \psi_ {t, j ^ {\prime}}
$$
Since  $\pi_t(v') + \pi_t(v_{t,j'}) = 1$  we have, direct from the algorithm, that  $\pi_t(v') + \pi_t(v_{t,j'})\psi_{t,j'} = \psi_{t,(j' - 1)}$  so the inductive hypothesis holds for  $j = j' - 1$ . We have hence shown that the inductive hypothesis holds for all  $j \in [\log(K)] \cup \{0\}$  and in particular for  $j = 0$ . Since  $p_{t,a}(v_{t,0}) = \mathbb{P}[a_t = a]$  we then have:
$$
\psi_ {t, 0} = \sum_ {a \in [ K ]} \mathbb {P} \left[ a _ {t} = a \right] \exp \left(- \eta \hat {\ell} _ {t, a}\right) \tag {25}
$$
Since  $\exp(-z) \leq 1 - z + z^2 / 2$  for all  $z \in \mathbb{R}_+$  we have, from Equation (25), that:
$$
\psi_ {t, 0} \leq \sum_ {a \in [ K ]} \mathbb {P} [ a _ {t} = a ] \left(1 - \eta \hat {\ell} _ {t, a} + \frac {\eta^ {2} \hat {\ell} _ {t , a}}{2}\right) = 1 - \eta \sum_ {a \in [ K ]} \mathbb {P} [ a _ {t} = a ] \hat {\ell} _ {t, a} + \frac {\eta^ {2}}{2} \sum_ {a \in [ K ]} \mathbb {P} [ a _ {t} = a ] \hat {\ell} _ {t, a} ^ {2}
$$
so since  $\ln (1 + z)\leq z$  for all  $z\in \mathbb{R}$  we have:
$$
\ln \left(\psi_ {t, 0}\right) \leq - \eta \sum_ {a \in [ K ]} \mathbb {P} \left[ a _ {t} = a \right] \hat {\ell} _ {t, a} + \frac {\eta^ {2}}{2} \sum_ {a \in [ K ]} \mathbb {P} \left[ a _ {t} = a \right] \hat {\ell} _ {t, a} ^ {2} \tag {26}
$$
Noting that  $\mathbb{P}[a_t = a]\hat{\ell}_{t,a} = [[a_t = a]]\ell_{t,a}$  for all  $a\in [K]$ , we have:
$$
\mathbb {E} \left[ \sum_ {a \in [ K ]} \mathbb {P} \left[ a _ {t} = a \right] \hat {\ell} _ {t, a} \right] = \mathbb {E} \left[ \ell_ {t, a _ {t}} \right]
$$
$$
\mathbb {E} \left[ \sum_ {a \in [ K ]} \mathbb {P} [ a _ {t} = a ] \hat {\ell} _ {t, a} ^ {2} \right] = \mathbb {E} \left[ \sum_ {a \in [ K ]} \frac {[   [ a _ {t} = a ]   ] \ell_ {t , a} ^ {2}}{\mathbb {P} [ a _ {t} = a ]} \right] = \sum_ {a \in [ K ]} \ell_ {t, a} ^ {2} \leq K
$$
Substituting these equations into Equation (26) (after taking expectations) gives us:
$$
\mathbb {E} \left[ \ln \left(\psi_ {t, 0}\right) \right] \leq - \eta \mathbb {E} \left[ \ell_ {t, a _ {t}} \right] + \eta^ {2} K / 2
$$
which, upon substitution into Equation (22) gives us:
$$
\mathbb {E} \left[ \Delta_ {t} - \Delta_ {t + 1} \right] \geq \eta \left(\mathbb {E} \left[ \ell_ {t, a _ {t}} \right] - \ell_ {t, y \left(x _ {t}\right)}\right) - \eta^ {2} K / 2 \tag {27}
$$
Note that this equation implies that the same equation also holds when the expectation is not implicitly conditional on the state of the algorithm at the start of trial  $t$ . Hence, we now drop the assumption that the expectation is conditional on the state of the algorithm at the start of trial  $t$ . Summing Equation (27) over all trials  $t \in [T]$  and then rearranging gives us:
$$
\mathbb {E} [ R (y) ] \leq \frac {1}{\eta} \left(\mathbb {E} \left[ \Delta_ {1} \right] - \mathbb {E} \left[ \Delta_ {T + 1} \right]\right) + \frac {\eta K T}{2} \tag {28}
$$
Now consider a trial  $t$ . For all  $v \in \mathcal{B}^{\dagger}$  let:
$$
V _ {t} (v) := \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} [   [ x _ {t} \in \mathcal {S} ]   ] w _ {t + 1} (\triangleleft (v), \mathcal {S}) + \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} [   [ x _ {t} \in \mathcal {S} ]   ] w _ {t + 1} (\triangleright (v), \mathcal {S})
$$
Now take any  $j \in [\log(K) - 1] \cup \{0\}$  and let  $v \coloneqq v_{t,j}$ . Note that:
$$
V _ {t} (v) = \beta_ {t} (\triangleleft (v)) \theta_ {t} (\triangleleft (v)) + \beta_ {t} (\triangleright (v)) \theta_ {t} (\triangleright (v))
$$
so that by definition of  $\pi_t(\triangleleft(v))$  and  $\pi_t(\triangleright(v))$  we have:
$$
V _ {t} (v) = \left(\theta_ {t} (\triangleleft (v)) + \theta_ {t} (\triangleright (v))\right) \left(\pi_ {t} (\triangleleft (v)) \beta_ {t} (\triangleleft (v)) + \pi_ {t} (\triangleright (v)) \beta_ {t} (\triangleright (v))\right)
$$
Without loss of generality assume that  $\triangleleft(v) \in \mathcal{P}_t$ . Then the above equation implies that:
$$
V _ {t} (v) = \left(\theta_ {t} (\triangleleft (v)) + \theta_ {t} (\triangleright (v))\right) \frac {\pi_ {t} (\triangleleft (v)) \psi_ {t , j + 1} + \pi_ {t} (\triangleright (v))}{\psi_ {t , j}}
$$
so by definition of  $\psi_{t,j}$  we have:
$$
V _ {t} (v) = \left(\theta_ {t} (\triangleleft (v)) + \theta_ {t} (\triangleright (v))\right) = \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} [   [ x _ {t} \in \mathcal {S} ]   ] w _ {t} (\triangleleft (v), \mathcal {S}) + \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} [   [ x _ {t} \in \mathcal {S} ]   ] w _ {t} (\triangleright (v), \mathcal {S})
$$
Note that this equation trivially holds for all  $v \in \mathcal{B}^{\dagger} \setminus \mathcal{P}_t$  and hence holds for all  $v \in \mathcal{B}^{\dagger}$ . Since for all such  $v$  and all  $S$  with  $x_t \notin S$  we have  $w_{t+1}(\triangleleft(v), S) = w_t(\triangleleft(v), S)$  and  $w_{t+1}(\triangleright(v), S) = w_t(\triangleright(v), S)$  we then have:
$$
\sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t + 1} (\triangleleft (v), \mathcal {S}) + \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t + 1} (\triangleright (v), \mathcal {S}) = \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t} (\triangleleft (v), \mathcal {S}) + \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t} (\triangleright (v), \mathcal {S})
$$
so, by induction on  $t$  we have, for all  $t \in [T + 1]$ , that:
$$
\sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t} (\triangleleft (v), \mathcal {S}) + \sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {t} (\triangleright (v), \mathcal {S}) = 1
$$
Hence, for all  $v \in \mathcal{B} \setminus r(\mathcal{B})$  and  $S \in 2^{\mathcal{X}}$ , we have  $w_{t}(v, S) \in [0,1]$ . We have now shown that  $\Delta_{T+1} \geq 0$  so that Equation 28 gives us:
$$
\mathbb {E} [ R (y) ] \leq \frac {1}{\eta} \mathbb {E} [ \Delta_ {1} ] + \frac {\eta K T}{2}
$$
which, by definition of  $\Delta_1$ , gives us the desired result.
E.5 Theorem B.2
The fact that the weighting  $w_{1}$  is valid is given by the following lemma:
Lemma E.1. For all  $v \in \mathcal{B}^{\dagger}$  we have:
$$
\sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} \left(w _ {1} (\triangleleft (v), \mathcal {S}) + w _ {t} (\triangleright (v), \mathcal {S})\right) = 1
$$
Proof. We will show that for all  $v \in \mathcal{B}'$  we have:
$$
\sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} w _ {1} (v, \mathcal {S}) = \frac {1}{2}
$$
which directly implies the result. So take some arbitrary  $v \in \mathcal{B}'$ . Define, for all  $t \in [T]$ , the sets:
$$
\mathcal {X} _ {t} ^ {\prime} := \left\{x _ {s} \mid s \in [ t ] \right\} \setminus \left\{x _ {1} \right\} \quad \text {a n d} \quad \mathcal {F} _ {t} := \left\{0, 1 \right\} ^ {\mathcal {X} _ {t} ^ {\prime} \cup \left\{x _ {1} \right\}}
$$
and for all  $x\in \mathcal{X}_t^\prime$  and  $f\in \mathcal{F}_t$  define the quantity:
$$
\beta (x, f) := \llbracket f (x) \neq f (n (x)) \rrbracket 1 / T + \llbracket f (x) = f (n (x)) \rrbracket (1 - 1 / T)
$$
which is defined since  $n(x)\in \mathcal{X}_t^\prime \cup \{x_1\}$ . For all  $t\in [T - 1]$  we have:
$$
\sum_ {f \in \mathcal {F} _ {t + 1}} \prod_ {x \in \mathcal {X} _ {t + 1} ^ {\prime}} \beta (x, f) = \sum_ {f \in \mathcal {F} _ {t}} \left(\prod_ {x \in \mathcal {X} _ {t} ^ {\prime}} \beta (x, f)\right) \sum_ {f \left(x _ {t + 1}\right) \in \{0, 1 \}} \beta \left(x _ {t + 1}, f\right) \tag {29}
$$
Given any  $f\in \mathcal{F}_t$  we have:
$$
\sum_ {f (x _ {t + 1}) \in \{0, 1 \}} \beta (x _ {t + 1}, f) = (1 - 1 / T) + 1 / T = 1
$$
and hence by Equation (29) we have:
$$
\sum_ {f \in \mathcal {F} _ {t + 1}} \prod_ {x \in \mathcal {X} _ {t + 1} ^ {\prime}} \beta (x, f) = \sum_ {f \in \mathcal {F} _ {t}} \prod_ {x \in \mathcal {X} _ {t} ^ {\prime}} \beta (x, f)
$$
Since  $\mathcal{X}_T' = \mathcal{X}'$  this implies, by induction, that:
$$
\sum_ {f \in \mathcal {F} _ {T}} \prod_ {x \in \mathcal {X} ^ {\prime}} \beta (x, f) = \sum_ {f \in \mathcal {F} _ {1}} \prod_ {x \in \mathcal {X} _ {1} ^ {\prime}} \beta (x, f) = \sum_ {f \in \mathcal {F} _ {1}} \prod_ {x \in \emptyset} \beta (x, f) = \sum_ {f \in \mathcal {F} _ {1}} 1 = | \mathcal {F} _ {1} | = 2 \tag {30}
$$
Note that we have a bijection  $\mathcal{G}:\mathcal{F}_T\to 2^{\mathcal{X}}$  defined by:
$$
\mathcal {G} (f) := \{x \in \mathcal {X} \mid f (x) = 1 \} \quad \forall f \in \mathcal {F} _ {T}
$$
and that for all  $(f,x)\in \mathcal{F}_T\times \mathcal{X}'$  we have:
$$
\beta (x, f) = \sigma (x, \mathcal {G} (f)) / T + (1 - \sigma (x, \mathcal {G} (f))) (1 - 1 / T)
$$
Hence, Equation (30) shows us that:
$$
\sum_ {\mathcal {S} \in 2 ^ {\mathcal {X}}} \prod_ {x \in \mathcal {X} ^ {\prime}} \left(\sigma (x, \mathcal {S}) \frac {1}{T} + (1 - \sigma (x, \mathcal {S})) \left(1 - \frac {1}{T}\right)\right) = 2
$$
Now that we have shown that the weighting  $w_{1}$  is valid we can utilise Theorem B.1 to prove our regret bound. For any set  $S \in 2^{\mathcal{X}}$  define:
$$
\phi (\mathcal {S}): = \sum_ {x \in \mathcal {X} ^ {\prime}} \sigma (x, \mathcal {S})
$$
Note that for all  $v\in \mathcal{B}^{\dagger}$  and  $S\in 2^{\mathcal{X}}$  we have:
$$
w _ {1} (v, \mathcal {S}) = \frac {1}{4} \left(\frac {1}{T}\right) ^ {\phi (\mathcal {S})} \left(1 - \frac {1}{T}\right) ^ {T - 1 - \phi (\mathcal {S})} \geq \frac {1}{4} \left(\frac {1}{T}\right) ^ {\phi (\mathcal {S})} \left(1 - \frac {1}{T}\right) ^ {T}
$$
so since  $T \ln (1 - 1 / T) \in \mathcal{O}(1)$  we have:
$$
- \ln \left(w _ {1} (v, \mathcal {S})\right) \leq \ln (4) + \phi (\mathcal {S}) \ln (T) - T \ln (1 - 1 / T) \in \mathcal {O} (\phi (\mathcal {S}) \ln (T) + 1) \tag {31}
$$
As in the statement of Theorem B.1 define, for all  $v \in \mathcal{B}$ , the set:
$$
\mathcal {Q} (y, v) := \{x \in \mathcal {X} \mid y (x) \in \Downarrow (v) \}
$$
First note that the graph (with vertex set  $\mathcal{X}$ ) formed by linking  $x$  to  $n(x)$  for every  $x \in \mathcal{X}'$  is a tree so that  $\Phi(y) \geq |\{y(x) | x \in \mathcal{X}\}| - 1$ . So since for all  $v \in \mathcal{B}'$  we have  $\mathcal{Q}(y, v) \neq \emptyset$  if and only if  $v$  has a descent in  $\{y(x) | x \in \mathcal{X}\}$  and each element of  $\{y(x) | x \in \mathcal{X}\}$  has  $\log(K)$  ancestors in  $\mathcal{B}'$  we have:
$$
\sum_ {v \in \mathcal {B} ^ {\prime}} \llbracket \mathcal {Q} (y, v) \neq \emptyset \rrbracket \leq \log (K) | \{y (x) \mid x \in \mathcal {X} \} | \leq \log (K) (\Phi (y) + 1) \tag {32}
$$
Now suppose we have some  $x \in \mathcal{X}'$ . If  $y(x) = y(n(x))$  then for all  $v \in \mathcal{B}'$  we have  $x, n(x) \in \mathcal{Q}(y, v)$  or  $x, n(x) \notin \mathcal{Q}(y, v)$  and hence  $\sigma(x, \mathcal{Q}(y, v)) = 0$ . On the other hand, if  $y(x) \neq y(n(x))$  then for any  $v \in \mathcal{B}'$  with  $\sigma(x, \mathcal{Q}(y, v)) = 1$  we must have that either  $x \in \mathcal{Q}(y, v)$  or  $n(x) \in \mathcal{Q}(y, v)$  so  $v$  is an ancestor of either  $x$  or  $n(x)$  and hence there can be at most  $2\log(K)$  such  $v$ . So in any case we have:
$$
\sum_ {v \in \mathcal {B} ^ {\prime}} \sigma (x, \mathcal {Q} (y, v)) \leq [   [ y (x) \neq y (n (x)) ]   ] 2 \log (K)
$$
$$
\sum_ {v \in \mathcal {B} ^ {\prime}} \phi (\mathcal {Q} (y, v)) = \sum_ {x \in \mathcal {X} ^ {\prime}} \sum_ {v \in \mathcal {B} ^ {\prime}} \sigma (x, \mathcal {Q} (y, v)) \leq 2 \log (K) \Phi (y) \tag {33}
$$
Equation (31) gives us:
$$
\left. - \sum_ {v \in \mathcal {B} ^ {\prime}} \llbracket \mathcal {Q} (y, v) \neq \emptyset \rrbracket \ln \left(w _ {1} (v, \mathcal {Q} (y, v))\right) \in \mathcal {O} \left(\ln (T) \sum_ {v \in \mathcal {B} ^ {\prime}} \phi (\mathcal {Q} (y, v)) + \sum_ {v \in \mathcal {B} ^ {\prime}} \llbracket \mathcal {Q} (y, v) \neq \emptyset \rrbracket\right)\right)
$$
Substituting in equations (32) and (33) then gives us:
$$
- \sum_ {v \in \mathcal {B} ^ {\prime}} \llbracket \mathcal {Q} (y, v) \neq \emptyset \rrbracket \ln \left(w _ {1} (v, \mathcal {Q} (y, v))\right) \in \mathcal {O} (\ln (K) \ln (T) \Phi (y))
$$
so by Theorem B.1 we have:
$$
\mathbb {E} [ R (y) ] \in \mathcal {O} \left(\frac {\eta K T}{2} + \frac {\ln (K) \ln (T) \Phi (y)}{\eta}\right)
$$
Since  $\eta = \rho \sqrt{\ln(K)\ln(T) / KT}$  we obtain the result.
E.6 Theorem C.1
Recall our sequence of trees  $\langle \mathcal{Z}_t \mid t \in [T] \setminus \{1\} \rangle$  noting that each of these trees is a contraction so that  $\tau_{i,i'}(\mathcal{Z}_t, \cdot)$  is defined for all  $i, i' \in \{0,1\}$ . Let  $\epsilon := 1 / T$ . Define  $\lambda': \mathcal{X} \to \mathbb{R}_+$  as follows. Given  $x \in \mathcal{X}$ , if there exists a leaf  $u \in \mathcal{J}^\star$  with  $\gamma(u) = x$  then  $\lambda'(x) = \lambda(u)$ . Otherwise  $\lambda'(x) = 1$ . Given  $t \in [T]$  define  $\hat{\lambda}_t: \mathcal{Z}_t \to \mathbb{R}_+$  such that for all  $u \in \mathcal{Z}_t$  we have that  $\hat{\lambda}_t(u) := \lambda'(\gamma(u))$  if  $u$  is a leaf of  $\mathcal{Z}_t$  and  $\hat{\lambda}_t(u) := 1$  otherwise. For all  $t \in [T]$  and  $f: \{x_{t'} \mid t' \in [t]\} \to \{0,1\}$  define:
$$
\mathcal {N} (f) := \left\{f ^ {\prime} \in \{0, 1 \} ^ {\mathcal {Z} _ {t}} \mid \forall u \in \mathcal {Z} _ {t} ^ {\star}, f ^ {\prime} (u) = f (\gamma (u)) \right\}
$$
$$
\hat {w} (f) := \left(\prod_ {t ^ {\prime} \in [ t ]: f \left(x _ {t ^ {\prime}}\right) = 1} \lambda^ {\prime} \left(x _ {t}\right)\right) \prod_ {t ^ {\prime} \in [ t ] \backslash \{1 \}} \left(\llbracket f \left(x _ {t}\right) \neq f (n \left(x _ {t}\right)) \rrbracket \epsilon + \llbracket f \left(x _ {t}\right) = f (n \left(x _ {t}\right)) \rrbracket (1 - \epsilon)\right)
$$
$$
\hat {\nu} (f) := \sum_ {f ^ {\prime} \in \mathcal {N} (f)} \prod_ {u \in \mathcal {Z} _ {t} \setminus \{r (\mathcal {Z} _ {t}) \}} \tau_ {f ^ {\prime} (\uparrow_ {\mathcal {Z} _ {t}} (u)), f ^ {\prime} (u)} (\mathcal {Z} _ {t}, u) \tilde {\kappa} _ {f ^ {\prime} (u)} (\mathcal {Z} _ {t}, \hat {\lambda} _ {t}, u)
$$
We now have the following lemma:
Lemma E.2. For all  $t\in [T]$  and  $f:\{x_{t^{\prime}}\mid t^{\prime}\in [t]\} \to \{0,1\}$  we have:
$$
\hat {w} (f) = \hat {\nu} (f)
$$
Proof. We prove by induction on  $t$ . Suppose the result holds for  $t = s$  (for some  $s \geq 2$ ). We now show that it holds for  $t = s + 1$  as well. Let  $f^*$  be the restriction of  $f$  onto the set  $\{x_{t'} \mid t' \in [s]\}$ . Let  $u^*$  and  $u'$  be the unique leaves in  $\mathcal{Z}_{s+1}^{\star}$  of which  $\gamma(u') = n(x_{s+1})$  and  $\gamma(u^*) = x_{s+1}$ . By the construction of  $\mathcal{Z}_{s+1}$  these vertices are siblings. Let  $u''$  be the parent (in  $\mathcal{Z}_{s+1}$ ) of both  $u^*$  and  $u'$ . First note that:
$$
\llbracket f \left(x _ {s + 1}\right) = 0 \rrbracket + \llbracket f \left(x _ {s + 1}\right) = 1 \rrbracket \lambda^ {\prime} \left(x _ {s + 1}\right) = \tilde {\kappa} _ {f \left(x _ {s + 1}\right)} \left(\mathcal {Z} _ {s + 1}, \hat {\lambda} _ {s + 1}, u ^ {*}\right) \tag {34}
$$
Since, by the construction of  $\mathcal{Z}_{s+1}$ , we have  $\gamma(\uparrow_{\mathcal{Z}_{s+1}}(u^*)) = \gamma(u'') = n(x_{s+1})$  we also have that  $d(\uparrow_{\mathcal{Z}_{s+1}}(u^*)) = d(u^*) - 1$  so that, since  $\phi_1 = \epsilon$ , we have:
$$
\llbracket f \left(x _ {s + 1}\right) \neq f \left(n \left(x _ {s + 1}\right)\right) \rrbracket \epsilon + \llbracket f \left(x _ {s + 1}\right) = f \left(n \left(x _ {s + 1}\right)\right) \rrbracket (1 - \epsilon) = \tau_ {f \left(n \left(x _ {s + 1}\right), f \left(x _ {s + 1}\right)\right)} \left(\mathcal {Z} _ {s + 1}, u ^ {*}\right) \tag {35}
$$
Equations (34) and (35) give us:
$$
\hat {w} (f) = \hat {w} \left(f ^ {*}\right) \tau_ {f \left(n \left(x _ {s + 1}\right)\right), f \left(x _ {s + 1}\right)} \left(\mathcal {Z} _ {s + 1}, u ^ {*}\right) \tilde {\kappa} _ {f \left(x _ {s + 1}\right)} \left(\mathcal {J}, \hat {\lambda} _ {s + 1}, u ^ {*}\right) \tag {36}
$$
Now suppose we have some  $f' \in \mathcal{N}(f)$ . We have  $\gamma(u'') = \gamma(u')$  and hence  $d(\uparrow_{\mathcal{Z}_{s+1}}(u')) = d(u'') = d(u')$  so since  $f'(u') = f(n(x_{s+1}))$  and  $\phi_0 = 0$  we have: