SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} u _ {i, j} \left(\mathbf {h} ^ {t}\right) = v _ {i, j} - p \left(\mathbf {h} ^ {t}\right) \\ = \left(v _ {i, j} - \beta_ {j}\right) + \left(\beta_ {j} - p \left(\mathbf {h} ^ {t}\right)\right) \\ = \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) \geq j \}} (v _ {i, j} - \beta_ {j}) + \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) = j \}} (\beta_ {j} - p (\mathbf {h} ^ {t}) \\ = \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) \geq j \}} (v _ {i, j} - \beta_ {j}) + \sum_ {k = j} ^ {K} \left[ \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) > k \}} (\beta_ {k} - \beta_ {k + 1}) + \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) = k \}} (\beta_ {k} - p (\mathbf {h} ^ {t})) \right]. \\ \end{array}
$$
$$
(S i n c e \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) > k \}} = 0 \forall k \geq j a n d \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) = k \}} = 1 i f a n d o n l y i f k = j.)
$$
2.  $x_{i}(\mathbf{h}^{t}) > j$ . Then  $x_{i}(\mathbf{h}^{t}) = j + \ell$ , for some  $\ell \in \{1, \dots, K - j\}$ . We have
$$
\begin{array}{l} u _ {i, j} \left(\mathbf {h} ^ {t}\right) = v _ {i, j} - p \left(\mathbf {h} ^ {t}\right) \\ = \left(v _ {i, j} - \beta_ {j}\right) + \left(\beta_ {j} - \beta_ {j + \ell}\right) + \left(\beta_ {j + \ell} - p \left(\mathbf {h} ^ {t}\right)\right) \\ = \left(v _ {i, j} - \beta_ {j}\right) + \left(\sum_ {k = j} ^ {j + \ell - 1} \beta_ {k} - \beta_ {k + 1}\right) + \left(\beta_ {j + \ell} - p \left(\mathbf {h} ^ {t}\right)\right). \tag {12} \\ \end{array}
$$
We are in the case where  $\mathbb{1}_{\{x_i(\mathbf{h}^t)\geq j\}} = 1$ $\mathbb{1}_{\{x_i(\mathbf{h}^t) > k\}} = 1$  if and only if  $k\in \{j,\ldots ,j + \ell -1\}$ , and  $\mathbb{1}_{\{x_i(\mathbf{h}^t) = k\}} = 1$  if and only if  $k = j + \ell$ . Adding indicators to the terms in (12) gives
$$
u _ {i, j} \left(\mathbf {h} ^ {t}\right) = \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) \geq j \right\}} \left(v _ {i, j} - \beta_ {j}\right) + \sum_ {k = j} ^ {K} \left[ \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) > k \right\}} \left(\beta_ {k} - \beta_ {k + 1}\right) + \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) = k \right\}} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \right].
$$
3.  $x_{i}(\mathbf{h}^{t}) < j$ . Then  $\mathbb{1}_{\{x_{i}(\mathbf{h}^{t}) \geq j\}} = 0$ ,  $\mathbb{1}_{\{x_{i}(\mathbf{h}^{t}) > k\}} = 0$  for all  $k \in \{j, \dots, K\}$ , and  $\mathbb{1}_{\{x_{i}(\mathbf{h}^{t}) = k\}} = 0$  for all  $k \in \{j, \dots, K\}$ . Thus we trivially have the identity required by the lemma statement since  $u_{i,j}(\mathbf{h}^{t}) = 0$  and the right hand side of (11) is zero as well.
Thus equation (11) holds in all three cases as required.
For each  $i\in [n]$  and  $j\in [K]$ , define
$$
A _ {i, j} = \sum_ {k = j} ^ {K} \left[ \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) > k \}} \left(\beta_ {k} - \beta_ {k + 1}\right) + \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) = k \}} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \right]. \tag {13}
$$
Then equation (11) is equivalent to
$$
u _ {i, j} \left(\mathbf {h} ^ {t}\right) = \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) \geq j \right\}} \left(v _ {i, j} - \beta_ {j}\right) + A _ {i, j}, \tag {14}
$$
Summing equation (13) over all  $j \in [K]$  gives
$$
\begin{array}{l} \sum_ {j = 1} ^ {K} A _ {i, j} = \sum_ {j = 1} ^ {K} \sum_ {k = j} ^ {K} \left[ \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) > k \}} \left(\beta_ {k} - \beta_ {k + 1}\right) + \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) = k \}} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \right] (15) \\ \stackrel {a} {=} \sum_ {k = 1} ^ {K} \sum_ {j = 1} ^ {k} \left[ \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) > k \}} \left(\beta_ {k} - \beta_ {k + 1}\right) + \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) = k \}} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \right] (16) \\ = \sum_ {k = 1} ^ {K} \left(\beta_ {k} - \beta_ {k + 1}\right) \sum_ {j = 1} ^ {k} \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) > k \}} + \sum_ {k = 1} ^ {K} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \sum_ {j = 1} ^ {k} \mathbb {1} _ {\{x _ {i} \left(\mathbf {h} ^ {t}\right) = k \}} (17) \\ = \sum_ {k = 1} ^ {K} \left(\beta_ {k} - \beta_ {k + 1}\right) \cdot k \cdot \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) > k \right\}} + \sum_ {k = 1} ^ {K} \left(\beta_ {k} - p \left(\mathbf {h} ^ {t}\right)\right) \cdot k \cdot \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) = k \right\}} (18) \\ \end{array}
$$
where equation (a) holds because we change the order of the double summations and in the double summations, we consider any (integer) pair of  $(j,k)$  such that  $1\leq k\leq j\leq K$
Then we can rewrite the utility  $u_{i}(\mathbf{h}^{t})$  as
$$
\begin{array}{l} u _ {i} (\mathbf {h} ^ {t}) = \sum_ {j = 1} ^ {K} u _ {i, j} (\mathbf {h} ^ {t}) \\ = \sum_ {j = 1} ^ {K} \left[ \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) \geq j \}} \left(v _ {i, j} - \beta_ {j}\right) + A _ {i, j} \right] \tag {By equation (11)} \\ = \sum_ {j = 1} ^ {K} \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) \geq j \right\}} \left(v _ {i, j} - \beta_ {j}\right) + \sum_ {j = 1} ^ {K} \left(\beta_ {j} - \beta_ {j + 1}\right) \cdot j \cdot \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) > j \right\}} + \sum_ {k = 1} ^ {K} \left(\beta_ {j} - p \left(\mathbf {h} ^ {t}\right)\right) \cdot j \cdot \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) = j \right\}} \\ (\text {B y e q u a t i o n (1 8)}) \\ = \sum_ {j = 1} ^ {K} \left[ \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) \geq j \}} (v _ {i, j} - \beta_ {j}) + j \left(\mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) > j \}} (\beta_ {j} - \beta_ {j + 1}) + \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) = j \}} (\beta_ {j} - p (\mathbf {h} ^ {t}))\right) \right], \\ \end{array}
$$
as required by the lemma statement.
Lemma 2. In the setting of Theorem 1, the edge weights of the graph  $G_{i}$  can be computed in polynomial time.
Proof. Recall the graph  $G_{i}$  is constructed given as parameters the number  $n$  of players, the number of units  $K$ , a player  $i$  with valuation  $\mathbf{v}_{i}$ , discretization level  $\varepsilon > 0$ , and a bid history  $H_{-i} = (\mathbf{b}_{-i}^{1}, \ldots, \mathbf{b}_{-i}^{T})$  by players other than  $i$ . Thus the goal is to show the edge weights of  $G_{i}$  can be computed in polynomial time in the bit length of these parameters.
Towards this end, we will show there exist efficiently computable values  $I_{>j}^{t}, I_{\geq j}^{t}, I_{j}^{t} \in \{0,1\}$  and  $q^{t} \in S_{i}$  such that the weight  $w_{e}$  of edge  $e = (z_{r,j}, z_{s,j+1})$  is equal to
$$
w _ {e} = \sum_ {t = 1} ^ {T} I _ {\geq j} ^ {t} \left(v _ {i, j} - r\right) + j \left(I _ {> j} ^ {t} (r - s) + I _ {j} ^ {t} (r - q ^ {t})\right). \tag {19}
$$
Roughly,  $q^t$  will correspond to the price and  $I_{>j}^t, I_{\geq j}^t, I_j^t$  will tell whether player  $i$  gets more than  $j$  units, at least  $j$  units, or exactly  $j$  units, respectively, at some profile  $\beta = (\beta_1, \ldots, \beta_K) \in S_i$  with  $\beta_j = r$  and  $\beta_{j+1} = s$ . The choice of  $\beta$  in will not matter, as long as  $\beta_j = r$  and  $\beta_{j+1} = s$ .
We prove equation (19) in several steps:
Step (i). For each  $t\in [T]$  , define  $\Gamma^t:\mathbb{R}\to \mathbb{R}$  , where
$$
\Gamma^ {t} (x) = \left| \left\{\left(\ell , j\right) \mid \left(b _ {\ell , j} ^ {t} > x \text {a n d} \ell \in [ n ] \backslash \{i \}, j \in [ K ]\right) \right. \right.
$$
$$
\left. \right.\left.\left. \operatorname {o r} \left(b _ {\ell , j} ^ {t} = x \text {a n d} \ell \in [ n ], \ell <   i, j \in [ K ]\right)\right\}\right|.
$$
Thus  $\Gamma^t (x)$  counts the bids in profile  $\mathbf{b}_{-i}^t$  that would have priority to a bid of value  $x$  submitted by player  $i$  (i.e. it counts bids strictly higher than  $x$  and submitted by players other than  $i$ , as well as bids equal to  $x$  but submitted by players lexicographically before  $i$ ).
Step (ii). Recall the edge is denoted  $e = (z_{r,j}, z_{s,j+1})$ . Let  $\beta \in S_i$  be an arbitrary bid profile with  $\beta_j = r$  and  $\beta_j = s$ . Also define  $\beta_{K+1} = 0$ .
If  $\Gamma^t (s) < K - j$ , then at  $\mathbf{h}^t = (\boldsymbol {\beta},\mathbf{b}_{-i}^t)$  player  $i$  receives one unit for each of the bids  $\beta_{1},\ldots ,\beta_{j + 1}$  since there are at most  $K - j - 1$  bids of other players that have higher priority than player  $i$ 's highest  $j + 1$  bids. Else, player  $i$  does not get more than  $j$  units. Thus  $I_{>j}^{t} = \mathbb{1}_{\{x_{i}(\mathbf{h}^{t}) > j\}}$ .
Step (iii). If  $\Gamma^t(r) \leq K - j$ , then player  $i$  receives at least  $j$  units at  $(\beta, \mathbf{b}_{-i}^t)$ . The corresponding indicator is
$$
I _ {\geq j} ^ {t} = \mathbb {1} _ {\{\Gamma^ {t} (s) \leq K - j \}} = \mathbb {1} _ {\{x _ {i} (\mathbf {h} ^ {t}) \geq j \}}.
$$
Step (iv). If  $\Gamma^t(r) \leq K - j$  and  $\Gamma^t(s) > K - j$ , then player  $i$  receives exactly  $j$  units at  $(\beta, \mathbf{b}_{-i}^t)$ . The indicator is
$$
I _ {j} ^ {t} = \mathbb {1} _ {\left\{\Gamma^ {t} (r) \leq K - j \right\}} \cdot \mathbb {1} _ {\left\{\Gamma^ {t} (s) > K - j \right\}} = \mathbb {1} _ {\left\{x _ {i} (\mathbf {h} ^ {t}) = j \right\}}.
$$
Now suppose  $I_{j}^{t} = 1$ . Then we show the price  $q^{t} = p(\mathbf{h}^{t})$  can be computed precisely without knowing the whole bid  $\beta$ .
Consider the multiset of bids  $B = \mathbf{b}_{-i}^{t} \cup \{\beta_{1},\dots,\beta_{j + 1}\}$ , recalling  $\beta_{K + 1}$  was defined as zero. Sort  $B$  in descending order. We know the top  $j - 1$  bids of player  $i$  are winning, so the price is not determined by any of them. Thus the price is determined by  $\beta_{j} = r$ ,  $\beta_{j + 1} = s$ , or by one of the bids in  $\mathbf{b}_{-i}^{t}$ . Remove elements  $\beta_{1},\ldots ,\beta_{j - 1}$  from  $B$  and set the price as follows:
- Case (iv.a). For  $(K + 1)$ -st price auction: set  $q^t$  to the  $(K + 2 - j)^{\mathrm{th}}$  highest value in  $B$ .
- Case (iv.b). For  $K$ -th price auction: set  $q^t$  to the  $(K + 1 - j)^{\mathrm{th}}$  highest value in  $B$ .
Step  $(\nu)$ . Combining steps (i-iv), the weight of edge  $e = (z_{r,j}, z_{s,j+1})$  can be rewritten as
$$
w _ {e} = \sum_ {t = 1} ^ {T} \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) \geq j \right\}} \left(v _ {i, j} - r\right) + j \left[ \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) > j \right\}} (r - s) + \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) = j \right\}} \left(r - p \left(\mathbf {h} ^ {t}\right)\right) \right]
$$
(By Definition 1)
$$
= \sum_ {t = 1} ^ {T} I _ {\geq j} ^ {t} \left(v _ {i, j} - r\right) + j \left[ I _ {> j} ^ {t} (r - s) + I _ {j} ^ {t} (r - q ^ {t})\right). \quad (\text {B y})
$$
Thus the weight of each edge can be computed in polynomial time as required.
B Appendix: Online Setting
In this section we include the material omitted from the main text for the online setting.
Before studying the full information and bandit feedback models in greater detail, we replace the set  $S_{i}$  of candidate bids for player  $i$  from equation (1) of the offline section by a coarser set
$$
\mathcal {S} _ {\varepsilon} = \{\varepsilon , 2 \varepsilon , \dots , \lceil v _ {i, 1} / \varepsilon \rceil \varepsilon \}.
$$
The regret, which was defined in equation (3), depends on whether the model is the  $K$ -th or  $(K + 1)$ -st price auction; however the next lemma holds for both variants of the auction.
Lemma 3. For all  $\varepsilon >0$ , let  $\mathrm{Reg}_i(\pi_i,H_{-i}^T,\varepsilon)$  be the counterpart of (3) where  $S_{i}$  is replaced by the set  $S_{\varepsilon}$ . Then  $\mathrm{Reg}_i(\pi_i,H_{-i}^T)\leq \mathrm{Reg}_i(\pi_i,H_{-i}^T,\varepsilon) + TK\varepsilon$
Proof. First observe that it is not beneficial to bid above  $v_{i,1}$ , so we can assume without loss of generality that the maximizer  $\beta$  in (3) satisfies  $\beta_j \leq v_{i,1}$  for all  $j \in [K]$ . Now we convert  $\beta$  into another bid vector  $\beta^{\varepsilon} \in S_{\varepsilon}^{K}$  as follows: for each  $j \in [K]$ , let
$$
\beta_ {j} ^ {\varepsilon} = \min  \{y \in S _ {\varepsilon} \mid y \geq \beta_ {j} \}.
$$
Clearly  $\beta_{j}\leq \beta_{j}^{\varepsilon}\leq \beta_{j} + \varepsilon$
Then we claim that the next inequalities hold:
$$
p \left(\boldsymbol {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}\right) \leq p \left(\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}\right) + \varepsilon ; \tag {20}
$$
$$
\mathbb {1} _ {\left\{x _ {i} \left(\boldsymbol {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}\right) \geq j \right\}} \geq \mathbb {1} _ {\left\{x _ {i} \left(\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}\right) \geq j \right\}}. \tag {21}
$$
Inequality (20) follows since
-  $p(\mathbf{b})$  is either the  $K$ -th or the  $(K + 1)$ -st largest element of  $\mathbf{b}$ , and
- increasing each entry of  $\mathbf{b}$  by at most  $\varepsilon$  can only increase  $p(\mathbf{b})$  by no more than  $\varepsilon$ .
Inequality (21) is due to the fact that bidding a higher price can only help to win the unit.
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {K} \left(v _ {i, j} - p (\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t})\right) \cdot \mathbb {1} _ {\{x _ {i} (\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}) \geq j \}} \leq \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {K} \left(v _ {i, j} - p (\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t})\right) \cdot \mathbb {1} _ {\{x _ {i} (\boldsymbol {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}) \geq j \}} \\ \leq \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {K} \left(v _ {i, j} - p \left(\boldsymbol {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}\right) - \varepsilon\right) \cdot \mathbb {1} _ {\left\{x _ {i} \left(\boldsymbol {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}\right) \geq j \right\}} \\ \leq \sum_ {t = 1} ^ {T} \sum_ {j = 1} ^ {K} \left(v _ {i, j} - p (\pmb {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t})\right) \cdot \mathbb {1} _ {\{x _ {i} (\pmb {\beta} ^ {\varepsilon}, \mathbf {b} _ {- i} ^ {t}) \geq j \}} + T K \varepsilon . \\ \end{array}
$$
This gives the desired statement of the lemma.
B.1 Full Information Feedback
In this section we include the omitted details for the full information setting.
We begin with the formal definition of the graph  $G^{t} = (V, E, w^{t})$  used by the online learning algorithm with full information feedback.
Definition 2 (The graph  $G^{t}$ ). Given valuation  $\mathbf{v}_{i}$  of player  $i$ , bid profile  $\mathbf{b}_{-i}^{t}$  of the players at round  $t$ , and  $\varepsilon > 0$ , construct a graph  $G^{t} = (V, E, w^{t})$  as follows.
- Vertices. Create a vertex  $z_{s,j}$  for each  $s \in S_{\varepsilon}$  and index  $j \in [K]$ . We say vertex  $z_{s,j}$  is in layer  $j$ . Add source  $z_{-}$  and sink  $z_{+}$ .
- **Edges.** For each index  $j \in [K - 1]$  and pair of bids  $r, s \in S_{\varepsilon}$  with  $r \geq s$ , create a directed edge from vertex  $z_{r,j}$  to vertex  $z_{s,j + 1}$ . Moreover, add edges from source  $z_{-}$  to each node in layer 1 and from each node in layer  $K$  to the sink  $z_{+}$ .
- Edge weights. For each edge  $e = (z_{r,j}, z_{s,j+1})$  or  $e = (z_{r,K}, z_+)$ , let  $\beta = (\beta_1, \ldots, \beta_K) \in S_{\varepsilon}^K$  be a bid vector with  $\beta_j = r$  and  $\beta_{j+1} = s$  (we define  $s = 0$  if  $j = K$ ). Define the weight of edge  $e$  as
$$
w ^ {t} (e) = \mathbb {1} _ {\left\{x _ {i} \left(\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}\right) \geq j \right\}} \left(v _ {i, j} - r\right) + j \left[ \mathbb {1} _ {\left\{x _ {i} \left(\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}\right) > j \right\}} (r - s) + \mathbb {1} _ {\left\{x _ {i} \left(\mathbf {h} ^ {t}\right) = j \right\}} \left(r - p \left(\boldsymbol {\beta}, \mathbf {b} _ {- i} ^ {t}\right)\right) \right].
$$
The edges incoming from  $z_{-}$  have weight zero.
The next observation follows immediately from the definition.
Observation 2. The next properties hold:
- The weight  $w^{t}(e)$  of each edge  $e$  of  $G^{t}$  can be computed efficiently (see Lemma 2).
- There is a bijective map between bid vectors  $\beta \in S_{\varepsilon}^{K}$  of player  $i$  and paths from the source to the sink of  $G^{t}$  (see proof of Theorem 1).
Next we include the main theorem with its proof for the online learning algorithm under full information feedback.
Theorem 2 (restated). For each player  $i$  and time horizon  $T$ , under full-information feedback, Algorithm 2 runs in time  $O(T^2)$  and guarantees the player's regret is at most  $O\left(v_{i,1}\sqrt{TK^3\log T}\right)$ .
Proof. The detailed algorithm description is presented as Algorithm 2. At a high level, the proof has three parts: (i) showing that Algorithm 2 is a correct implementation of the Hedge algorithm where each expert is a path from source to sink in the DAG  $G^{t}$  for time  $t$ , (ii) bounding its regret for an appropriate choice of the learning rate, and (iii) bounding the runtime.
Step I: Algorithm 2 is a correct implementation of Hedge. We show that Algorithm 2 is the same as the Hedge algorithm in which every path in the DAG is equivalent to an expert in the Hedge algorithm.
To do so, for each vertex  $u$  in the DAG, let  $\phi^t(u, \cdot)$  be a probability distribution over the outneighbors of  $u$ . Then, given the recursive sampling of the bids based on the probability distribution  $\phi^t$ 's, the probability that a path  $\mathfrak{p}$  is chosen in Algorithm 2 is equal to
$$
P ^ {t} (\mathfrak {p}) = \prod_ {e \in \mathfrak {p}} \phi^ {t} (e), \tag {22}
$$
where  $\phi^t$  's are updated in equation (5), which is restated below:
$$
\phi^ {t} (e) = \phi^ {t - 1} (e) \cdot \exp \left(\eta \cdot w ^ {t - 1} (e)\right) \cdot \frac {\Gamma^ {t - 1} (v)}{\Gamma^ {t - 1} (u)}, \quad \text {f o r a l l} e = (u, v) \in E (G ^ {t}). \quad (5 \text {r e v i s i t e d}.
$$
On the other hand, in the Hedge algorithm, the path probabilities are updated as follows: Given a learning rate  $\eta$ , define  $P_h^1(\mathfrak{p}) = \prod_{e \in \mathfrak{p}} \phi^1(e)$ , and for  $t \geq 2$ ,
$$
P _ {h} ^ {t} (\mathfrak {p}) = \frac {P _ {h} ^ {t - 1} (\mathfrak {p}) \exp \left(\eta \sum_ {e \in \mathfrak {p}} w ^ {t - 1} (e)\right)}{\sum_ {\mathfrak {q}} P _ {h} ^ {t - 1} (\mathfrak {q}) \exp \left(\eta \sum_ {e \in \mathfrak {q}} w ^ {t - 1} (e)\right)}. \tag {23}
$$
The subscript  $h$  in  $P_h^t (\mathfrak{p})$  stands for Hedge. To show that the update rule in Algorithm 2 is equivalent to the one in Hedge, we first prove the following statement:
$(\dagger)$  For any vertex  $u$  in the graph, let  $\mathcal{P}(u)$  be the set of all paths from  $u$  to  $z_{+}$ . Then
$$
\Gamma^ {t - 1} (u) = \sum_ {\mathfrak {q} \in \mathcal {P} (u)} \prod_ {e \in \mathfrak {q}} \left[ \phi^ {t - 1} (e) \exp \left(\eta w ^ {t - 1} (e)\right) \right]. \tag {24}
$$
We prove equation (24) by induction on  $u$ , from the bottom layer to the top layer. If  $u = z_{+}$ , by definition  $\Gamma^{t - 1}(z_{+}) = 1$ , and (24) holds. Now suppose that (24) holds for all  $u$  in the  $(k + 1)$ -st layer, for some  $0 \leq k \leq K$ . Then if  $u$  is in the  $k$ -th layer, the recursion of  $\Gamma^{t - 1}$  gives that
$$
\begin{array}{l} \Gamma^ {t - 1} (u) = \sum_ {v: (u, v) \in E} \phi^ {t - 1} ((u, v)) \exp (\eta w ^ {t - 1} ((u, v)) \cdot \Gamma^ {t - 1} (v) \\ = \sum_ {v: (u, v) \in E} \phi^ {t - 1} ((u, v)) \exp (\eta w ^ {t - 1} ((u, v))) \cdot \sum_ {\mathfrak {q} \in \mathcal {P} (v)} \prod_ {e \in \mathfrak {q}} [ \phi^ {t - 1} (e) \exp (\eta w ^ {t - 1} (e)) ] \\ = \sum_ {\mathfrak {q} \in \mathcal {P} (u)} \prod_ {e \in \mathfrak {q}} \left[ \phi^ {t - 1} (e) \exp (\eta w ^ {t - 1} (e)) \right], \\ \end{array}
$$
and therefore (24) holds.
Having shown equation (24), we prove by induction on  $t$  our claim that Algorithm 2 is a correct implementation of Hedge, i.e. that  $P^t(\mathfrak{p}) = P_h^t(\mathfrak{p})$  for all paths  $\mathfrak{p}$  and rounds  $t$ . For  $t = 1$ , by definition of our initialization we have  $P_h^1(\mathfrak{p}) = P^1(\mathfrak{p})$ . Suppose that at time  $t - 1$ , for every path  $\mathfrak{p}$  we have  $P_h^{t-1}(\mathfrak{p}) = P^{t-1}(\mathfrak{p}) = \prod_{e \in \mathfrak{p}} \phi^{t-1}(e)$ . Then at time  $t$ , it holds that
$$
\begin{array}{l} P ^ {t} (\mathfrak {p}) = \prod_ {e \in \mathfrak {p}} \phi^ {t} (e) \stackrel {\mathrm {(a)}} {=} \prod_ {e = (u, v) \in \mathfrak {p}} \left[ \phi^ {t - 1} (e) \cdot \exp \left(\eta \cdot w ^ {t - 1} (e)\right) \cdot \frac {\Gamma^ {t - 1} (v)}{\Gamma^ {t - 1} (u)} \right] \\ \stackrel {\mathrm {(b)}} {=} P _ {h} ^ {t - 1} (\mathfrak {p}) \exp \left(\eta \sum_ {e \in \mathfrak {p}} w ^ {t - 1} (e)\right) \cdot \frac {\Gamma^ {t - 1} (z _ {+})}{\Gamma^ {t - 1} (z _ {-})}, \\ \end{array}
$$
where step (a) is due to equation (5), and step (b) follows using telescoping and the induction hypothesis  $P_h^{t-1}(\mathfrak{p}) = \prod_{e \in \mathfrak{p}} \phi^{t-1}(e)$ .
Given equation (23), by applying equation (24) to  $u \in \{z_{-}, z_{+}\}$  and the induction hypothesis  $P_h^{t - 1}(\mathfrak{p}) = \prod_{e \in \mathfrak{p}} \phi^{t - 1}(e)$ , the induction is complete.
Thus Algorithm 2 is a correct implementation of Hedge.
Step II: Regret upper bound. Let
$$
\varepsilon = v _ {i, 1} \sqrt {K / T}. \tag {25}
$$
Applying Lemma 3 yields
$$
\operatorname {R e g} _ {i} \left(\pi_ {i}, H _ {- i} ^ {T}\right) \leq \operatorname {R e g} _ {i} \left(\pi_ {i}, H _ {- i} ^ {T}, \varepsilon\right) + T K \varepsilon = \operatorname {R e g} _ {i} \left(\pi_ {i}, H _ {- i} ^ {T}, \varepsilon\right) + v _ {i, 1} \sqrt {T K ^ {3}}, \tag {26}
$$
where  $\operatorname{Reg}_i(\pi_i, H_{-i}^T, \varepsilon)$  is the counterpart of (3) where  $\mathcal{S}_i$  is replaced by the set
We also claim that  $\sum_{e\in \mathfrak{p}}w^{t}(e)\leq Kv_{i,1}$  for each path  $\mathfrak{p}$  from source to sink in  $G^t$ . To see this, let  $\beta$  be the bid vector corresponding to path  $\mathfrak{p}$ . As shown in Lemma 1 and using the fact that edges outgoing from  $z_{-}$  have weight zero, we have  $u_{i}(\boldsymbol {\beta},b_{-i}^{t}) = \sum_{e\in \mathfrak{p}}w^{t}(e)$ . Since player  $i$ 's utility satisfies  $u_{i}(\boldsymbol {\beta},\mathbf{b}_{-i}^{t})\leq V_{i}(x_{i}(\boldsymbol {\beta},\mathbf{b}_{-i}^{t}))\leq Kv_{i,1}$ , we obtain  $\sum_{e\in \mathfrak{p}}w^{t}(e)\leq Kv_{i,1}$ .
By Step I, Algorithm 2 is a correct implementation of Hedge. To bound the regret of the algorithm, we will invoke Corollary 1—which is a slight variant of [CBL06, Theorem 2.2]—with the following parameters:
-  $N$  experts, where each expert is a path from source to sink in  $G^t$ ;
- learning rate  $\eta$ , time horizon  $T$ , and maximum reward  $L = Kv_{i,1}$ ;
- initial distribution  $\sigma$  on the experts, where  $\sigma_{\mathfrak{p}} = P^{1}(\mathfrak{p})$  for each expert (path from source to sink)  $\mathfrak{p}$ .
By Corollary 1, we obtain
$$
\operatorname {R e g} _ {i} \left(\pi_ {i}, H _ {- i} ^ {T}, \varepsilon\right) \leq \frac {1}{\eta} \max  _ {\mathfrak {p} \in [ N ]} \log \left(\frac {1}{\sigma_ {\mathfrak {p}}}\right) + \frac {T L ^ {2} \eta}{8}. \tag {27}
$$
Recall Algorithm 2 initially selects a path by starting at the source  $z_{-}$  and then performing an unbiased random walk in the layered DAG until reaching the sink  $z_{+}$ . Since the number of vertices in each layer is  $\lceil v_{i,1} / \varepsilon \rceil$ , the initial probability of selecting a particular expert (i.e. path from source to sink)  $\mathfrak{p}$  is
$$
\sigma_ {\mathfrak {p}} = P ^ {1} (\mathfrak {p}) \geq \frac {1}{\left[ v _ {i , 1} / \varepsilon \right] ^ {K}}, \quad \forall \mathfrak {p} \in [ N ]. \tag {28}
$$
Substituting (28) in (27) yields