SlowGuess/NeurIPS_2023_OCR huggingface dataset:
by the technique used in the proof of Proposition 2 by Kveton et al. [25], which provides a regret lower bound for combinatorial semi-bandits. To see this, suppose that  $N = MK$  for simplicity. The action set of  $K$ -path combinatorial semi-bandit (in Proposition 2 by Kveton et al. [25]) is expressed as  $\mathcal{A}' = \{A_k \mid k \in [K]\} \cup \{\emptyset\}$ , where  $A_k = \{M(k - 1) + j \mid j \in [M]\}$ . Note that this does not satisfy the assumption that the action set is closed under inclusion, which is posed in our problem setting. In order to address this assumption, we set  $\mathcal{A} = \{A \subseteq [N] \mid \exists A' \in \mathcal{A}', A \subseteq A'\}$ , which is closed under inclusion. Let us consider bandit task assignment problems with action set  $\mathcal{A}$ , instead of  $\mathcal{A}'$ . We suppose that, for each  $k \in [K]$ ,  $r_{ti}$  and  $c_{ti}$  take the same values for all  $i \in A_i$ , i.e., there exists  $r_{tk}'$  and  $c_{tk}'$  such that  $r_{ti} = r_{tk}'$  and  $c_{ti} = c_{tk}'$  for all  $i \in A_i$ . If  $r'_tk$  and  $c'_tk$  are i.i.d. for different  $t$ , this is a proper problem instance in our model. (Our model does not assume that  $r_{ti}$  and  $c_{ti}$  are independent for different tasks  $i$  as stated in Section 2). For any action  $A_t \in \mathcal{A} \setminus \{\emptyset\}$ , let  $A'_t \in \mathcal{A}'$  be such that  $A_t \subseteq A'_t$ . Then, if  $(A_t)_{t=1}^T$  is feasible,  $(A'_t)_{t=1}^T$  is also feasible, and the reward for the latter is greater than or equal to the former. This means that any algorithm can be converted to one that takes only actions in  $\mathcal{A}'$  without sacrificing the performance. Hence, it is sufficient to consider only algorithms that choose actions from  $\mathcal{A}'$  in the proof of lower bounds. Therefore, regret lower bounds with action set  $\{\{k\} \mid k \in [K]\} \cup \{\emptyset\}$ , multiplied with  $M$ , apply to our problem setting. Hence, the lower bound of (32) implies (33) as well.
The rest of this section is dedicated to the proofs of (30) and (31).
Proof of (30) If  $M = 1$  and  $c_{ti} = C_1$  for all  $t$  and  $i$  with probability one, the problem is equivalent to a standard  $N$ -armed bandit problem with time horizon  $T' = \Theta(T / C_1)$ . As shown in Theorem 5.1 by Auer et al. [6], any algorithm for the  $N$ -armed bandit problem suffers  $\Omega(\min\{\sqrt{NT'}, T'\})$ -regret in the worst case with time horizon  $T'$ . By substituting  $T' = \Theta(T / C_1)$  into this lower bound, we obtain (30).
Proof of (31) Set  $p = \frac{C_1}{C_{\mathrm{u}} + C_{\mathrm{l}}}$ . Fix an arbitrary algorithm. For some fixed  $i^* \in [N]$  and  $\epsilon \in [0, p/2]$ , define a distribution  $\mathcal{D}_{i^*, \epsilon}$  of  $c$  and  $r$  as follows:
$$
c _ {i} = \left\{ \begin{array}{l l} C _ {\mathrm {u}} & \text {w i t h p r o b a b i l i t y} \quad p - \epsilon \cdot \mathbf {1} [ i = i ^ {*} ] \\ C _ {\mathrm {l}} & \text {w i t h p r o b a b i l i t y} \quad 1 - p + \epsilon \cdot \mathbf {1} [ i = i ^ {*} ] \end{array} \right.,
$$
$$
r _ {i} = 1 \quad (i \in [ N ]).
$$
We also denote this distribution with  $\epsilon = 0$  by  $\mathcal{D}_0$ , which does not depend on  $i^*$ . Denote  $\bar{c} = pC_{\mathrm{u}} + (1 - p)C_1 = \frac{2C_1C_{\mathrm{u}}}{C_1 + C_{\mathrm{u}}} = 2pC_{\mathrm{u}}$ . Then the expected processing time can be expressed as
$$
\bar {c} _ {i} = \mathbf {E} \left[ c _ {i} \right] = \bar {c} - \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {\mathrm {l}}\right) \mathbf {1} [ i = i ^ {*} ]. \tag {34}
$$
Denote the number of completing the task  $i$  during  $T$  rounds by  $T_{i}$ . Then the expected regret can be expressed as
$$
\begin{array}{l} R _ {T} \geq \underset {i ^ {*}, \epsilon} {\mathbf {E}} \left[ (T - C _ {\mathrm {u}}) (\bar {c} - \epsilon \cdot (C _ {\mathrm {u}} - C _ {1})) ^ {- 1} - \sum_ {i = 1} ^ {N} T _ {i} \right] \\ \geq \frac {T}{\bar {c}} \left(1 + \frac {1}{\bar {c}} \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {1}\right)\right) - \mathbf {E} _ {i ^ {*}, \epsilon} \left[ \sum_ {i = 1} ^ {N} T _ {i} \right] - \frac {2 C _ {\mathrm {u}}}{C _ {1}}, \tag {35} \\ \end{array}
$$
where  $\mathbf{E}_{i^{*},\epsilon}$  means the expectation for the case in which  $c$  and  $r$  follow  $\mathcal{D}_{i^{*},\epsilon}$ . As the expected value of the number of rounds to process the task  $i$  is expressed as  $\mathbf{E}_{i^{*},\epsilon}[\bar{c}_i T_i]$ , we have
$$
\mathbf {E} _ {i ^ {*}, \epsilon} \left[ \sum_ {i = 1} ^ {N} \bar {c} _ {i} T _ {i} \right] \leq T.
$$
From (34), the left-hand side of this can be expressed as
$$
\mathbf {E} _ {i ^ {*}, \epsilon} \left[ \sum_ {i = 1} ^ {N} \bar {c} _ {i} T _ {i} \right] = \mathbf {E} _ {i ^ {*}, \epsilon} \left[ \bar {c} \sum_ {i = 1} ^ {N} T _ {i} - \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {1}\right) T _ {i ^ {*}} \right].
$$
Combining the above, we obtain
$$
\mathbf {E} _ {i ^ {*}, \epsilon} \left[ \sum_ {i = 1} ^ {N} T _ {i} \right] \leq \frac {1}{c} \left(T + \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {\mathrm {l}}\right) \mathbf {E} _ {i ^ {*}, \epsilon} [ T _ {i ^ {*}} ]\right). \tag {36}
$$
Similarly, we can show that
$$
\mathbf {E} _ {0} \left[ \sum_ {i = 1} ^ {N} T _ {i} \right] \leq \frac {T}{\bar {c}},
$$
where  $\mathbf{E}_0$  means the expectation for the case in which  $c$  and  $r$  follow  $\mathcal{D}_0$ . This means that there exists  $i^*$  such that
$$
\mathbf {E} _ {0} \left[ T _ {i ^ {*}} \right] \leq \frac {T}{\bar {c} N}
$$
We assume that  $i^{*}$  satisfies this in the following.
From (35) and (36), we have
$$
R _ {T} \geq \epsilon \cdot \frac {C _ {\mathrm {u}} - C _ {\mathrm {l}}}{\bar {c}} \cdot \left(\frac {T}{\bar {c}} - \mathbf {E} _ {i ^ {*}, \epsilon} [ T _ {i ^ {*}} ]\right) - \frac {2 C _ {\mathrm {u}}}{C _ {\mathrm {l}}}. \tag {37}
$$
By a similar argument to Lemma 6.6 of by Badanidiyuru et al. [7], we can show that
$$
\underset {i ^ {*}, \epsilon} {\mathbf {E}} \left[ T _ {i} ^ {*} \right] \leq \frac {3}{4} \frac {T}{\bar {c} - \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {\mathrm {l}}\right)} \quad \text {i f} \quad \epsilon \leq \frac {1}{1 6} \sqrt {\frac {p \bar {c} N}{T}}.
$$
Hence, if  $\epsilon = \min \left\{\frac{p}{4}, \frac{1}{16}\sqrt{\frac{p\overline{c}N}{T}}\right\}$ , we have
$$
\frac {3}{4} \frac {T}{\bar {c} - \epsilon \cdot \left(C _ {\mathrm {u}} - C _ {\mathrm {l}}\right)} \leq \frac {3}{4} \frac {T}{\bar {c} - p C _ {\mathrm {u}} / 4} = \frac {3}{4} \frac {T}{\bar {c} - \bar {c} / 8} = \frac {6}{7} \frac {T}{\bar {c}}.
$$
Combining this with (37), we obtain
$$
\begin{array}{l} R _ {T} \geq \frac {1}{7} \epsilon \cdot \frac {C _ {\mathrm {u}} - C _ {1}}{\bar {c}} \frac {T}{\bar {c}} - \frac {2 C _ {\mathrm {u}}}{C _ {1}} = \frac {1}{7} \cdot \frac {C _ {\mathrm {u}} - C _ {1}}{\bar {c}} \frac {T}{\bar {c}} \min  \left\{\frac {p}{4}, \frac {1}{1 6} \sqrt {\frac {p \bar {c} N}{T}} \right\} - \frac {2 C _ {\mathrm {u}}}{C _ {1}} \\ = \frac {1}{7} \cdot \frac {C _ {\mathrm {u}} - C _ {\mathrm {l}}}{2 p C _ {\mathrm {u}}} \frac {T}{2 p C _ {\mathrm {u}}} \min  \left\{\frac {p}{4}, \frac {1}{1 6} \sqrt {\frac {2 p ^ {2} C _ {\mathrm {u}} N}{T}} \right\} - \frac {2 C _ {\mathrm {u}}}{C _ {\mathrm {l}}} \\ = \Omega \left(\frac {C _ {\mathrm {u}} - C _ {\mathrm {l}}}{C _ {\mathrm {l}} C _ {\mathrm {u}}} \min  \left\{T, \sqrt {C _ {\mathrm {u}} N T} \right\}\right) - \frac {2 C _ {\mathrm {u}}}{C _ {\mathrm {l}}}, \\ \end{array}
$$
which completes the proof of (31).
Figure 4: Instance specified by parameters  $\bar{c} = (2.19, 4.6, 5.35, 1.42)$  and  $\bar{r} = (0.38, 0.43, 0.35, 0.47)$ , which are generated from uniform distributions over  $[C_1, C_u]$  and  $[0, 1]$ , respectively.
Figure 5: The average for instances in which the entries of  $\bar{c}$  and  $\bar{r}$  are generated from uniform distributions over  $[C_1, C_{\mathrm{u}}]$  and  $[0,1]$ , respectively. The shaded areas represent the standard deviations of the empirical regret.
B Notes on experimental setup
In the implementation on UCB-BV1 proposed by Ding et al. [15], by considering each element in  $\mathcal{A} = \{A\subseteq [N]\mid |A| = M\}$  as a single arm, we transform an instance of the bandit task assignment into a budget constrained multi-armed bandit problem with  $\binom{N}{M}$ -arms. To satisfy the constraints, we wait until all tasks in the previously chosen set are completed before deciding the next arm, i.e., the budget consumed by choosing  $A_{t}$  is given as  $\max_{i\in A_t}\{c_{ti}\}$ .
Similarly, in the implementation of CombUCB1 proposed by Kveton et al. [25], each time a set  $A_{t}$  in  $\mathcal{A}$  is selected and it waits until all the tasks in  $A_{t}$  are completed before selecting the next set. However, the way of choosing the set is different from that of UCB-BV1: we maintain the UCB scores for each task  $i \in [N]$ , not for each set  $A \in \mathcal{A}$ , and then choose a set so that the sum of the UCB scores for tasks in it are maximized, as suggested in the paper by Kveton et al. [25]. Data and source code to reproduce the experimental results in this paper are included in the supplementary materials.
This section provides the results of numerical experiments performed to evaluate (i) the performance for randomly generated instances and (ii) how the algorithm behaves under different settings, such as varying values of  $N$  and  $C_{\mathrm{u}}$ . Settings of parameters not specified in the captions are the same as in Section 5 and in Appendix B.
Figures 4 and 5 show the results of synthetic experiments for randomly chosen  $\bar{r}$  and  $\bar{c}$ . This experiment confirms that the proposed algorithm works well empirically in randomly generated environments, including cases where the values of  $\bar{r}$  are not necessarily constant.
Some results of an empirical evaluation of the algorithm's sensitivity to parameters  $N$  and  $C_{\mathrm{u}}$  are given in Figures 6, 7, and 8. Figure 6 suggests that the values of the regret scale linearly with respect to  $N$ , which is consistent with the regret upper bound of (7). Figure 7 shows the computation time for varying problem sizes, which implies that the proposed algorithm is superior in terms of computational complexity. Figure 8 represents how false beliefs about  $C_{\mathrm{u}}$  affect the performance of the algorithms. Experimental results suggest that the performance of the proposed algorithm is sensitive to underestimation of  $C_{\mathrm{u}}$ , while it is robust to overestimation of  $C_{\mathrm{u}}$ .
Figure 6: Dependency of the regret on the problem size.
Figure 7: Dependency of the computational time on the problem size.
Figure 8: Impact of the misspecification of  $C_{\mathrm{u}}$  on the regret. Note that the true value of  $C_{\mathrm{u}}$  is 6.