SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} R (K) = R _ {\mathrm {b a t c h}} (K) = \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} _ {\mathcal {T}}} \left[ \mathcal {L} \big (\boldsymbol {x}; \mathcal {I} (\mathcal {T}, \boldsymbol {\Theta} ^ {(K)}) \big) \right] - \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} _ {\mathcal {T}}} \left[ \mathcal {L} \big (\boldsymbol {x}; \mathcal {I} (\mathcal {T}, \boldsymbol {\Theta} ^ {(K), *}) \big) \right] \\ = \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} _ {\mathcal {T}}} \left[ \mathcal {L} \left(\boldsymbol {x}; \mathcal {I} \left(\mathcal {T}, \boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]\right)\right) \right] - \mathbb {E} _ {\mathcal {T} \sim \mathcal {P} (\mathcal {T}), \boldsymbol {x} \sim \mathcal {D} _ {\mathcal {T}}} \left[ \mathcal {L} \left(\boldsymbol {x}; \mathcal {I} \left(\mathcal {T}, \boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]\right)\right) \right] \\ = h \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \Omega_ {K} ^ {*} \right]\right) - h \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right) \\ = h \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \Omega_ {K} ^ {*} \right]\right) - f \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \Omega_ {K} ^ {*} \right]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) + f \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \Omega_ {K} ^ {*} \right]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - f \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) \\ + f \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) - h \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right), \\ \end{array}
$$
and after applying properties of the arm pulling mechanism, it is equivalent to
$$
\begin{array}{l} R (K) \leq h (\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]) - f (\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}) \\ + f \left(\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - \hat {f} \left(\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) \\ + \hat {f} \left(\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - \hat {f} \left(\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ^ {*} ]; \boldsymbol {\theta} ^ {(K - 1)}\right) \\ + \hat {f} (\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]; \boldsymbol {\theta} ^ {(K - 1)}) - f (\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]; \boldsymbol {\theta} ^ {(K - 1)}) + f (\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]; \boldsymbol {\theta} ^ {(K - 1)}) - h (\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]) \\ \leq \underbrace {\left| h \left(\boldsymbol {\Theta} ^ {(K - 1) , *} [ \Omega_ {K} ^ {*} ]\right) - f \left(\boldsymbol {\Theta} ^ {(K - 1) , *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) \right|} _ {I _ {5}} + \underbrace {\left| f \left(\boldsymbol {\Theta} ^ {(K - 1) , *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - \hat {f} \left(\boldsymbol {\Theta} ^ {(K - 1) , *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) \right|} _ {I _ {6}} \\ + \underbrace {\left| \hat {f} \left(\Theta^ {(K - 1) , *} \left[ \Omega_ {K} ^ {*} \right] ; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - \hat {f} \left(\Theta^ {(K - 1)} \left[ \Omega_ {K} ^ {*} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) \right|} _ {I _ {7}} + \underbrace {\left| \hat {f} \left(\Theta^ {(K - 1)} \left[ \Omega_ {K} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) - f \left(\Theta^ {(K - 1)} \left[ \Omega_ {K} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) \right|} _ {I _ {8}} \\ + \underbrace {\left| f \left(\Theta^ {(K - 1)} \left[ \Omega_ {K} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) - h \left(\Theta^ {(K - 1)} \left[ \Omega_ {K} \right]\right) \right|} _ {I _ {9}} \\ \end{array}
$$
where the average value of estimated benefit scores for individual tasks  $\mathcal{T}_{K,i} \in \Omega_K$  is represented as  $\widehat{f}(\Theta^{(K-1)}[\Omega_K]) = \frac{1}{|\Omega_K|} \cdot \sum_{\mathcal{T}_{K,i} \in \Omega_K} f(\Theta^{(K-1)}[\mathcal{T}_{K,i}]) = \frac{1}{|\Omega_K|} \cdot \sum_{\mathcal{T}_{K,i} \in \Omega_K} f(\Theta^{(k-1)} - \eta_2 \cdot \nabla_\Theta \mathcal{L}(D_{K,i}^q; \Theta_{K,i}^{(J)}; \pmb{\theta}^{(K-1)})$ , and the inequality is due to the pulling mechanism of BASS. Here,  $I_5, I_9$  individually correspond to  $I_1, I_2$  in the single-task setting and can be bounded by Lemma D.3, Corollary D.4. Term  $I_7$  can be upper bounded by  $I_3 + I_4$  from the single-task setting above. Then, for the rest terms  $I_6, I_8$ , we proceed to bound them separately.
D.2.1 Bounding error terms and assembling the regret bound
We begin with the term  $I_{8}$ , and then proceed to  $I_{6}$ . For the chosen batch of tasks  $\Omega_{K}$  in the round  $K$ , we will have  $f_{1}(\Theta^{(K - 1)}[\Omega_{K}]) = f_{1}\big(\Theta^{(K - 1)} - \eta_{2}\cdot \nabla_{\Theta}\big(\frac{1}{|\Omega_{K}|}\sum_{\mathcal{T}_{K,i}\in \Omega_{K}}\mathcal{L}(D_{K,i}^{q};\Theta_{K,i}^{(J)})\big);\pmb{\theta}_{1}^{(K - 1)}\big)$ , In this case, the average value of estimation sampling probabilities for tasks  $\mathcal{T}_{K,i}\in \Omega_K$  is
$$
\begin{array}{l} \widehat {f} \left(\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]\right) = \widehat {f} _ {1} \left(\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]\right) + \widehat {f} _ {2} \left(\boldsymbol {\Theta} ^ {(K - 1)} [ \Omega_ {K} ]\right) \\ = \frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \left[ f _ {1} \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \mathcal {T} _ {K, i} \right]; \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right) + f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, i} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, i} ^ {q}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) \right] \\ \end{array}
$$
[Bounding the  $f_{1}$  output difference] Next, let us first proceed to bound the output difference with respect to the exploitation module  $f_{1}$ , where we can transform this term into
$$
\begin{array}{l} f _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) - \widehat {f} _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) \\ = f _ {1} \left(\boldsymbol {\Theta} _ {1} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \left(\frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \mathcal {L} \left(D _ {K, i} ^ {q}; \boldsymbol {\Theta} _ {K, i} ^ {(J)}\right)\right); \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right) \\ - \frac {1}{| \Omega_ {K} |} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} f _ {1} \left(\boldsymbol {\Theta} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K, j} ^ {q}; \boldsymbol {\Theta} _ {K, j} ^ {(J)}\right); \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right) \\ = \frac {1}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left(f _ {1} \left(\boldsymbol {\Theta} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \left(\frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \mathcal {L} \left(D _ {K, i} ^ {q}; \boldsymbol {\Theta} _ {K, i} ^ {(J)}\right)\right); \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right) \right. \\ \left. - f _ {1} \left(\boldsymbol {\Theta} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K, j} ^ {q}; \boldsymbol {\Theta} _ {K, j} ^ {(J)}\right); \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right)\right). \\ \end{array}
$$
Then, applying the Lipschitz continuity property will lead to
$$
\begin{array}{l} f _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) - \widehat {f} _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) \\ \leq \frac {\eta_ {2} \cdot \xi_ {L}}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \nabla_ {\boldsymbol {\Theta}} \big (\frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \mathcal {L} (D _ {K, j} ^ {q}; \boldsymbol {\Theta} _ {K, j} ^ {(J)}) \big) - \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D _ {K, i} ^ {q}; \boldsymbol {\Theta} _ {K, i} ^ {(J)}) \| _ {2}. \\ \end{array}
$$
Here, by the definition of the outer-loop optimization of first-order meta-learning, we will have an alternative form the inequality, denoted by
$$
\begin{array}{l} f _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) - \widehat {f} _ {1} \left(\Theta^ {(K - 1)} [ \Omega_ {K} ]\right) \leq \\ \frac {\eta_ {2} \cdot \xi_ {L}}{| \Omega_ {K} |} \cdot \left(\sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \nabla_ {\Theta} \big (\mathcal {L} (D _ {K, j} ^ {q}; \Theta_ {K, j} ^ {(J)}) \big) - \nabla_ {\Theta} \mathcal {L} (D _ {K, i} ^ {q}; \Theta_ {K, i} ^ {(J)}) \| _ {2}\right). \\ \end{array}
$$
For the term in the parentheses on the RHS, substituting the backward operation with the  $H(\cdot ,\cdot)$  mapping function, we have
$$
\begin{array}{l} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \nabla_ {\Theta} \left(\mathcal {L} \left(D _ {K, j} ^ {q}; \Theta_ {K, j} ^ {(J)}\right)\right) - \nabla_ {\Theta} \mathcal {L} \left(D _ {K, i} ^ {q}; \Theta_ {K, i} ^ {(J)}\right) \| _ {2} \\ = \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \nabla_ {\Theta} \left(\mathcal {L} \left(D _ {K, j} ^ {q}; \Theta_ {K, j} ^ {(J)}\right)\right) - \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \nabla_ {\Theta} \mathcal {L} \left(D _ {K, i} ^ {q}; \Theta_ {K, i} ^ {(J)}\right) \| _ {2} \\ \leq \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \| \nabla_ {\Theta} \mathcal {L} (D _ {K, j} ^ {q}; \Theta_ {K, j} ^ {(J)}) - \nabla_ {\Theta} \mathcal {L} (D _ {K, i} ^ {q}; \Theta_ {K, i} ^ {(J)}) \| _ {2} \\ = \frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \| H \left(\mathcal {T} _ {K, i}, \boldsymbol {\Theta} ^ {(K - 1)}\right) - H \left(\mathcal {T} _ {K, j}, \boldsymbol {\Theta} ^ {(K - 1)}\right) \| _ {2} \\ \leq \left| \Omega_ {K} \right| \cdot S _ {1} (K). \\ \end{array}
$$
with  $|\Omega_K| = B$ . Therefore, the  $f_1$  part of error term  $I_8$  could be bounded by  $f_1(\Theta^{(K-1)}[\Omega_K]) - \widehat{f}_1(\Theta^{(K-1)}[\Omega_K]) \leq \eta_2 \cdot \xi_L \cdot B \cdot S_1(K)$ , where the upper bound  $S_1(K) \leq \mathcal{O}(\sqrt{m_{\mathcal{F}}L_{\mathcal{F}}})$  can be found in the procedure bounding term  $I_{4.1}$ .
[Bounding the  $f_{2}$  output difference] Then, with  $\chi_{K} = \Theta^{(K - 1)}[\Omega_{K}]$ , we proceed to bound the output difference with respect to the exploration module, which is represented by
$$
\begin{array}{l} f _ {2} \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right) - \widehat {f} _ {2} \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right) \\ = f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - \frac {1}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {q}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) \\ = \frac {1}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left(f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {q}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right)\right) \\ \end{array}
$$
By adopting the Lipschitz continuity property of  $f_{2}$ , we will have
$$
\begin{array}{l} f _ {2} \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right) - \widehat {f} _ {2} \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \Omega_ {K} \right]\right) \\ \leq \frac {\xi_ {L}}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left\| \left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q}\right) \right] - \left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {s}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {q}\right) \right] \right\| _ {2} \\ \leq \frac {\xi_ {L}}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left\| \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s}\right) - \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {s}\right) \right\| _ {2} + \left\| \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q}\right) - \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K, j} ^ {q}\right) \right\| _ {2} \\ \leq \frac {\xi_ {L} ^ {2}}{| \Omega_ {K} |} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left\| \boldsymbol {\chi} _ {K} ^ {s} - \boldsymbol {\chi} _ {K, j} ^ {s} \right\| _ {2} + \left\| \boldsymbol {\chi} _ {K} ^ {q} - \boldsymbol {\chi} _ {K, j} ^ {q} \right\| _ {2} \\ = \frac {\xi_ {L} ^ {2} \cdot \eta_ {2}}{\left| \Omega_ {K} \right|} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \| \nabla_ {\Theta} \big (\frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \mathcal {L} \left(D _ {K, i} ^ {q}; \Theta_ {K, i} ^ {(J)}\right) - \nabla_ {\Theta} \mathcal {L} \left(D _ {K, j} ^ {q}; \Theta_ {K, j} ^ {(J)}\right) \| _ {2} \\ + \frac {\xi_ {L} ^ {2}}{| \Omega_ {K} |} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \left\| \chi_ {K} ^ {s} - \chi_ {K, j} ^ {s} \right\| _ {2} \\ \leq \eta_ {2} \cdot \xi_ {L} ^ {2} B \cdot S _ {1} (K) + \frac {\xi_ {L} ^ {2} \cdot \eta_ {1}}{| \Omega_ {K} |} \cdot \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \| \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \Theta_ {K, i} ^ {(J)} - \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \Theta_ {K, j} ^ {(J)} \| _ {2} \\ \end{array}
$$
where the last inequality is by applying the conclusion when bounding the output difference w.r.t. the exploitation module  $f_{1}$ . Then, for the second term on the RHS,
$$
\begin{array}{l} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \| \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \boldsymbol {\Theta} _ {K, i} ^ {(J)} - \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \boldsymbol {\Theta} _ {K, j} ^ {(J)} \| _ {2} \leq \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \boldsymbol {\Theta} _ {K, i} ^ {(J)} - \boldsymbol {\Theta} _ {K, j} ^ {(J)} \| _ {2} \\ \leq \frac {1}{| \Omega_ {K} |} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| (\boldsymbol {\Theta} ^ {K - 1} - \sum_ {\tau \in [ \tau ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D _ {K, i} ^ {s}; \boldsymbol {\Theta} _ {K, i} ^ {(\tau)})) - (\boldsymbol {\Theta} ^ {K - 1} - \sum_ {\tau \in [ J ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D _ {K, j} ^ {s}; \boldsymbol {\Theta} _ {K, j} ^ {(\tau)})) \| _ {2} \\ = \frac {1}{\left| \Omega_ {K} \right|} \sum_ {\mathcal {T} _ {K, j} \in \Omega_ {K}} \sum_ {\mathcal {T} _ {K, i} \in \Omega_ {K}} \| \sum_ {\tau \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {K, i} ^ {s}; \Theta_ {K, i} ^ {(\tau)}\right) - \sum_ {\tau \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {K, j} ^ {s}; \Theta_ {K, j} ^ {(\tau)}\right) \| _ {2} \\ \leq | \Omega_ {K} | J \cdot \mathcal {O} (\sqrt {m _ {\mathcal {F}} L _ {\mathcal {F}}}) \\ \end{array}
$$
where the last inequality is due to Lemma D.15. Summing up all the results above will give us the upper bound for  $f_{2}$  output difference  $f_{2}(\Theta^{(K - 1)}[\Omega_{K}]) - \widehat{f}_{2}(\Theta^{(K - 1)}[\Omega_{K}]) \leq \mathcal{O}(\eta_{2} \cdot \xi_{L}^{2}B \cdot \sqrt{m_{\mathcal{F}}L_{\mathcal{F}}} + \eta_{1} \cdot BJ \cdot \sqrt{m_{\mathcal{F}}L_{\mathcal{F}}})$ .
[Similar procedure for term  $I_{6}$ ] Analogously, we can apply the same derivation for the error term  $I_{6}$ , which leads to
$$
\begin{array}{l} I _ {6} = f (\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}) - \hat {f} (\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}) \\ = f _ {1} \left(\boldsymbol {\Theta} ^ {(k - 1), *} - \eta_ {2} \nabla_ {\boldsymbol {\Theta}} \left(\frac {1}{| \Omega_ {K} ^ {*} |} \sum_ {\mathcal {T} _ {i ^ {*}} \in \Omega_ {K} ^ {*}} \mathcal {L} \left(D _ {i ^ {*}} ^ {q}; \boldsymbol {\Theta} _ {i ^ {*}} ^ {(J)}\right)\right); \tilde {\boldsymbol {\theta}} _ {1} ^ {(K - 1)}\right) \\ - \frac {1}{| \Omega_ {K} ^ {*} |} \cdot \sum_ {\mathcal {T} _ {i ^ {*}} \in \Omega_ {K} ^ {*}} f _ {1} \left(\boldsymbol {\Theta} ^ {(k - 1), *} - \eta_ {2} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D _ {i ^ {*}} ^ {q}; \boldsymbol {\Theta} _ {i ^ {*}} ^ {(J)}); \tilde {\boldsymbol {\theta}} _ {1} ^ {(K - 1)}\right). \\ \end{array}
$$
Following a similar procedure as that of term  $I_{8}$  will give us a similar bound as
$$
\begin{array}{l} I _ {6} = f \left(\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - \hat {f} \left(\boldsymbol {\Theta} ^ {(K - 1), *} [ \Omega_ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) \\ \leq \mathcal {O} \left(\eta_ {2} \cdot \xi_ {L} ^ {2} B \cdot \sqrt {m _ {\mathcal {F}} L _ {\mathcal {F}}} + B J \cdot \sqrt {m _ {\mathcal {F}} L _ {\mathcal {F}}} \eta_ {1}\right). \\ \end{array}
$$
where the learning rate  $\eta_1, \eta_2 \leq \mathcal{O}\left(\frac{1}{m_{\mathcal{F}}}\right)$  is a small value. Then, the upper bounds for error terms  $I_6, I_8$  are given as desired.
[Assembling the results] Then, combining all the results, we would have
$$
R (K) \leq \mathcal {O} (\frac {1}{\sqrt {K}}) \cdot \left(\sqrt {2 \xi_ {1}} + \frac {3 L}{\sqrt {2}} + (1 + 2 \gamma_ {1}) \sqrt {2 \log (\frac {K}{\delta})}\right) + \mathcal {O} (\frac {\xi_ {L} ^ {2} K B J \sqrt {L _ {\mathcal {F}}}}{\sqrt {m _ {\mathcal {F}}}}) + \gamma_ {m}
$$
$$
\gamma_ {m} = \left(1 + \mathcal {O} \left(\frac {K L ^ {3} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}\right)\right) \mathcal {O} \left(\frac {K ^ {3} L}{\rho \sqrt {m}} \log (m)\right) + \mathcal {O} \left(\frac {K ^ {4} L ^ {2} \log^ {1 1 / 6} (m)}{\rho^ {4 / 3} m ^ {1 / 6}}\right) + \frac {\xi_ {L} K L ^ {4} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}
$$
Similarly, with proper networks width  $m$  as in Theorem 5.2, the majority of the terms above can be cancelled to  $\mathcal{O}(1)$ . With increasingly large network width  $m$  under the over-parameterization settings,  $\gamma_1, \gamma_m$  will also become diminutive.
D.3 Performance Guarantee for the Exploitation and Exploration Modules
In this subsection, we would like to give the performance guarantee for the proposed BASS framework, and the corresponding performance bound can be applied to derive an upper bound for the error terms  $I_{1}, I_{2}$  for the single-task settings and  $I_{5}, I_{9}$  under the batch settings. Up to meta-training iteration  $k \in [K]$  (before updating the meta-parameters and BASS), we denote all the past records received as  $\mathcal{P}_{k - 1}$ . Before presenting the main lemmas, we first introduce the following operator. Inspired by [3], with two arbitrary vectors  $\tilde{\chi}, \chi$  such that  $\| \tilde{\chi} \|_2 \leq 1$ ,  $\| \chi \|_2 = 1$ , we have the following operator
$$
\phi (\tilde {\boldsymbol {x}}, \boldsymbol {\chi}) = \left(\frac {\tilde {\boldsymbol {\chi}}}{\sqrt {2}}, \frac {\boldsymbol {\chi}}{2}, c\right) \tag {11}
$$
as the concatenation of the two vectors  $\frac{\tilde{\chi}}{\sqrt{2}}, \frac{\chi}{2}$  and one constant  $c$ , where  $c = \sqrt{\frac{3}{4} - (\frac{\|\tilde{\chi}\|_2}{\sqrt{2}})^2} \geq \frac{1}{2}$ . And this operator transforms the transformed vector into unit norm,  $\| \phi (\tilde{\chi},\chi)\| _2 = 1$ . The idea of this operator is to make the gradients  $\nabla_{\theta}f_{1}(\cdot ;\pmb{\theta}_{1})$  of the exploitation model, which is the input of the exploration model  $f_{2}(\cdot ;\pmb{\theta}_{2})$ , comply with the normalization requirement and the separateness assumption (Assumption 5.1). For the sake of analysis, we will adopt this operation in the following proof. Note that this operator is just one possible solution, and our results could be easily generalized to other forms of input gradients under the unit-length and separateness assumption. Similar ideas are also applied in previous works [9]. We begin to bound the single-task settings with the following lemma.
Lemma D.1. For the constants  $c_{g}^{\prime} > 0$ ,  $0 < \rho \leq \mathcal{O}\left(\frac{1}{L}\right)$  and  $\xi_1 \in (0,1)$ , given the past records  $\mathcal{P}_{k-1}$ , we suppose  $m, \eta_1, \eta_2$  satisfy the conditions in Theorem 5.2, and randomly draw the parameter  $\{\pmb{\theta}_1^{(k)}, \pmb{\theta}_2^{(k)}\} \sim \{\widetilde{\pmb{\theta}}_1^{(\tau)}, \widetilde{\pmb{\theta}}_2^{(\tau)}\}_{\tau \in [k]}$ . Consider the past records  $\mathcal{P}_{k-1}$  up to round  $k$  are generated by a fixed policy when witness the candidate arms  $\{\Omega_{task}^{(\tau)}\}_{\tau \in [k]}$ . Then, with probability at least  $1 - \delta$  given an arm-reward pair  $(\mathcal{T}_{k,\widehat{i}}, r_{k,\widehat{i}})$ , we have
$$
\begin{array}{l} \mathbb {E} _ {\mathcal {T} _ {k, i} \sim \mathcal {P} (\mathcal {T})} \left[ | f _ {2} \Bigg (\phi \big (\frac {[ \nabla_ {\pmb {\theta}} f _ {1} (\pmb {\chi} _ {k , \widehat {i}} ^ {s}) ; \nabla_ {\pmb {\theta}} f _ {1} (\pmb {\chi} _ {k , \widehat {i}} ^ {q}) ]}{c _ {g} ^ {\prime} L}, \pmb {\chi} _ {k, \widehat {i}} ^ {q}); \pmb {\theta} _ {2} ^ {(k - 1)} \bigg) - \left(r _ {\tau} - f _ {1} (\pmb {\chi} _ {k, \widehat {i}} ^ {q}; \pmb {\theta} _ {1} ^ {(k - 1)})\right) | \mid \Omega_ {t a s k} ^ {(k)}, \mathcal {P} _ {k - 1} \right] \\ \leq \frac {1}{\sqrt {k}} \cdot \left(\sqrt {2 \xi_ {1}} + \frac {3 L}{\sqrt {2}} + (1 + 2 \gamma_ {1}) \sqrt {2 \log (\frac {k}{\delta})}\right) \\ \end{array}
$$
$$
\gamma_ {1} = 2 + \mathcal {O} \left(\frac {k ^ {3} L}{\rho \sqrt {m}} \log m\right) + \mathcal {O} \left(\frac {L ^ {2} k ^ {4}}{\rho^ {4 / 3} m ^ {1 / 6}} \log^ {1 1 / 6} (m)\right).
$$
Proof. The proof of this lemma is inspired by Lemma C.1 from [9]. First, we can derive the output upper bound