SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} = \left| f \left(\Theta^ {(K - 1), *} \left[ \mathcal {T} _ {K} ^ {*} \right]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - f \left(\Theta^ {(K - 1), *} \left[ \mathcal {T} _ {K} ^ {*} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) \right. \\ + f \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \mathcal {T} _ {K} ^ {*} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) - f \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \mathcal {T} _ {K} ^ {*} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) \\ \end{array}
$$
$$
\begin{array}{l} \leq | f (\boldsymbol {\Theta} ^ {(K - 1), *} [ \mathcal {T} _ {K} ^ {*} ]; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}) - f (\boldsymbol {\Theta} ^ {(K - 1), *} [ \mathcal {T} _ {K} ^ {*} ]; \boldsymbol {\theta} ^ {(K - 1)}) | \\ + \left| f \left(\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \mathcal {T} _ {K} ^ {*} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) - f \left(\boldsymbol {\Theta} ^ {(K - 1)} \left[ \mathcal {T} _ {K} ^ {*} \right]; \boldsymbol {\theta} ^ {(K - 1)}\right) \right|. \\ \end{array}
$$
Then, inserting the inequality will lead to
$$
\begin{array}{l} R (K) \leq \underbrace {| h (\boldsymbol {\Theta} ^ {(K - 1)} [ \mathcal {T} _ {K} ]) - f (\chi_ {K} ; \boldsymbol {\theta} ^ {(K - 1)}) |} _ {I _ {1}} + \underbrace {| f (\chi_ {K} ^ {*} ; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}) - h (\boldsymbol {\Theta} ^ {(K - 1) , *} [ \mathcal {T} _ {K} ^ {*} ]) |} _ {I _ {2}} \\ + \underbrace {\left| f \left(\Theta^ {(K - 1) , *} \left[ \mathcal {T} _ {K} ^ {*} \right] ; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - f \left(\Theta^ {(K - 1) , *} \left[ \mathcal {T} _ {K} ^ {*} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) \right|} _ {I _ {3}} \\ + \underbrace {\left| f \left(\Theta^ {(K - 1) , *} \left[ \mathcal {T} _ {K} ^ {*} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) - f \left(\Theta^ {(K - 1)} \left[ \mathcal {T} _ {K} ^ {*} \right] ; \boldsymbol {\theta} ^ {(K - 1)}\right) \right|} _ {I _ {4}}. \\ \end{array}
$$
Here, the terms  $I_1, I_2$  refer to the approximation error for the two bandit models (our possessed model  $f(\cdot; \pmb{\theta}^{(K-1)})$  and the pseudo model  $f(\cdot; \tilde{\pmb{\theta}}^{(K-1)})$ ). Then, the third term  $I_3$  bounds the output difference when given the same input  $\Theta^{(K-1),*}[\mathcal{T}_K]$  to two separate bandit models, and the final
term  $I_{4}$  refers to the difference of the meta-model parameters when adapted to the same task with two individual sets of parameters. Here, the terms  $I_{1}, I_{2}$  can be bounded by Lemma D.1, Corollary D.2. Then, the point is to bound the difference term  $I_{4}$  when given different inputs to the bandit model.
D.1.1 Bounding error terms and assembling the regret bound
[Bounding term  $I_3$ ] For error term  $I_3$ , it focuses on bounding the output difference between two bandit models  $f(\cdot; \tilde{\pmb{\theta}}^{(K-1)})$ ,  $f(\cdot; \pmb{\theta}^{(K-1)})$  given the same input  $\Theta^{(K-1),*}[\mathcal{T}_K^*]$ , and we have
$$
\begin{array}{l} I _ {3} = \left| \right. f \left( \right.\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \right. \mathcal {T} _ {K} ^ {*}; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\left. \right) - f \left( \right.\boldsymbol {\Theta} ^ {(K - 1), *} \left[ \right. \mathcal {T} _ {K} ^ {*}; \boldsymbol {\theta} ^ {(K - 1)}\left. \right)\left. \right.\left. \right\rvert   = \left| f \left(\boldsymbol {\chi} _ {K} ^ {*}; \tilde {\boldsymbol {\theta}} ^ {(K - 1)}\right) - f \left(\boldsymbol {\chi} _ {K} ^ {*}; \boldsymbol {\theta} ^ {(K - 1)}\right)\right| \\ \leq \underbrace {\left| f _ {1} \left(\chi_ {K} ^ {*} ; \tilde {\boldsymbol {\theta}} _ {1} ^ {(K - 1)}\right) - f _ {1} \left(\chi_ {K} ^ {*}; \boldsymbol {\theta} _ {1} ^ {(K - 1)}\right) \right|} _ {I _ {3. 1}} \\ + \underbrace {| f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {s , *}); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {q , *}) \right]; \tilde {\boldsymbol {\theta}} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {s , *}); \nabla_ {\boldsymbol {\theta}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {q , *}) \right] ; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) | .} _ {I _ {3. 2}} \\ \end{array}
$$
With the defined  $\xi_L$ , applying Lemma D.11 as well as Corollary D.12, we will have
$$
I _ {3. 1} \leq \left(1 + \mathcal {O} \left(\frac {K L ^ {3} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}\right)\right) \cdot \mathcal {O} \left(\frac {K ^ {3} L}{\rho \sqrt {m}} \log (m)\right) + \mathcal {O} \left(\frac {K ^ {4} L ^ {2} \log^ {1 1 / 6} (m)}{\rho^ {4 / 3} m ^ {1 / 6}}\right)
$$
Then, for term  $I_{3,2}$ , we have
$$
\begin{array}{l} I _ {3. 2} = \left| f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \tilde {\boldsymbol {\theta}} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) \right| \\ \leq \left| f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \tilde {\boldsymbol {\theta}} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) \right| \\ + | f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) |. \\ \end{array}
$$
Here, for the first term on the RHS, we apply Lemma D.11 as well as Corollary D.12 to bound.
Then, for the second term, with Gaussian initialization of weight matrices, for the over-parameterized FC network  $f$  with Lipschitz-smooth activation functions (e.g., Sigmoid), we can have  $|f(\pmb{x}) - f(\pmb{x}^{\prime})|, \| \nabla f(\pmb{x}) - \nabla f(\pmb{x}^{\prime}) \| \leq \xi \cdot \| \pmb{x} - \pmb{x}^{\prime} \|$  due to its Lipschitz continuity / smoothness property [47, 18]. Meanwhile, we also have the Lipschitz continuity property for over-parameterized FC network  $f^{\prime}$  with ReLU activation [3], such that  $|f^{\prime}(\pmb{x}) - f^{\prime}(\pmb{x}^{\prime})| \leq \xi^{\prime} \cdot \| \pmb{x} - \pmb{x}^{\prime} \|$ . By the Gaussian initialization of BASS's weight matrices and the properties of over-parameterized neural networks [3, 47, 18], we have  $\xi_{L} = \max \{\xi, \xi^{\prime}\} \leq \mathcal{O}(c_{\xi}^{L})$  being the Lipschitz constant for our  $f_{1}, f_{2}$ , where  $c_{\xi} > 1$  is a small constant. Applying the conclusion above, we will have
$$
\begin{array}{l} \left. \right.\left| f _ {2} \left(\left[ \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right)\right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right)\right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right)\right| \\ \leq \xi_ {L} \cdot \left\| \left[ \nabla_ {\bar {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\bar {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right] - \xi_ {L} \cdot \left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right] \right\| \\ \leq \xi_ {L} \cdot \left\| \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\chi_ {K} ^ {s, *}\right) - \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {s, *}\right) \right\| + \xi_ {L} \cdot \left\| \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {q, *}\right) - \nabla_ {\tilde {\boldsymbol {\theta}}} f _ {1} \left(\chi_ {K} ^ {q, *}\right) \right\| \\ \leq \xi_ {L} \cdot \frac {K L ^ {4} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}} \\ \end{array}
$$
where the last inequality is by Theorem 5 in [3] and the proof of Lemma D.11. With the above results, it will give us
$$
I _ {3} \leq \left(1 + \mathcal {O} \left(\frac {K L ^ {3} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}\right)\right) \mathcal {O} \left(\frac {K ^ {3} L}{\rho \sqrt {m}} \log (m)\right) + \mathcal {O} \left(\frac {K ^ {4} L ^ {2} \log^ {1 1 / 6} (m)}{\rho^ {4 / 3} m ^ {1 / 6}}\right) + \frac {\xi_ {L} K L ^ {4} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}
$$
[Bounding term  $I_4$ ] On the other hand, applying the analogous procedure for term  $I_4$ , denoting  $\chi_K^* = \Theta^{(K-1),*}[\mathcal{T}_K^*]$  and  $\bar{\chi}_K^* = \Theta^{(K-1)}[\mathcal{T}_K^*]$  for the brevity of notation, we can have
$$
\begin{array}{l} I _ {4} = | f (\boldsymbol {\Theta} ^ {(K - 1), *} [ \mathcal {T} _ {K} ^ {*} ]; \boldsymbol {\theta} ^ {(K - 1)}) - f (\boldsymbol {\Theta} ^ {(K - 1)} [ \mathcal {T} _ {K} ^ {*} ]; \boldsymbol {\theta} ^ {(K - 1)}) | \\ \leq \underbrace {\xi_ {L} \cdot \| \Theta^ {(K - 1) , *} [ \mathcal {T} _ {K} ^ {*} ] - \Theta^ {(K - 1)} [ \mathcal {T} _ {K} ^ {*} ] \| _ {2}} _ {I _ {4. 1}} \\ + \underbrace {| f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {s , *}); \nabla_ {\boldsymbol {\theta}} f _ {1} (\boldsymbol {\chi} _ {K} ^ {q , *}) \right] ; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} (\bar {\boldsymbol {\chi}} _ {K} ^ {s , *}); \nabla_ {\boldsymbol {\theta}} f _ {1} (\bar {\boldsymbol {\chi}} _ {K} ^ {q , *}) \right] ; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) | .} _ {I _ {4. 2}} \\ \end{array}
$$
where  $\chi_{K}^{s,*},\chi_{K}^{q,*}$  respectively represents the support set and query set for task  $\mathcal{T}_K^*$  and the metaparameters  $\Theta^{(K - 1),*}$ . Similar notation also applies to  $\bar{\chi}_{K}^{*} = \Theta^{(K - 1)}[\mathcal{T}_{K}^{*}]$ . And the inequality is due to the fact that ReLU networks are naturally Lipschitz continuous w.r.t. some coefficient  $\xi_L$  when they are wide enough [3], as we have discussed above.
[Bounding term  $I_{4,1}$ ] Based on the meta-optimization procedure (inner-loop optimization + outer-loop optimization), we have
$$
\begin{array}{l} I _ {4. 1} = \xi_ {L} \cdot \| \boldsymbol {\Theta} ^ {(K - 1), *} [ \mathcal {T} _ {K} ^ {*} ] - \boldsymbol {\Theta} ^ {(K - 1)} [ \mathcal {T} _ {K} ^ {*} ] \| _ {2} \\ = \xi_ {L} \cdot \| \left(\boldsymbol {\Theta} ^ {(K - 1), *} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K} ^ {q, *}; \boldsymbol {\Theta} _ {K} ^ {(J), *}\right)\right) - \left(\boldsymbol {\Theta} ^ {(K - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K} ^ {q, *}; \boldsymbol {\Theta} _ {K} ^ {(J)}\right)\right) \| _ {2} \\ \end{array}
$$
where  $\Theta_K^{(J),*}$  is the task-specific parameter of  $\mathcal{T}_K^*$  after adapting on  $\Theta^{(K - 1),*}$  with inner-loop optimization, and the  $\Theta_K^{(J)}$  is the similar parameter after adapting on  $\Theta^{(K - 1)}$ . Here, we simplify the formula by representing the gradient derivation (inner-loop + outer-loop) with the mapping  $H:\mathcal{T}\times \Theta \mapsto \mathbb{R}^p$ , which leads to
$$
\begin{array}{l} \left\| \boldsymbol {\Theta} ^ {(K - 1), *} \left[ \mathcal {T} _ {K} ^ {*} \right] - \boldsymbol {\Theta} ^ {(K - 1)} \left[ \mathcal {T} _ {K} ^ {*} \right] \right\| _ {2} \\ = \| \left(\boldsymbol {\Theta} ^ {(K - 1), *} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D ^ {q}; \boldsymbol {\Theta} _ {K} ^ {(J), *})\right) - \left(\boldsymbol {\Theta} ^ {(K - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (D ^ {q}; \boldsymbol {\Theta} _ {K} ^ {(J)})\right) \| _ {2} \\ = \| \left(\boldsymbol {\Theta} ^ {(K - 1), *} - \boldsymbol {\Theta} ^ {(K - 1)}\right) - \eta_ {2} \cdot \left(H \left(\mathcal {T} _ {K} ^ {*}, \boldsymbol {\Theta} ^ {(K - 1), *}\right) - H \left(\mathcal {T} _ {K} ^ {*}, \boldsymbol {\Theta} ^ {(K - 1)}\right)\right) \| _ {2} \\ = \| \left(\boldsymbol {\Theta} ^ {(K - 2), *} - \boldsymbol {\Theta} ^ {(K - 2)}\right) - \eta_ {2} \cdot \left(H \left(\mathcal {T} _ {K} ^ {*}, \boldsymbol {\Theta} ^ {(K - 1), *}\right) - H \left(\mathcal {T} _ {K} ^ {*}, \boldsymbol {\Theta} ^ {(K - 1)}\right)\right) \\ - \eta_ {2} \cdot \left(H \left(\mathcal {T} _ {K - 1} ^ {*}, \boldsymbol {\Theta} ^ {(K - 2), *}\right) - H \left(\mathcal {T} _ {K - 1} ^ {*}, \boldsymbol {\Theta} ^ {(K - 2)}\right)\right) \| _ {2} \\ \leq \sum_ {k \in [ K ]} \eta_ {2} \cdot \left\| H \left(\mathcal {T} _ {k} ^ {*}, \boldsymbol {\Theta} ^ {(k - 1), *}\right) - H \left(\mathcal {T} _ {k} ^ {*}, \boldsymbol {\Theta} ^ {(k - 1)}\right) \right\| _ {2} \\ \end{array}
$$
Recall that the past arms, including the actual chosen arms  $\{\mathcal{T}_1,\mathcal{T}_2,\dots ,\mathcal{T}_K\}$  as well as the optimal ones  $\{\mathcal{T}_1^*,\mathcal{T}_2^*,\ldots ,\mathcal{T}_K^*\}$  are all from the candidate pool where each candidate arm is drawn i.i.d. from the task distribution  $\mathcal{P}(\mathcal{T})$ . Therefore, denoting the bound as  $\| H(\mathcal{T}_K^*,\Theta^{(K - 1),*}) - H(\mathcal{T}_K^*,\Theta^{(K - 1)})\| _2\leq S_1(K)$ , we can have the upper bound as  $I_{4.1}\leq \eta_{2}\cdot \xi_{L}K\cdot S_{1}(K)$ .
Then, for the term  $S_{1}(K)$ , by definition we have  $\| H(\mathcal{T}_K^*,\Theta^{(K - 1),*}) - H(\mathcal{T}_K^*,\Theta^{(K - 1)})\| \leq S_1(K)$ , applying mean-reduction for the sample loss will further leads to
$$
\begin{array}{l} \| H (\mathcal {T} _ {K} ^ {*}, \Theta^ {(K - 1), *}) - H (\mathcal {T} _ {K} ^ {*}, \Theta^ {(K - 1)}) \| = \| \nabla_ {\Theta} \mathcal {L} (D _ {K} ^ {q, *}; \Theta_ {K} ^ {(J), *}) - \nabla_ {\Theta} \mathcal {L} (D _ {K} ^ {q, *}; \Theta_ {K} ^ {(J)}) \| _ {2} \\ = \| \frac {1}{| D _ {K} ^ {q , *} |} \sum_ {\boldsymbol {x} \in D _ {K} ^ {q, *}} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (\boldsymbol {x}; \boldsymbol {\Theta} _ {K} ^ {(J), *}) - \frac {1}{| D _ {K} ^ {q , *} |} \sum_ {\boldsymbol {x} \in D _ {K} ^ {q, *}} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (\boldsymbol {x}; \boldsymbol {\Theta} _ {K} ^ {(J)}) \| _ {2} \\ \leq \frac {1}{| D _ {K} ^ {q , *} |} \sum_ {\boldsymbol {x} \in D _ {K} ^ {q, *}} \| \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (\boldsymbol {x}; \boldsymbol {\Theta} _ {K} ^ {(J), *}) - \nabla_ {\boldsymbol {\Theta}} \mathcal {L} (\boldsymbol {x}; \boldsymbol {\Theta} _ {K} ^ {(J)}) \| _ {2}. \\ \end{array}
$$
This inequality essentially bound the gradient difference when given the same input task  $\mathcal{T}_K^*$  w.r.t. different sets of model parameters. Based on the conclusion from Lemma 9 of [47] and Lemma
B.3 of [11], we have  $\| \nabla_{\Theta_l}f(x;\Theta_K)\| _F,\| \nabla_{\Theta_l}\mathcal{L}(x;\Theta_K)\| _F\leq \mathcal{O}(\sqrt{m_{\mathcal{F}}}),\forall l\in [L_{\mathcal{F}}]$  for any set of parameters within the sphere  $\Theta_K\in \mathcal{B}(\Theta_0,\omega)$  where  $\Theta_0$  is the center and  $\omega$  is the corresponding radius (which is a small value). With a total of  $L_{\mathcal{F}}$  layers for the meta-model and each layer of  $m_{\mathcal{F}}$  hidden units, this will give us  $\| \nabla_{\Theta}\mathcal{L}(\pmb {x};\Theta_{K}^{(J),*})\|_{2},\| \nabla_{\Theta}\mathcal{L}(\pmb {x};\Theta_{K}^{(J)})\|_{2}\leq \mathcal{O}(\sqrt{m_{\mathcal{F}}L_{\mathcal{F}}})$  (Lemma D.15). And this makes  $S_{1}(K)\leq \mathcal{O}(\sqrt{m_{\mathcal{F}}L_{\mathcal{F}}})$  . Since we have  $\eta_1,\eta_2\leq \mathcal{O}(\frac{1}{m_{\mathcal{F}}})$  , summarizing the results above, the upper bound can then be derived.
[Bounding term  $I_{4,2}$ ] Next, we proceed to bound  $I_{4,2}$ , which will be
$$
\begin{array}{l} I _ {4. 2} = \left| f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) - f _ {2} \left(\left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\bar {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\bar {\chi} _ {K} ^ {q, *}\right) \right]; \boldsymbol {\theta} _ {2} ^ {(K - 1)}\right) \right| \\ \leq \xi_ {L} \cdot \left\| \left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\boldsymbol {\chi} _ {K} ^ {q, *}\right) \right] - \left[ \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\tilde {\boldsymbol {\chi}} _ {K} ^ {s, *}\right); \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\tilde {\boldsymbol {\chi}} _ {K} ^ {q, *}\right) \right] \right\| _ {2} \\ \leq \xi_ {L} \cdot \left\| \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {s, *}\right) - \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\bar {\chi} _ {K} ^ {s, *}\right) \right\| _ {2} + \xi_ {L} \cdot \left\| \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\chi_ {K} ^ {q, *}\right) - \nabla_ {\boldsymbol {\theta}} f _ {1} \left(\bar {\chi} _ {K} ^ {q, *}\right) \right\| _ {2} \\ \leq \xi_ {L} ^ {2} \cdot \left\| \chi_ {K} ^ {s, *} - \bar {\chi} _ {K} ^ {s, *} \right\| _ {2} + \xi_ {L} ^ {2} \cdot \left\| \chi_ {K} ^ {q, *} - \bar {\chi} _ {K} ^ {q, *} \right\| _ {2} \\ \end{array}
$$
where the inequalities are due to the Lipschitz continuity / smoothness properties of overparameterized FC networks as we discussed above. Here, we notice that the second term on the RHS can be bounded by directly applying the proving procedure of term  $I_{4,1}$ . Then, for the first term on the RHS, we can following a similar procedure as for  $I_{4,1}$ , by
$$
\begin{array}{l} \left\| \boldsymbol {\chi} _ {K} ^ {s, *} - \bar {\boldsymbol {\chi}} _ {K} ^ {s, *} \right\| _ {2} = \left\| \mathcal {I} \left(\mathcal {T} _ {k} ^ {*}, \boldsymbol {\Theta} ^ {(k - 1), *}\right) - \mathcal {I} \left(\mathcal {T} _ {k} ^ {*}, \boldsymbol {\Theta} ^ {(k - 1)}\right) \right\| _ {2} \\ = \| \left(\boldsymbol {\Theta} ^ {(K - 1), *} - \eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K} ^ {s, *}; \boldsymbol {\Theta} _ {K} ^ {(j), *}\right)\right) - \left(\boldsymbol {\Theta} ^ {(K - 1)} - \eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K} ^ {s, *}; \boldsymbol {\Theta} _ {K} ^ {(j)}\right)\right) \| _ {2} \\ = \| \left(\boldsymbol {\Theta} ^ {(K - 2), *} - \boldsymbol {\Theta} ^ {(K - 2)}\right)) - \left(\eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K - 1} ^ {s, *}; \boldsymbol {\Theta} _ {K - 1} ^ {(j)}\right) - \eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\boldsymbol {\Theta}} \mathcal {L} \left(D _ {K - 1} ^ {s, *}; \boldsymbol {\Theta} _ {K - 1} ^ {(j), *}\right) \right. \\ - \left(\eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {K} ^ {s, *}; \Theta_ {K} ^ {(j)}\right) - \eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {K} ^ {s, *}; \Theta_ {K} ^ {(j), *}\right) \| _ {2} \right. \\ \leq \eta_ {1} \cdot \sum_ {k \in [ K ]} \| \sum_ {j \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {k} ^ {s, *}; \Theta_ {k} ^ {(j)}\right) - \eta_ {1} \cdot \sum_ {j \in [ J ]} \nabla_ {\Theta} \mathcal {L} \left(D _ {k} ^ {s, *}; \Theta_ {k} ^ {(j), *}\right) \| _ {2} \\ \leq \mathcal {O} \left(\eta_ {1} \cdot K J \sqrt {m _ {\mathcal {F}} L _ {\mathcal {F}}}\right) \\ \end{array}
$$
where the last inequality is due to Lemma D.15 and by iterating through  $K$  meta-training iterations. Summing up the results above, we will have  $I_{4.2} \leq \mathcal{O}(\eta_2\xi_L^2 \cdot K\sqrt{m_{\mathcal{F}}L_{\mathcal{F}}}) + \mathcal{O}(\eta_1\xi_L^2 \cdot KJ\sqrt{m_{\mathcal{F}}L_{\mathcal{F}}})$ .
[Summing up the results] Then, combining all the results, we would have
$$
R _ {\text {s i n g l e}} (K) \leq \mathcal {O} \left(\frac {1}{\sqrt {K}}\right) \cdot \left(\sqrt {2 \xi_ {1}} + \frac {3 L}{\sqrt {2}} + (1 + 2 \gamma_ {1}) \sqrt {2 \log (\frac {K}{\delta})}\right) + \mathcal {O} \left(\frac {\xi_ {L} ^ {2} K J \sqrt {L _ {\mathcal {F}}}}{\sqrt {m _ {\mathcal {F}}}}\right) + \gamma_ {m}
$$
$$
\gamma_ {1} = 2 + \mathcal {O} \left(\frac {K ^ {3} L}{\rho \sqrt {m}} \log m\right) + \mathcal {O} \left(\frac {L ^ {2} K ^ {4}}{\rho^ {4 / 3} m ^ {1 / 6}} \log^ {1 1 / 6} (m)\right)
$$
$$
\gamma_ {m} = \left(1 + \mathcal {O} (\frac {K L ^ {3} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}})\right) \mathcal {O} (\frac {K ^ {3} L}{\rho \sqrt {m}} \log (m)) + \mathcal {O} \left(\frac {K ^ {4} L ^ {2} \log^ {1 1 / 6} (m)}{\rho^ {4 / 3} m ^ {1 / 6}}\right) + \frac {\xi_ {L} K L ^ {4} \log^ {5 / 6} (m)}{\rho^ {1 / 3} m ^ {1 / 6}}
$$
Note that the majority of the terms above can be cancelled to  $\mathcal{O}(1)$  with proper networks width  $m$  indicated in Theorem 5.2. With increasingly large network width  $m$ , these terms will also become diminutive enough to achieve our regret bound in the main body.
D.2 Extending the result to the batch settings (Proof of Theorem 5.2)
With the results and conclusions from Subsection D.1, we proceed to provide the proof of Theorem 5.2 under the batch settings. Recall that in our original problem formulation and Algorithm 1, we are expected to select a batch of  $B$  arms in each meta-training iteration, denoted by  $\{\Omega_k\}_{k\in [K]}$ . Note that each of the candidate arms from  $\Omega_{\mathrm{task}}^{(k)}$  are drawn i.i.d. from the task distribution  $\mathcal{P}(\mathcal{T})$ . Meantime,
we will have the corresponding optimal arm batches, denoted by  $\{\Omega_k^*\}_{k\in [K]}$ , which minimizes the loss objective in Eq. 5. Recall that we update the meta-model parameters with
$$
\boldsymbol {\Theta} ^ {(k)} = \boldsymbol {\Theta} ^ {(k - 1)} - \eta_ {2} \cdot \nabla_ {\boldsymbol {\Theta}} \left(\frac {1}{| \Omega_ {k} |} \sum_ {\mathcal {T} _ {k, i} \in \Omega_ {k}} \mathcal {L} (D _ {k, i} ^ {q}; \boldsymbol {\Theta} _ {k, i} ^ {(J)})\right) = \boldsymbol {\Theta} ^ {(k - 1)} - \frac {\eta_ {2}}{| \Omega_ {k} |} \sum_ {\mathcal {T} _ {k, i} \in \Omega_ {k}} \nabla_ {\boldsymbol {\Theta}} \left(\mathcal {L} (D _ {k, i} ^ {q}; \boldsymbol {\Theta} _ {k, i} ^ {(J)})\right)
$$
where  $\Theta_{k,i}^{(J)}$  is the task-specific parameter for  $\mathcal{T}_{k,i}$  after the inner-loop optimization for  $J$  steps.
Analogously, for the notation brevity and the sake of analysis, we denote  $f(\Theta^{(K - 1)}[\Omega_K];\pmb{\theta}^{(k - 1)}) = f_1(\pmb{\chi}_k^q;\pmb{\theta}_1^{(k - 1)}) + f_2\left([ \nabla_\theta f_1(\pmb{\chi}_k^s);\nabla_\theta f_1(\pmb{\chi}_k^q)];\pmb{\theta}_2^{(k - 1)}\right)$  where we have  $\pmb{\chi}_k^q \coloneqq \Theta^{(K - 1)}[\Omega_K]$  being the meta-parameters adapted to batch of tasks  $\Omega_K$ , and the batch-specific parameter is defined as  $\chi_k^s \coloneqq \frac{1}{|\Omega_K|}\sum_{T_{k,\hat{i}}\in \Omega_K}[\Theta_{k,\hat{i}}^{(J)}]$ . Then, the regret under the batch setting can be denoted by