SlowGuess/NeurIPS_2023_OCR huggingface dataset:
MTRL can be seen as a more specific case of meta-learning in Denevi et al. [2019], Khodak et al. [2019], which has the type of representation learning we study frequently called as "feature learning." It is possible to give guarantees for excess transfer risk in this more general setting. For example, the work of Denevi et al. [2019], under a generative model assumption, is restricted to linear predictors and convex losses. In their Thm. 5, they get a rate of  $O\left(\frac{1}{\sqrt{n}} + \frac{1}{\sqrt{t}}\right)$ , on excess transfer risk. However, for the feature learning setting, as they point out, this rate contains terms with hidden dependence on  $n$  and  $t$ . Their guarantee for feature learning, with linear representations (which is more restrictive than ours) Corollary 7, gets a rate of  $O\left(\frac{1}{\sqrt{n}} + \frac{1}{t^{1/4}}\right)$ .
Another work of Khodak et al. [2019] mainly considers two settings of convex and strongly convex losses, respectively. Interestingly, fast rates in terms of the number of tasks can be obtained. Thm. 5.1 in the work, obtains the following rates,  $O\left(\frac{1}{\sqrt{n}} + \frac{1}{\sqrt{nt}}\right)$  for convex Lipschitz losses and  $O\left(\frac{1}{n} + \frac{1}{\sqrt{nt}}\right)$  for strongly-convex Lipschitz losses.