SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Shengzhuang Chen<sup>1</sup> Long-Kai Huang<sup>2</sup> Jonathan Richard Schwarz<sup>3</sup> Yilun Du<sup>4</sup> Ying Wei<sup>1,5*</sup>
$^{1}$ City University of Hong Kong  $^{2}$ Tencent AI Lab  $^{3}$ University College London  $^{4}$ Massachusetts Institute of Technology  $^{5}$ Nanyang Technological University szchen9-c@my.cityu.edu.hk {hlongkai, schwarzjn}@gmail.com yilundu@mit.edu ying.wei@ntu.edu.sg
The success of meta-learning on out-of-distribution (OOD) tasks in the wild has proved to be hit-and-miss. To safeguard the generalization capability of the meta-learned prior knowledge to OOD tasks, in particularly safety-critical applications, necessitates detection of an OOD task followed by adaptation of the task towards the prior. Nonetheless, the reliability of estimated uncertainty on OOD tasks by existing Bayesian meta-learning methods is restricted by incomplete coverage of the feature distribution shift and insufficient expressiveness of the meta-learned prior. Besides, they struggle to adapt an OOD task, running parallel to the line of cross-domain task adaptation solutions which are vulnerable to overfitting. To this end, we build a single coherent framework that supports both detection and adaptation of OOD tasks, while remaining compatible with off-the-shelf meta-learning backbones. The proposed Energy-Based Meta-Learning (EBML) framework learns to characterize any arbitrary meta-training task distribution with the composition of two expressive neural-network-based energy functions. We deploy the sum of the two energy functions, being proportional to the joint distribution of a task, as a reliable score for detecting OOD tasks; during meta-testing, we adapt the OOD task to in-distribution tasks by energy minimization. Experiments on four regression and classification datasets demonstrate the effectiveness of our proposal.
Meta-learning [48, 6] that builds general-purpose learners with limited data has been under constant investigation, recently demonstrating its potential to even advance few-shot learning of large language models [36, 44]. Analogous to the notorious domain shift [23] that degrades the performance of deep learning, meta-testing tasks that are out of the distribution of meta-training tasks (a.k.a. out-of-distribution (OOD) tasks) put the meta-trained prior knowledge at high risk of losing effectiveness [46]. In real-world applications, though, out-of-distribution tasks are highly prevalent, e.g., bin picking for a robot that has never been meta-trained on environments involving bins [55], MRI-based pancreas segmentation given a host of meta-training tasks with pathology images [35], and etc. Thus, it is imperative to secure the generalization ability of the meta-learned prior (i.e., meta-generalization) to OOD tasks, especially in safety-critical applications such as medical image analysis.
The first step to securing meta-generalization to a task is to develop awareness of whether the task is OOD or not, i.e., OOD task detection. Existing solutions in literature have pursued a variety of Bayesian meta-learning methods [7, 54, 41, 10, 43] that balance between flexibility and tractability of
*Correspondence to Ying Wei
Figure 1: Comparison of EBML and Bayesian meta-learning baselines for OOD detection.
solving the hierarchical probabilistic model  $p(\mathbf{Y}_i|\mathbf{X}_i) = \iint p(\mathbf{Y}_i|\mathbf{X}_i,\phi_i)p(\phi_i|\theta)p(\theta)d\phi_id\theta$ , where  $\mathcal{T}_i = \{\mathbf{X}_i,\mathbf{Y}_i\}$  represents the  $i$ -th task.  $\theta$  and  $\phi_i$  denote parameters of the meta-model and task-specific model, respectively. Unfortunately, these methods present some limitations in their practical usage. (1) Incomplete OOD coverage: given that the Bayesian uncertainty is trained via maximizing the posterior  $p(\mathbf{Y}_i|\mathbf{X}_i)$  above, it is not necessarily high when encountering an OOD task that shares the predictive function  $p(\mathbf{Y}_i|\mathbf{X}_i)$  with some meta-training tasks but differs substantially in feature distributions  $p(\mathbf{X}_i)$ . We verify this in Figure 1a and Appendix D. (2) Limited expressiveness: for tractability purpose, the meta-learned prior  $p(\phi_i|\theta)$  predicates on simple known distributions, e.g., Maximum A Posterior (MAP) estimation [6, 53] and Gaussian [41, 7, 43], which may struggle to align with the complex probabilistic structure of the meta-training task distribution (see Figure 1b). This misalignment inevitably leads to unreliable estimation of OOD tasks.
Upon detection of an OOD task, secondly, adaptation of the meta-learned prior promotes its generalization to this OOD task. We dub this strategy during meta-testing as OOD task adaptation, which is closely related to cross-domain meta-learning [4, 25, 34, 49]. The core philosophy behind cross-domain meta-learning is the introduction of task-specific parameters which are inferred via either gradient descent [28, 29] or feed-forward amortized encoder [42, 8] on the support set of each OOD task. Learning task-specific parameters, however, is prone to overfitting given the usually very limited size of a support set (e.g., 5 examples only in 5-way 1-shot classification).
The limitations are further complicated by the detachment of the existing solution to OOD task detection from that to OOD task adaptation. An explicit prior model is absent in existing Bayesian meta-learning methods for OOD task detection, so that adapting the prior during meta-testing to accommodate an OOD task is ambitious to achieve. On the other hand, cross-domain meta-learning approaches by design do not offer uncertainty estimation, thereby being a risky OOD task detector. Pursuing a coherent framework that supports both detection and adaptation of OOD tasks remains an open question, which motivates our proposal of a novel probabilistic meta-learning framework.
By virtue of the flexibility and expressiveness of energy-based models [24] in modelling complex data distributions, we propose the Energy-Based Meta-Learning (EBML) framework that overcomes the above-mentioned limitations. Specifically, we derive an energy-based model to explicitly model any meta-training task distribution, resulting in the composition of an explicit prior energy function and a complexity energy function. The sum of the two energy functions, trained directly to meet the joint distribution  $p(\mathbf{X}_i,\mathbf{Y}_i)$  and parameterized with neural networks, has completeness and expressiveness advantages that give it an edge in detection of OOD tasks. During meta-testing, we iteratively update the parameter for a task that has been identified as OOD by gradient descent of energy minimization, which eventually adapts the prior towards in-distribution tasks and maximally leverages the meta-learned prior for alleviating overfitting.
The key contributions of this research are outlined below. (1) Coherence and generality: we provide a coherent probabilistic model that allows both detection and adaptation of OOD tasks. Also, EBML is agnostic to meta-learning backbones, being general to secure meta-generalization for arbitrary off-the-shelf meta-learning approaches against OOD tasks. (2) Practical efficacy: we conduct our experiments on three regression and one classification datasets, on which EBML outperforms SOTA Bayesian meta-learning methods for OOD task detection with an improvement of up to  $7\%$  on AUROC and cross-domain meta-learning approaches for OOD task adaption with up to  $1.5\%$  improvement.
Bayesian Meta-learning There has been a line of literature on Bayesian meta-learning algorithms with predictive uncertainty estimation for safeguarding safety-critical and few-shot applications. Grant
et al. [11] first recast gradient-based meta-learning as a tractable hierarchical Bayesian inference problem. Much of the subsequent research attempts to solve the problem with various approximations. Assuming a sufficient number of meta-training tasks, almost all works use a point estimate for the initialization [41, 19, 10]. However, estimates of exceptions including [54] rely on SVGD [32] for inference and require significant computation for an ensemble of task-specific weights. Several studies that estimate the uncertainty in task-specific parameters after inner-loop adaptation have explored MAP estimates [e.g. 47], sampling from a neural network [10, 53, 42], and variational inference [41, 7, 43]. The uncertainties considered in these methods are often modelled using isotropic Gaussians which suffer from limited expressiveness.
Meta-learning towards OOD Generalization Recent cross-domain meta-learning methods [e.g. 25, 4, 49, 34] deal with a distribution shift between meta-training and meta-testing tasks, by typically parameterizing deep networks with a large set of task-agnostic and a small set of task-specific weights that encode shared representations and task-specific representations for the training domains, respectively. The works of [42, 1, 34] augment a shared pre-trained backbone with task-specific FiLM [40] layers whose parameters are estimated through an encoder network conditioned on the task's support set. TSA [28] and URL [29] propose to attach task-specific adaptors in matrix form to the pre-trained backbone at test time, inferring their parameters by gradient descent on the support set for each task from scratch. On the other hand, SUR [4] and URT [31] pre-train multiple backbones each for an ID training domain, and meta-learn an attention mechanism to selectively combine the pre-trained representations into task-specific ones for ID and OOD classification. While these methods generally have improved performance in the OOD domains of tasks, they nevertheless are not designed with any explicit mechanism for detecting OOD tasks, i.e., lacking OOD awareness.
EBMs for OOD Detection Recently, there has been increasing interest in leveraging EBMs for detecting testing samples that are OOD w.r.t. the training data distribution. Liu et al. [33] directly use the energy score for OOD input detection, while Grathwohl et al. in JEM [12] use gradient norm of the energy function as an alternative OOD score; both yield more superior OOD detection performance than traditional density-based detection methods. There are also a number of works that investigate the OOD detection capability of hybrid and latent variable EBMs [38, 14, 13], and more advanced training techniques for improving the density modelling hence OOD detection performance of EBMs [5, 2, 57, 3]. While all aforementioned works focus on the standard supervised and unsupervised learning scenarios, Willette et al. in [52] study OOD detection in meta-learning. However, their work differs from EBML in that (a) EBML aims to detect a meta-testing task that is OOD of the meta-training tasks whereas [52] focuses on detecting a query sample that is OOD of the support samples in a meta-testing task, and (b) EBML explicitly meta-learns the distribution of meta-training tasks via the two proposed EBMs and develops the Energy Sum to flag those high-energy tasks as OOD tasks; while [52] resorts to post-hoc OOD detection via energy scaling (akin to temperature scaling in softmax output) without learning any EBM. Moreover, we offer EBML as a generic and flexible probabilistic meta-learning framework that supports both detection and adaptation of OOD tasks.
3 Preliminaries: Energy-based Models
An energy-based model (EBM) [24] expresses a probability density  $p(\mathbf{x})$  for  $\mathbf{x} \in \mathbb{R}^D$  as
$$
p _ {\theta} (\mathbf {x}) = \frac {\exp \left(- E _ {\theta} (\mathbf {x})\right)}{Z (\theta)}, \tag {1}
$$
where  $E_{\theta}(\mathbf{x})$  is the energy function parametrized by  $\theta$  that maps each point  $\mathbf{x}$  in the input space to a scalar value known as the energy.  $Z(\theta) = \int_{\mathbf{x}}\exp (-E_{\theta}(\mathbf{x}))d\mathbf{x}$  is the partition function that is a constant w.r.t. the variable  $\mathbf{x}$ . Training  $p_{\theta}(\mathbf{x})$  to fit some data distribution  $p_D(\mathbf{x})$  requires maximizing the log-likelihood  $\mathcal{L}(\theta) = \mathbb{E}_{\mathbf{x}\sim p_D(\mathbf{x})}[\log p_\theta (\mathbf{x})]$  w.r.t.  $\theta$ . Though an intractable integral in  $Z_{\theta}$  is involved in this objective, it is not a concern when computing the gradient [3, 12]
$$
\nabla_ {\theta} \mathcal {L} = \mathbb {E} _ {\mathbf {x} ^ {\prime} \sim p _ {\theta}} \left[ \nabla_ {\theta} E _ {\theta} \left(\mathbf {x} ^ {\prime}\right) \right] - \mathbb {E} _ {\mathbf {x} \sim p _ {D}} \left[ \nabla_ {\theta} E _ {\theta} (\mathbf {x}) \right]. \tag {2}
$$
Intuitively, Eqn. (2) encourages  $E_{\theta}$  to assign low energy to the samples from the real data distribution  $p_{D}$  while assigning high energy to those from the model distribution  $p_{\theta}$ . Computing Eqn. (2), thus, requires drawing samples from  $p_{\theta}$ , which is challenging. Recent approaches [12, 3] on training EBMs resort to stochastic gradient Langevin dynamics (SGLD) [51] which generates samples following
$$
\mathbf {x} ^ {0} \sim p _ {0} (\mathbf {x}), \quad \mathbf {x} ^ {k + 1} = \mathbf {x} ^ {k} - \frac {\eta^ {2}}{2} \frac {\partial E _ {\theta} \left(\mathbf {x} ^ {k}\right)}{\partial \mathbf {x} ^ {k}} + \eta \mathbf {z} ^ {k}. \tag {3}
$$
The  $K$ -step sampling starts from an (typically uniform) initial distribution  $p_0(\mathbf{x})$ .  $\mathbf{z}^k \sim \mathcal{N}(\mathbf{0}, \mathbf{I}) \in \mathbb{R}^D$  is a perturbation, and  $\eta \in \mathbb{R}^+$  controls the step size and noise magnitude. Denote the distribution  $q_{\theta}$  by Eqn. (3), which signifies  $\mathbf{x}' = \mathbf{x}^{K} \sim q_{\theta}$ . When  $\eta \rightarrow 0$  and  $K \rightarrow \infty$ , then  $q_{\theta} \rightarrow p_{\theta}$  under some regularity conditions [51]. Consequently, the gradient of Eqn. (2) is approximated in practice [3, 12] by
$$
\nabla_ {\theta} \mathcal {L} = \mathbb {E} _ {\mathbf {x} ^ {\prime} \sim \operatorname {s t o p} _ {-} \operatorname {g r a d} (q _ {\theta})} \left[ \nabla_ {\theta} E _ {\theta} (\mathbf {x} ^ {\prime}) \right] - \mathbb {E} _ {\mathbf {x} \sim p _ {D}} \left[ \nabla_ {\theta} E _ {\theta} (\mathbf {x}) \right], \tag {4}
$$
where the gradient does not back-propagate into SGLD sampling.
4 Energy-Based Meta-learning
For clarity, we use the notation  $\mathcal{P}_{ID}$  to denote the unknown meta-training ID task distribution where the  $i$ -th training task is  $\mathcal{T}_i$ . We let  $\mathbf{X}_i$ ,  $\mathbf{Y}_i$  to denote sets of samples  $\{\mathbf{x}_{ij}, y_{ij}\}$  in  $\mathcal{T}_i$ , and  $\mathcal{T}_i^s$ ,  $\mathcal{T}_i^q$  to denote support and query sets, respectively. The size of  $\mathcal{T}_i$ ,  $\mathcal{T}_i^s$ ,  $\mathcal{T}_i^q$  is denoted by  $N_i$ ,  $N_i^s$ ,  $N_i^q$ , respectively. The subscript  $i$  denotes the task index, and  $j$  denotes the sample index.
4.1 Energy-based Modelling of Task Distribution
As illustrated in Introduction, existing probabilistic meta-learning methods maximizing the predictive likelihood  $p(\mathbf{Y}|\mathbf{X})$  suffer from incomplete OOD coverage. To this end, we model the meta-training task distribution by (1) formulating the joint distribution  $p(\mathbf{X}_i,\mathbf{Y}_i)$  of each task  $\mathcal{T}_i$  and (2) maximizing the log-likelihood of all meta-training tasks. Concretely, by Kolmogorov's extension and de Finneti's theorems [22], we have the expected log-likelihood of the meta-training tasks as  $\mathbb{E}_{\mathcal{P}_{ID}}[\log p(\mathcal{T}_i)] = \mathbb{E}_{\mathcal{P}_{ID}}[\log p(\mathbf{X}_i,\mathbf{Y}_i)] = \mathbb{E}_{\mathcal{P}_{ID}}[\log \int_{\phi_i}\prod_{j = 1}^{N_i}p(\mathbf{x}_{ij},y_{ij}|\phi_i)p(\phi_i)d\phi_i]$ . Each  $p(\mathcal{T}_i)$  is written in a factorized form over  $N_{i}$  conditional independent distributions with  $\phi_{i}$  being the task-specific latent variable. Due to the intractable integral over  $\phi_{i}$  in high dimension, we resort to amortized inference [8, 41] and learn with a lower-bound instead. This gives the ELBO
$$
\mathbb {E} _ {\mathcal {P} _ {I D}} \left[ \log p \left(\mathcal {T} _ {i}\right) \right] \geq \mathbb {E} \left[ \mathbb {E} _ {\boldsymbol {\phi} _ {i} \sim q _ {\psi} \left(\boldsymbol {\phi} _ {i} \mid \mathcal {T} _ {i} ^ {s}\right)} \left[ \log \prod_ {j = 1} ^ {N _ {i}} p \left(\mathbf {x} _ {i j}, y _ {i j} \mid \boldsymbol {\phi} _ {i}\right) \right] - \mathrm {K L} \left(q _ {\psi} \left(\boldsymbol {\phi} _ {i} \mid \mathcal {T} _ {i} ^ {s}\right) | | p \left(\boldsymbol {\phi} _ {i}\right)\right) \right]. \tag {5}
$$
Following the conventional wisdom [41, 28, 6],  $q_{\psi}$  is conditioned on the support set only during meta-training to align the inference procedure, i.e.,  $\phi_i \sim q_{\psi}(\phi_i | \mathcal{T}_i^s)$ , for meta-training and meta-testing. It remains now to parameterize the three distributions in Eqn. (5) including (a) the task-specific data distribution  $p(\mathbf{x}_{ij}, y_{ij} | \phi_i)$ , (b) the prior latent distribution  $p(\phi_i)$ , and (c) the posterior latent distribution  $q_{\psi}(\phi_i | \mathcal{T}_i^s)$ . Prior works parameterize these distributions in simple known forms, e.g., Gaussians [41, 7, 43] or MAP estimation [6, 53], which may be insufficient to match the complex probabilistic structure of the meta-training task distribution. To increase the expressiveness, we turn to EBMs for parameterizing the two distributions of  $p(\mathbf{x}_{ij}, y_{ij} | \phi_i)$  and  $p(\phi_i)$ . For one reason, EBMs are known to be sufficiently flexible and expressive for characterizing complex arbitrary density functions [3] not limiting to only uni-modal distributions like isotropic Gaussians and MAP estimation; for another, the energy function of an EBM is directly proportional to the negative log-likelihood, paving the way for OOD detection in Section 4.2.
(a) Task-specific data EBM We model  $p(\mathbf{x}_{ij}, y_{ij}|\phi_i)$  by an energy function parameterized with  $\omega$ ,
$$
p \left(\mathbf {x} _ {i j}, y _ {i j} \mid \phi_ {i}\right) = p _ {\omega} \left(\mathbf {x} _ {i j}, y _ {i j} \mid \phi_ {i}\right) = \frac {\exp \left(- E _ {\omega} \left(\mathbf {x} _ {i j} , y _ {i j} , \phi_ {i}\right)\right)}{Z \left(\omega , \phi_ {i}\right)}, \tag {6}
$$
where  $E_{\omega}$  denotes the task-specific data energy function conditioned on the latent  $\phi_{i}$ , and  $Z(\omega, \phi_{i})$  is the corresponding partition function. Note that the parameter  $\omega$  of this EBM is shared by all tasks.
(b) Latent prior EBM Inspired by [39], we model the prior latent distribution  $p(\phi_i)$  as an unconditional EBM parameterized by  $\lambda$ ; training such a EBM offers expressiveness benefits over a fixed non-informative prior distribution, e.g., isotropic Gaussian distribution. Specifically,
$$
p \left(\phi_ {i}\right) = p _ {\lambda} \left(\phi_ {i}\right) = \frac {\exp \left(- E _ {\lambda} \left(\phi_ {i}\right)\right)}{Z (\lambda)}, \forall i. \tag {7}
$$
(c) Latent posterior As many meta-learning algorithms have already carefully designated the posterior latent distribution  $q_{\psi}(\phi_i|\mathcal{T}_i^s)$ , we simply follow the same implementation of  $q_{\psi}$  in the chosen base meta-learning algorithm, e.g., MAP estimation in [8, 42, 1, 53]. This design favorably empowers EBML to be a generic and flexible framework compatible with off-the-shelf meta-learning algorithms.
Grounded on the above parameterization, we are now ready to derive our EBML meta-training objective as below by plugging the two EBMs defined in Eqn. (6) and Eqn. (7) into Eqn. (5). The derivation shares the spirit with Eqn. (4), and more details can be found in Appendix A.1.
$$
\begin{array}{l} \underset {\omega , \psi , \lambda} {\arg \max} \mathbb {E} _ {\mathcal {T} _ {i} \sim \mathcal {P} _ {I D}} \left[ \mathbb {E} _ {\phi_ {i} \sim q _ {\psi} (\phi_ {i} | \mathcal {T} _ {i} ^ {s})} [ \sum_ {j = 1} ^ {N _ {i}} - E _ {\omega} (\mathbf {x} _ {i j}, y _ {i j}, \phi_ {i}) + \mathbb {E} _ {p _ {\omega} (\mathbf {x} ^ {\prime}, y ^ {\prime} | \phi_ {i})} [ E _ {\omega} (\mathbf {x} _ {i j} ^ {\prime}, y _ {i j} ^ {\prime}, \phi_ {i}) ] ] \right] \\ \left. - \mathbb {E} _ {q _ {\psi} \left(\phi_ {i} \mid \mathcal {T} _ {i} ^ {s}\right)} \left[ E _ {\lambda} \left(\phi_ {i}\right) \right] + \mathbb {E} _ {p _ {\lambda} \left(\phi_ {i} ^ {\prime}\right)} \left[ E _ {\lambda} \left(\phi_ {i} ^ {\prime}\right) \right] + \mathcal {H} \left(q _ {\psi} \left(\phi_ {i} \mid \mathcal {T} _ {i} ^ {s}\right)\right) \right]. \tag {8} \\ \end{array}
$$
Solving the above meta-training objective involves sampling of  $\mathbf{x}'$ ,  $y'$  from  $p_{\omega}$  and  $\phi_i'$  from  $p_{\lambda}$ , in order to compute the expectations  $\mathbb{E}_{p_{\omega}(\mathbf{x}', y'| \phi_i)}$  and  $\mathbb{E}_{p_{\lambda}(\phi_i')}$  as Monte-Carlo averages. We follow the similar SGLD sampling procedure in Eqn. (3). Besides, since the majority of state-of-the-art meta-learning algorithms [8, 42, 1, 53] adopt the MAP estimation of the latent posterior  $q_{\psi}$  which is deterministic, the last entropy term of  $\mathcal{H}$  essentially becomes zero and the expectations in the first and second terms are trivial to solve. For this reason, we focus on base meta-learning algorithms with MAP approximation in the following sections, which not only simplifies computation but also maintains the state-of-the-art performance. We left a discussion on EBML with distributional  $q_{\psi}$  in Appendix C.3. The complete pseudo codes for meta-training of EBML are available in Appendix E.
4.2 EBML for OOD Detection
Detecting an OOD task w.r.t. the meta-training distribution constitutes an essential first step to guard successful meta-generalization. A straightforward solution is density-based OOD detection, for which the OOD score of a task following the Bayesian principle boils down to its log-likelihood  $\log p(\mathbf{X}_i^s,\mathbf{Y}_i^s) = \log \mathbb{E}_{\phi_i\sim p_\lambda (\phi_i)}[p_\omega (\mathbf{X}_i^s,\mathbf{Y}_i^s |\phi_i)]$  Despite the meta-learned latent prior EBM  $p_{\lambda}(\phi_{i})$  that is readily available, estimating this log-likelihood still presents daunting challenges. First, when the latent prior is expressed in the form of a distribution over model parameters in very high dimension, MCMC sampling from  $p_{\lambda}(\phi_{i})$  is almost computationally infeasible. Second, especially when the latent prior exhibits multi-modality, drawing a considerable number of samples to achieve a low-variance MC estimation of the integral is prohibitively costly.
On this account, we define the OOD score of a task to be faithful to our proposed ELBO approximation of its log-likelihood in Eqn. (5), which gives
Figure 2: The roles of  $E_{\omega}$  and  $E_{\lambda}$  in Energy Sum in detecting OOD tasks. Each dot denotes a task. Left: We perturb each support sample of a task  $\mathcal{T}_i$  by  $\pmb{\eta}_{ij} \sim \mathcal{N}_i(0, \sigma_i)$  where we sample  $\sigma_i$  from  $[0,1]$  uniformly. The  $y$ -axis shows the average energy  $\mathbb{E}_{\mathbf{x}_{ij}^s, \mathbf{y}_{ij}^s} \sim \mathcal{T}_i^s, \pmb{\eta}_{ij} \sim \mathcal{N}_i[E_{\omega}(\mathbf{x}_i^s, \mathbf{y}_i^s, \pmb{\phi}_i)]$  and the  $x$ -axis plots the variance  $\sigma_i^2$ . Right: We first compute the mean of the overall ID task latent prior as  $\phi_{ID} = \mathbb{E}_{\phi_i \sim p_{ID}}[\phi_i]$ . The  $y$ -axis shows the energy  $E_{\lambda}(\phi_{ID} + \pmb{\eta}_i)$  where  $\pmb{\eta}_i \sim \mathcal{N}(0,1)$  for the  $i$ -th task and the  $x$ -axis plots the Euclidean distance of the perturbed latent from  $\phi_{ID}$ .
$$
\mathbb {E} _ {q _ {\psi} \left(\phi_ {i} \mid \mathcal {T} _ {i} ^ {s}\right)} \left[ \sum_ {j} ^ {N _ {i} ^ {s}} E _ {\omega} \left(\mathbf {x} _ {i j} ^ {s}, y _ {i j} ^ {s}, \phi_ {i}\right) + E _ {\lambda} \left(\phi_ {i}\right) \right]. \tag {9}
$$