SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiayi Shen $^{1}$ , Xiantong Zhen $^{1,2}$ , * Qi (Cheems) Wang $^{3}$ , Marcel Worring $^{1}$
<sup>1</sup>University of Amsterdam, Netherlands, {j. shen, m. worring}@uva.nl
$^{2}$  Inception Institute of Artificial Intelligence, Abu Dhabi, UAE, zhenxt@gmail.com
$^{3}$  Kaiyuan Mathematical Sciences Institute, Changsha, China, hhq123go@gmail.com
This paper focuses on the data-insufficiency problem in multi-task learning within an episodic training setup. Specifically, we explore the potential of heterogeneous information across tasks and meta-knowledge among episodes to effectively tackle each task with limited data. Existing meta-learning methods often fail to take advantage of crucial heterogeneous information in a single episode, while multi-task learning models neglect reusing experience from earlier episodes. To address the problem of insufficient data, we develop Heterogeneous Neural Processes (HNPs) for the episodic multi-task setup. Within the framework of hierarchical Bayes, HNPs effectively capitalize on prior experiences as meta-knowledge and capture task-relatedness among heterogeneous tasks, mitigating data-insufficiency. Meanwhile, transformer-structured inference modules are designed to enable efficient inferences toward meta-knowledge and task-relatedness. In this way, HNPs can learn more powerful functional priors for adapting to novel heterogeneous tasks in each meta-test episode. Experimental results show the superior performance of the proposed HNPs over typical baselines, and ablation studies verify the effectiveness of the designed inference modules.
Deep learning models have made remarkable progress with the help of the exponential increase in the amount of available training data [1]. However, many practical scenarios only have access to limited labeled data [2]. Such data-insufficiency sharply degrades the model's performance [2, 3]. Both meta-learning and multi-task learning have the potential to alleviate the data-insufficiency issue. Meta-learning can extract meta-knowledge from past episodes and thus enables rapid adaptation to new episodes with a few examples only [4-7]. Meanwhile, multi-task learning exploits the correlation among several tasks and results in more accurate learners for all tasks simultaneously [8-11]. However, the integration of meta-learning and multi-task learning in overcoming the data-insufficiency problem is rarely investigated.
In episodic training [4], existing meta-learning methods [4-7, 12, 13] in every meta-training or meta-test episode learn a single-task. In this paper, we refer to this conventional setting as episodic single-task learning. This setting restricts the potential for these models to explore task-relatedness within each episode, leaving the learning of multiple heterogeneous tasks in a single episode underexplored. We consider multiple tasks in each episode as episodic multi-task learning. The crux of episodic multi-task learning is to generalize the ability of exploring task-relatedness from metatraining to meta-test episodes. The differences between episodic single-task learning and episodic multi-task learning are illustrated in Figure 1. To be specific, we restrict the scope of the problem
*Currently with United Imaging Healthcare, Co., Ltd., China.
Figure 1: Illustration of episodic multi-task learning. Each row corresponds to a meta-training or meta-test episode. Different colors represent different label spaces among episodes; the same color with different shades represents different categories in the same task. Compared with episodic single-task learning, episodic multi-task learning simultaneously handles several related tasks in a single episode.
setup to the case where tasks in each meta-training or meta-test episode are heterogeneous but also relate to each other by sharing the same target space.
The neural process (NP) family [12, 13], as typical meta-learning probabilistic models [14], efficiently quantifies predictive uncertainty with limited data, making it in principle well-suited for tackling the problem of data-insufficiency. However, in practice, it is challenging for vanilla NPs [12] with a global latent variable to encode beneficial heterogeneous information in each episode. This issue is also known as the expressiveness bottleneck [15, 16], which weakens the model's capacity to handle insufficient data, especially when faced with diverse heterogeneous tasks.
To better resolve the data-insufficiency problem, we develop Heterogeneous Neural Processes (HNPs) for episodic multi-task learning. As a new member of the NP family, HNPs improve the expressiveness of vanilla NPs by introducing a hierarchical functional space with global and local latent variables. The remainder of this work is structured as follows: We introduce our method in Section (2). Related work is overviewed in Section (3). We report experimental results with analysis in Section (4), after which we conclude with a technical discussion, existing limitations, and future extensions. In detail, our technical contributions are two-fold:
- Built on the hierarchical Bayes framework, our developed HNPs can simultaneously generalize meta-knowledge from past episodes to new episodes and exploit task-relatedness across heterogeneous tasks in every single episode. This mechanism makes HNPs more powerful when encoding complex conditions into functional priors.
- We design transformer-structured inference modules to infer the hierarchical latent variables, capture task-relatedness, and learn a set of tokens as meta-knowledge. The designed modules can fuse the meta-knowledge and heterogeneous information from context samples in a unified manner, boosting the generalization capability of HNPs across tasks and episodes.
Experimental results show that the proposed HNPs together with transformer-structured inference modules, can exhibit superior performance on regression and classification tasks under the episodic multi-task setup.
Notations 2. We will now formally define episodic multi-task learning. For a single episode  $\tau$ , we consider  $M$  heterogeneous but related tasks  $\mathcal{I}_{\tau}^{1:M} = \{\mathcal{I}_{\tau}^{m}\}_{m=1}^{M}$ . Notably, the subscript denotes an episode, while superscripts are used to distinguish tasks in this episode. In the episodic multi-task setup, tasks in a single episode are heterogeneous since they are sampled from different task distributions  $\{p(\mathcal{I}^m)\}_{m=1}^{M}$ , but are related at the same time as they share the target space  $\mathcal{Y}_{\tau}$ .
2 For ease of presentation, we abbreviate a set  $\{(\cdot)^m\}_{m=1}^M$  as  $(\cdot)^{1:M}$ , where  $M$  is a positive integer. Likewise,  $\{(\cdot)_o\}_{o=1}^O$  is abbreviated as  $(\cdot)_{1:O}$ . For convenience, the notation table is provided in Appendix B.
To clearly relate to the modeling of vanilla neural processes [12], this paper follows its nomenclature to define each task. Note that in vanilla neural processes context and target are often respectively called support and query in conventional meta-learning [4, 5]. Each task  $\mathcal{T}_{\tau}^{m}$  contains a context set with limited training data  $\mathcal{C}_{\tau}^{m} = \{\bar{x}_{\tau,i}^{m}, \bar{y}_{\tau,i}^{m}\}_{i=1}^{N_{\mathcal{C}}}$  and a target set  $\mathcal{T}_{\tau}^{m} = \{x_{\tau,j}^{m}, y_{\tau,j}^{m}\}_{j=1}^{N_{\mathcal{T}}}$ , where  $N_{\mathcal{C}}$  and  $N_{\mathcal{T}}$  are the numbers of context samples and target samples, respectively.  $\bar{x}_{\tau,i}^{m}$  and  $x_{\tau,j}^{m}$  represent features of context and target samples; while  $\bar{y}_{\tau,i}^{m}, y_{\tau,j}^{m} \in \mathcal{Y}_{\tau}$  are their corresponding targets, where  $i = 1, 2, \dots, N_{\mathcal{C}}$ ;  $j = 1, 2, \dots, N_{\mathcal{T}}$ ;  $m = 1, 2, \dots, M$ . For simplicity, we denote the set of target samples and their corresponding ground-truths by  $\mathbf{x}_{\tau}^{m} = \{x_{\tau,j}^{m}\}_{j=1}^{N_{\mathcal{T}}}$ ,  $\mathbf{y}_{\tau}^{m} = \{y_{\tau,j}^{m}\}_{j=1}^{N_{\mathcal{T}}}$ . For an episode  $\tau$ , episodic multi-task learning aims to perform simultaneously well on each corresponding target set  $\mathcal{T}_{\tau}^{m}$ ,  $m = 1, 2, \dots, M$ , given the collection of context sets  $\mathcal{C}_{\tau}^{1:M}$ .
For classification, this paper follows the protocol of meta models [4, 5, 17], such as  $O$ -way  $K$ -shot setup, clearly suffering from the data-insufficiency problem. Thus, episodic multi-task classification can be cast as a  $M$ -task  $O$ -way  $K$ -shot supervised learning problem. An episode has  $M$  related classification tasks, and each of them has a context set with  $K$  different instances from each of the  $O$  classes [5]. It is worth mentioning that the target spaces of meta-training episodes do not overlap with any categories in those of meta-test episodes.
2.1 Modeling and Inference of Heterogeneous Neural Processes
We now present the proposed heterogeneous neural process. The proposed model inherits the advantages of multi-task learning and meta-learning, which can exploit task-relatedness among heterogeneous tasks and extract meta-knowledge from previous episodes. Next, we characterize the generative process, clarify the modeling within the hierarchical Bayes framework, and derive the approximate evidence lower bound (ELBO) in optimization.
Generative Processes. To get to our proposed method HNPs, we extend the distribution over a single function  $p(f_{\tau})$  as used in vanilla NPs to a joint distribution of multiple functions  $p(f_{\tau}^{1:M})$  for all heterogeneous tasks in a single episode  $\tau$ . In detail, the underlying multi-task function distribution  $p(f_{\tau}^{1:M})$  is inferred from a collection of context sets  $\mathcal{C}_{\tau}^{1:M}$  and learnable meta-knowledge  $\omega, \nu^{1:M}$ . Note that  $\omega$  represents the shared meta-knowledge for all tasks, and  $\nu^{m}$  denotes the task-specific meta-knowledge corresponding to the task distribution  $p(\mathcal{T}^m)$ . Hence, we can formulate the predictive distribution for every single episode as follows:
$$
p \left(\mathcal {T} _ {\tau} ^ {1: M} \mid \mathcal {C} _ {\tau} ^ {1: M}; \omega , \nu^ {1: M}\right) = \int p \left(\mathbf {y} _ {\tau} ^ {1: M} \mid \mathbf {x} _ {\tau} ^ {1: M}, f _ {\tau} ^ {1: M}\right) p \left(f _ {\tau} ^ {1: M} \mid \mathcal {C} _ {\tau} ^ {1: M}; \omega , \nu^ {1: M}\right) d f _ {\tau} ^ {1: M}, \tag {1}
$$
where  $p(f_{\tau}^{1:M}|\mathcal{C}_{\tau}^{1:M};\omega,\nu^m)$  denotes the data-dependent functional prior for multiple tasks of the episode  $\tau$ . The functional prior encodes context sets from all heterogeneous tasks and quantifies uncertainty in the functional space. Nevertheless, it is less optimal to characterize multi-task function generative processes with vanilla NPs, since the single latent variable limits the capacity of the latent space to specify the complicated functional priors. This expressiveness bottleneck in vanilla NPs is particularly severe for our episodic multi-task learning since each episode has diverse heterogeneous tasks with insufficient data.
Modeling within the Hierarchical Bayes Framework. To mitigate the expressiveness bottleneck of vanilla NPs, we model HNPs by parameterizing each task-specific function within a hierarchical Bayes framework. As illustrated in Figure 2, HNPs integrate a global latent representation  $\mathbf{z}_{\tau}^{m}$  and a set of local latent parameters  $\mathbf{w}_{\tau,1:O}^{m}$  to model each task-specific function  $f_{\tau}^{m}$ . Specifically, the latent variables are introduced at different levels:  $\mathbf{z}_{\tau}^{m}$  encodes task-specific context information from  $\mathcal{C}_{\tau}^{m}$  and  $\nu^{m}$  in the representation level.  $\mathbf{w}_{\tau,1:O}^{m}$  encode prediction-aware information for a task-specific decoder from  $\mathcal{C}_{\tau}^{1:M}$  and  $\omega$  in the parameter level, where  $O$  is the dimension of the decoder. For
Figure 2: Graphical model of the proposed HNPs in a single episode. Filled shapes indicate observations. Probabilistic and deterministic variables are indicated by unfilled circles and diamonds, respectively.
example, the dimension is the size of the target space when performing classification tasks.
Notably, each local latent parameter is conditioned on the global latent representation, which controls access to all context sets in the episode for the corresponding task. Our method differs from previous hierarchical architectures [16, 18-20] in the NP family since the local latent parameters of our HNPs are prediction-aware and explicitly constitute a decoder for the subsequent inference processes.
In practice, we assume that distributions of each task-specific function are conditionally independent. Thus, with the introduced hierarchical latent variables for each task in the episode, we can factorize the prior distribution over multiple functions in Eq. (1) into:
$$
p \left(f _ {\tau} ^ {1: M} \mid \mathcal {C} _ {\tau} ^ {1: M}; \omega , \nu^ {1: M}\right) = \prod_ {m = 1} ^ {M} p \left(\mathbf {z} _ {\tau} ^ {m} \mid \mathcal {C} _ {\tau} ^ {m}; \nu^ {m}\right) p \left(\mathbf {w} _ {\tau , 1: O} ^ {m} \mid \mathbf {z} _ {\tau} ^ {m}, \mathcal {C} _ {\tau} ^ {1: M}; \omega\right), \tag {2}
$$
where  $p(\mathbf{z}_{\tau}^{m}|\mathcal{C}_{\tau}^{m};\nu^{m})$  and  $p(\mathbf{w}_{\tau,1:O}^{m}|\mathbf{z}_{\tau}^{m},\mathcal{C}_{\tau}^{1:M};\omega)$  are prior distributions of the global latent representation and the local latent parameters to induce the task-specific function distribution.
By integrating Eq. (2) into Eq. (1), we rewrite the modeling of HNPs in the following form:
$$
\begin{array}{l} p \left(\mathcal {T} _ {\tau} ^ {1: M} \mid \mathcal {C} _ {\tau} ^ {1: M}; \omega , \nu^ {1: M}\right) = \prod_ {m = 1} ^ {M} \int \left\{\int p \left(\mathbf {y} _ {\tau} ^ {m} \mid \mathbf {x} _ {\tau} ^ {m}, \mathbf {w} _ {\tau , 1: O} ^ {m}\right) \right. \tag {3} \\ p (\mathbf {w} _ {\tau , 1: O} ^ {m} | \mathbf {z} _ {\tau} ^ {m}, \mathcal {C} _ {\tau} ^ {1: M}; \omega) d \mathbf {w} _ {\tau , 1: O} ^ {m} \bigg \} p (\mathbf {z} _ {\tau} ^ {m} | \mathcal {C} _ {\tau} ^ {m}; \nu^ {m}) d \mathbf {z} _ {\tau} ^ {m}, \\ \end{array}
$$
where  $p(\mathbf{y}_{\tau}^{m}|\mathbf{x}_{\tau}^{m},\mathbf{w}_{\tau ,1:O}^{m})$  is the function distribution for the task  $\mathcal{I}_\tau^m$  in HNPs. This distribution is obtained by the matrix multiplication of  $\mathbf{x}_{\tau}^{m}$  and all local latent parameters  $\mathbf{w}_{\tau ,1:O}^{m}$ .
Compared with most NP models [12, 16, 18, 19] employing only latent representations, HNPs infer both latent representations and parameters in the hierarchical architecture from multiple heterogeneous context sets and learnable meta-knowledge. Our model specifies a richer and more intricate functional space by leveraging the hierarchical uncertainty inherent in the context sets and meta-knowledge. This theoretically yields more powerful functional priors to induce multi-task function distributions.
Moreover, we claim that the developed model constitutes an exchangeable stochastic process and demonstrate this via Kolmogorov Extension Theorem [21]. Please refer to Appendix B for the proof.
Approximate ELBO. Since both exact functional posteriors and priors are intractable, we apply variational inference to the proposed HNPs in Eq. (3). This results in the approximate ELBO:
$$
\begin{array}{l} L _ {\mathrm {H N P s}} (\omega , \nu^ {1: M}, \theta , \phi) = \sum_ {m = 1} ^ {M} \left\{\mathbb {E} _ {q _ {\theta} (\mathbf {z} _ {\tau} ^ {m} | \mathcal {T} _ {\tau} ^ {m}; \nu^ {m})} \left\{\mathbb {E} _ {q _ {\phi} (\mathbf {w} _ {\tau , 1: O} ^ {m} | \mathbf {z} _ {\tau} ^ {m}, \mathcal {T} _ {\tau} ^ {1: M}; \omega)} [ \log p (\mathbf {y} _ {\tau} ^ {m} | \mathbf {x} _ {\tau} ^ {m}, \mathbf {w} _ {\tau , 1: O} ^ {m}) ] \right. \right. \\ \left. - \mathbb {D} _ {\mathrm {K L}} \left[ q _ {\phi} \left(\mathbf {w} _ {\tau , 1: O} ^ {m} \mid \mathbf {z} _ {\tau} ^ {m}, \mathcal {T} _ {\tau} ^ {1: M}; \omega\right) \mid \mid p _ {\phi} \left(\mathbf {w} _ {\tau , 1: O} ^ {m} \mid \mathbf {z} _ {\tau} ^ {m}, \mathcal {C} _ {\tau} ^ {1: M}; \omega\right) \right] \right\} - \mathbb {D} _ {\mathrm {K L}} \left[ q _ {\theta} \left(\mathbf {z} _ {\tau} ^ {m} \mid \mathcal {T} _ {\tau} ^ {m}; \nu^ {m}\right) \mid \mid p _ {\theta} \left(\mathbf {z} _ {\tau} ^ {m} \mid \mathcal {C} _ {\tau} ^ {m}; \nu^ {m}\right) \right] \Bigg \}, \tag {4} \\ \end{array}
$$
where  $q_{\theta}(\mathbf{z}_{\tau}^{m}|\mathcal{T}_{\tau}^{m};\nu^{m})$  and  $q_{\phi}(\mathbf{w}_{\tau,1:O}^{m}|\mathbf{z}_{\tau}^{m},\mathcal{T}_{\tau}^{1:M};\omega)$  are variational posteriors of their corresponding latent variables.  $\theta$  and  $\phi$  are parameters of inference modules for  $\mathbf{z}_{\tau}^{m}$  and  $\mathbf{w}_{\tau,1:O}^{m}$ , respectively. Following the protocol of vanilla NPs [12], the priors use the same inference modules as variational posteriors for tractable optimization. In this way, the KL-divergence terms in Eq. (4) encourage all latent variables inferred from the context sets to stay close to those inferred from the target sets, enabling effective function generation with few examples. Details on the derivation of the approximate ELBO and its tractable optimization are attached in Appendix C.
2.2 Transformer-Structured Inference Module
In order to infer the prior and variational posterior distributions in Eq. (4), it is essential to develop well-designed approximate inference modules. This is non-trivial and closely related to the performance of HNPs. Here we adopt a transformer structure as the inference module to better exploit task-relatedness from the meta-knowledge and the context sets in the episode. More specifically, the previously mentioned meta-knowledge  $\omega = \omega_{1:O}$  and  $\nu^{1:M}$  are instantiated as learnable tokens to induce the distributions of hierarchical latent variables in the proposed model.
Without loss of generality, in the next, we provide an example of transformer-structured inference modules for prior distributions in classification scenarios. Details of the inference modules in regression scenarios can be found in Appendix D. In Figure 3, a diagram of the transformer-structured
Figure 3: A diagram of transformer-structured inference modules of HNPs for the first metatraining episode in Figure 1 under the 3-task 5-way 1-shot setting. For clarity, we display the inference process of the local latent parameters specific to the third task in the episode.
inference modules is displayed under the 3-task 5-way 1-shot setting. In this case, the number of context samples is the same as the size of the target space, and thus we have  $\mathcal{C}_{\tau}^{m} = \{\bar{x}_{\tau ,o}^{m},\bar{y}_{\tau ,o}^{m}\}_{o = 1}^{O}$  where  $O$  is set as 5. In episodic training, labels in context sets are always available during inference.
Transformer-Structured Inference Module  $\{\theta, \nu^m\}$  for  $\mathbf{z}_{\tau}^{m}$ . In the proposed HNPs, each global latent representation encodes task-specific information relevant to the considered task in the episode as  $p_{\theta}(\mathbf{z}_{\tau}^{m}|\mathcal{C}_{\tau}^{m};\nu^{m})$ . The learnable token  $\nu^{m}$  preserves the meta-knowledge from previous episodes for specific tasks, which are sampled from the corresponding task distribution  $p(\mathcal{I}^m)$ . The role of  $\nu^{m}$  is to help the model adapt efficiently to such specific tasks in meta-test episodes.
In detail, we set the dimension of the learnable token  $\nu^{m}$  to the same as that of the features  $\bar{x}_{\tau,1:O}^{m}$ . Then the transformer-structured inference module  $\theta$  fuses them in a unified manner by taking  $[\bar{x}_{\tau,1:O}^{m};\nu^{m}]$  as the input. The module  $\theta$  outputs the mean and variance of the corresponding prior distribution. The inference steps for the global latent representation  $\mathbf{z}_{\tau}^{m}$  are:
$$
\left[ \tilde {x} _ {\tau , 1: O} ^ {m}; \tilde {\nu} ^ {m} \right] = \mathsf {M S A} \left(\mathsf {L N} \left(\left[ \tilde {x} _ {\tau , 1: O} ^ {m}; \nu^ {m} \right]\right)\right) + \left[ \bar {x} _ {\tau , 1: O} ^ {m}; \nu^ {m} \right], \tag {5}
$$
$$
\left[ \widehat {x} _ {\tau , 1: O} ^ {m}; \widehat {\nu} ^ {m} \right] = \operatorname {M L P} \left(\operatorname {L N} \left(\left[ \widetilde {x} _ {\tau , 1: O} ^ {m}; \widetilde {\nu} ^ {m} \right]\right)\right) + \left[ \widetilde {x} _ {\tau , 1: O} ^ {m}; \widetilde {\nu} ^ {m} \right], \tag {6}
$$
$$
p _ {\theta} \left(\mathbf {z} _ {\tau} ^ {m} \mid \mathcal {C} _ {\tau} ^ {m}; \nu^ {m}\right) = \mathcal {N} \left(\mathbf {z} _ {\tau} ^ {m}; \mu_ {\mathbf {z} _ {\tau} ^ {m}}, \sigma_ {\mathbf {z} _ {\tau} ^ {m}}\right), \tag {7}
$$
where  $\mu_{\mathbf{z}_{\tau}^{m}} = \mathrm{MLP}(\widehat{\nu}^{m})$ ,  $\sigma_{\mathbf{z}_{\tau}^{m}} = \mathrm{Softplus}(\mathrm{MLP}(\widehat{\nu}^{m}))$ . The transformer-structured inference module includes a multi-headed self-attention (MSA) and three multi-layer perceptrons (MLP). The layer normalization (LN) is "pre-norm" as done in [22]. Softplus is the activation function to output the appropriate value as the variance of the prior distribution [23].
Transformer-Structured Inference Module  $\{\phi ,\omega_{1:O}\}$  for  $\mathbf{w}_{\tau ,1:O}^{m}$ . Likewise, each learnable token  $\omega_{o}$  corresponds to a local latent parameter  $\mathbf{w}_{\tau ,o}^{m}$ . With the learnable tokens  $\omega_{1:O}$ , we reformulate the prior distribution of local latent parameters as  $p_{\phi}(\mathbf{w}_{\tau ,1:O}^{m}|\mathbf{z}_{\tau}^{m},\mathcal{C}_{\tau}^{1:M};\omega_{1:O})$ . In this way, we learn the shared knowledge, inductive biases across all tasks, and their distribution at a parameter level, which in practical settings can capture epistemic uncertainty.
To be specific, the prior distribution can be factorized as  $\prod_{o=1}^{O} p_{\phi}\left(\mathbf{w}_{\tau,o}^{m} \mid \mathbf{z}_{\tau}^{m}, \mathcal{C}_{\tau}^{1:M}; \omega_{o}\right)$ , where all local latent parameters are assumed to be conditionally independent. For each local latent parameter  $\mathbf{w}_{\tau,o}^{m}$ , the transformer-structured inference module  $\phi$  takes  $[\bar{x}_{\tau,o}^{1:M}, \omega_{o}]$  as input and outputs the corresponding prior distribution, where  $\bar{x}_{\tau,o}^{1:M}$  are deep features from the same class  $o$  in the episode and  $\omega_{o}$  is the corresponding learnable token. Here the inference steps for  $\mathbf{w}_{\tau,o}^{m}$  are as follows:
$$
\left[ \widetilde {x} _ {\tau , o} ^ {1: M}; \widetilde {\omega} _ {o} \right] = \operatorname {M S A} \left(\mathrm {L N} \left(\left[ \bar {x} _ {\tau , o} ^ {1: M}; \omega_ {o} \right]\right)\right) + \left[ \bar {x} _ {\tau , o} ^ {1: M}; \omega_ {o} \right], \tag {8}
$$
$$
\left[ \widehat {x} _ {\tau , o} ^ {1: M}; \widehat {\omega} _ {o} \right] = \operatorname {M L P} \left(\mathrm {L N} \left(\left[ \widetilde {x} _ {\tau , o} ^ {1: M}; \widetilde {\omega} _ {o} \right]\right)\right) + \left[ \widetilde {x} _ {\tau , o} ^ {1: M}; \widetilde {\omega} _ {o} \right], \tag {9}
$$
$$
p _ {\phi} \left(\mathbf {w} _ {\tau , o} ^ {m} \mid \mathbf {z} _ {\tau} ^ {m}, \mathcal {C} _ {\tau} ^ {1: M}; \omega_ {o}\right) = \mathcal {N} \left(\mathbf {w} _ {\tau , o} ^ {m}; \mu_ {\mathbf {w} _ {\tau , o} ^ {m}}, \sigma_ {\mathbf {w} _ {\tau , o} ^ {m}}\right), \tag {10}
$$
where  $\mu_{\mathbf{w}_{\tau ,o}^{m}} = \mathrm{MLP}(\widehat{\omega}_{o},\mathbf{z}_{\tau}^{m(i)})$ $\sigma_{\mathbf{w}_{\tau ,o}^{m}} = \mathrm{Softplus}(\mathrm{MLP}(\widehat{\omega}_{o},\mathbf{z}_{\tau}^{m(i)}))$ $\mathbf{z}_{\tau}^{m(i)}$  is a Monte Carlo sample from the variational posterior of the corresponding global latent representation during meta-training.