Contrastive Meta-Learning for Partially Observable Few-Shot Learning
Authors: Adam Jelley
Amos Storkey
Antreas Antoniou
Sam Devlin
Venue: ICLR 2023 poster (2023)
Keywords: Contrastive Learning
Meta-Learning
Few-Shot Learning
Partial Observability

Abstract: Many contrastive and meta-learning approaches learn representations by identifying common features in multiple views. However, the formalism for these approaches generally assumes features to be shared across views to be captured coherently. We consider the problem of learning a unified representation from partial observations, where useful features may be present in only some of the views. We approach this through a probabilistic formalism enabling views to map to representations with different levels of uncertainty in different components; these views can then be integrated with one another through marginalisation over that uncertainty. Our approach, Partial Observation Experts Modelling (POEM), then enables us to meta-learn consistent representations from partial observations. We evaluate our approach on an adaptation of a comprehensive few-shot learning benchmark, Meta-Dataset, and demonstrate the benefits of POEM over other meta-learning methods at representation learning from partial observations. We further demonstrate the utility of POEM by meta-learning to represent an environment from partial views observed by an agent exploring the environment.

Published as a conference paper at ICLR 2023
FOR PARTIALLY OBSERVABLE FEW-SHOT LEARNING
CONTRASTIVE
Adam Jelley', Amos Storkey', Antreas Antoniou',
Sam Devlin
Schooo of Informatics, University of Edinburgh, Microsoof Research, Cambridge
ABSTRACT
Many contrastive and meta-learning approaches learn representations by identify- ing common features multiple views. However, the formalism for these ap- proaches generally assumes features to be shared across views to be captured coherently. We consider the problem of learning a unified representation from partial observations, where useful features may be present in only some of the views. We approach this through probabilistic formalism enabling views to map representations with different levels of uncertainty in different components; these views can then be integrated with one another through marginalisation over that uncertainty. Our approach, Partial Observation Experts Modelling (POEM), then enables us to meta-learn consistent representations from partial observations. We evaluate our approach on an adaptation of comprehensive few-shot learn- ing benchmark, Mcta-Datasee and demonstrate the benefits of POEM over other meta-learning methods at representation learning from partial observations. We further demonstrate the utility of POEM by meta-learning to represent an environ- ment from partial views observed by an agent exploring the environment
Minimiaa
Maximise Consistency
1s] Standard (Mote-)
Exparts Modal (POEM)
Figure 1: Standard contrastive (meta-) learners minimise a relative distance between representations. This encourages the learning of features that are consistent in all views; in the above example this corresponds to the pattern on the bird's wing. To better handle partial observability, where features may be disjoint between views, we propose Partial Observation Experts Modelling (POEM). POEM instead maximises consistency between multiple views, by utilising representation uncertainty to learn which features of the entity are captured by view, and then combining these representations together by weighting features by their uncertainty via a product experts model (Hinton 2002).
INTRODUCTION
Modern contrastive learning methods (Radford et al. 2021 Chen et al, 2020 He et al. 2020 Oord ct al.) 2019), and embedding- based meta-learning methods such Prototypical Networks Snnll ct al. 2017] Vinyals et al. 2016 Sung et al.) 2018 8 Edwards & Storkey 2017), learn representations
by minmiizing relative distance between representations of related items compared with unrelated
Implementation code is available wwiph/aiil.ppp//ttttoooo./////aa

---

Published as a conference paper at ICLR 2023
items (Ericsson al. 2021 However, argue that these approaches may learn to disregard po- tentially relevant features from views that only inform part of the representation in order to achieve better representational consistency, demonstrated in Figure [ We refer to such partially informa- tive views as partial observations. The difficulty with partial observations occurs because distances computed between representations must include contributions from all parts of the representation vector. If the views provided are diverse, and therefore contain partially disjoint features, their rep- resentations may appear different to a naive distance metric. For example, two puzzle pieces may contain different information about the whole picture. We call this the problem of integrative repre- sentation learning, where we wish to obtain a representation that integrates different but overlapping information from each element of set.
In this paper, we provide a probabilistic formalism for a few-shot objective that able to learn to capture representations in partially ohservable settings. It does so by building on a product of experts (Hinton (002)) to utilise representation uncertainty: a high variance indicates that the given view of the data poorly informs the given component, while low variance indicates it informs well. Given multiple views of the data, the product of experts component in POEM combines the representations, weighting by the variance, to get maximally informative and consistent representation from the views.
representation component
To comprehensively evaluate approach, adapt large-scale few-shot learning benchmark, Meta-Dataset (Triantafillou et al. 2020), to evaluate representation learning from partial observa- tions. We demonstrate that our approach, Partial Observation Experts Modelling (POEM), is able to outperform standard few-shot baselines on our adapted benchmark, Partially Observed Meta- while still matching state-of-the-art on the standard benchmark. Fi-
Dataset PPO-Meta-Datasee nally, we demonstrate the potential for our approach to be applied to meta-learn representations of environments from the partial views observed by an agent exploring that environment.
The main contributions of this work are: 1) A probabilistic formalism, POEM, that enables repre- sentation learning under partial observability; 2) Comprehensive experimental evaluation of POEM on an adaptation of Meta- Dataset designed to evaluate representation learning under partial observ- ability, demonstrating that this approach outperforms standard baselines in this setting while still matching tate--of-the-art on the standard fully observed benchmark; 3) A demonstration of poten- tial application of POEM to meta-learn representations of environments from partial observations.
RELATED WORK
2.1
CONTRASTIVE LEARNING
Contrastive learning extracts features that are present in multiple views of a data item, by encour- aging representations of related views be close in embedding space (Ericsson al. 2021). In computer vision and natural language applications these views typically consist of different augmen- tations of data items, which are carefully crafted to preserve semantic features, and thereby inductive bias to encourage the contrastive learner to retain these consistent features (Le-Khae et al.) 2020). challenge in this approach is to prevent representational 'collapse', where all views are mapped to the same representation. Standard contrastive approaches such as Contrastive Predictive Coding (Oord et al. 2019). MoCo (He al. 2020), and SimCLR (Chen al., (020) handle this by computing feature space distance measures relative to the distances for negative views pairs of views that are encouraged to be distinct in the embedding space. In this work we take a similar approach, where the negative views are partial observations of distinct items, but we aim to learn to unify features from differing views, not just retain the consistent features. We learn to learn a contrastive representation from partial views. We note that state-of-thee art representation learning approaches such as CLIP (Radford et al. [021]. I). which leverage contrastive learning across modali- ties, also suffer from extracting only a limited subset of features (Fürst al. 2022) due using an embedding- based approach (Vinyals et al. 2016) to match image and text representations.
an
2.2 EMBEDDING-BBAED META-LEARNING
Embedding basedd meta-leamers similarly learn representations of classes by extracting features that are consistently present in the data samples (generally referred to as shots in the meta-learning liter- ature) provided for each class, such that the class of new samples can be identified with similarity
2

---

Published as a conference paper at ICLR 2023
measure (Hospedales al. 2020). These methods generally differ in terms of their approach to combine features, and the distance metric used. Prototypical Networks (Snell et al. 2017) ) use a Eu- clidian distance between the query representation and the average over the support representations for a class (referred to as a prototype). Relation Networks (Sung et al.) 2018) use the same proto- type representation Prototypical Networks, but compute the similarity between the query and the prototype rather than using a Euclidian distance. Matching Networks (Vinyals al. (016) use Cosine distance between the query sample and each support sample as a weighting over the support labels, and so perform few-shot classification with- out unifying the support representations. None of these approaches are designed to unify partially informative support samples. The approach closest to that proposed in this paper is by Edwards & Storkey (2017), where the authors map the different views to statistic with an associated covariance through a variational approach. However there is no control of the contribution of each view to the variance, and the covariance is spherical, so the approach is also unsuitable for partial observation.
parameterised relation module to learn to
2.3 OPTIMISATION- BASED META-LEARNE
The few-shot classification task can also be solved without learning embeddings. One sensible baseline, fine-tuning of previously pre-trained large model, simply treats each few-shot task standard classification problem (Nakamura & Harada 2019p. For each task, one or more additional output layers are added top of a pre-trained embedding network and trained to predict the classes of the support set (alongside optionally finetuning the embedding network). This can then be utilised to predict the classes of the query set.
Taking this approach a step further, Model-Agnostic Meta-Learning (MAML) (Finn et al. 2017) learns the initialisation of the embedding network, such that it can be rapidly fine-tuned on new few-shot task. Given the generality of this approach. many variants of this method now exist, such as MAML++. Antoniou et al. (2018). Meta-SGD (Li al. 2017). CAVIA (Zintgraf al. 2019) and (o-Proto-MAMI. ( (Trantafillou et al. 2020). One variant, LEO Rusu et al. 2019), performs the meta- optimissiiiiinn on latent representation of the embedding parameters, learned using a relational network (Sung et al. [2018). However, none of these variants of this fundamental optimisation based approach to few-shot learning (referred to as 'MAML' for the remainder of this work) have mechanism for integrating partial information from the entire support set at inference time, or for comparison with partial query observation.
2.4 OTHER
APPROACHES
Probabilisitic meta-learning methods, such as VERSA (Gordon et al.] 2019). DKT (Patacchiola et al. 2020) and Amortised Bayesian Prototype Meta-Learning (Sun al. [2021), I). often unify both embedding- and optimisation based meta-learning by learning to output a posterior distribution that captures uncertainty in predictions, but do not uncertainty in features to optimally combine support set information. Other recent work, such as DeepEMD (Zhang et al.] 2022). has considered the use of attention mechanisms or transformers with image patches (Hiller et al. 2022 Dong al.) 2020), or augmentations (Chen et al. 2021a). However, the purpose of these approaches is to iden- tity informative patches features within each support example. to improve fine-grained few-shot learning performance or interpretability where relevant features may occupy only small region of the samples. As far as we know, there are no existing meta-learning methods that aim to integrate partial information from across the support set for comparison with partially informative query.
2.5 PARTIAL OBSERVABILITY AND PRODUCT OF EXPERTS
Factor analysis is the linear counterpart to modern representation learners, but where partial observ- ability is inherently expressed in the model. The inferential model for the latent space in factor analysis is product of each of the conditional Gaussian factors. In general, this form of inferential model can be captured as a product of experts (Hinton) 2002). When those experts are Gaussian distributions (Williams et al. 2001). this product of experts is fully tractable. By focusing on the inferential components rather than the linear model, it is possible to generalise factor analysis in- ference to nonlinear mappings (Tang et al. 2012). However, when only inferential component is required (as with representation learning), the product experts can be used more flexibly, as in approach below.
3

---

Published as a conference paper at ICLR 2023
3 THEORETICAL FORMALISM
In this section, we introduce POEM, which incorporates a product of experts model for combining different views with prior representation, and then uses that representation to classify query view.
3.1 PRODUCT OF EXPERT PROTOTYPES
Let us consider data corresponding to partial observations, or views, of set of items. In common with most few-shot frameworks, we arrange the data into support sets and query sets. Each support set consists of M data items: S = [X'"Im =, 1, 2....... M) where the mth item X collects V views, where V may vary with m. Let x denote the uth view of the mth data item, such that X = [x]" X ..... XVm }. The items in the support set are sampled randomly from the training dataset. The query point, denoted x", here consists of a single different view corresponding to one and only one of the M items in the support set (although in general simultaneously).. We cast our representation learning problem as meta-leaming task. We must learn unified representation derived from the support that can be compared with a representation of the query view. We want that comparison to enable us to infer which support set item m = m the query view belongs to.
may consider N query points
In this paper we are concerned with partial observability; that is, not every data view will inform the whole representation. So instead of mapping each view deterministic point representation, we map each view to a distributional representation where each component is a normalised density that indicates the uncertainty in that component (called a factor). We denote this conditional density é, and on implementation parameterise the parameters of the distribution & with neural network. We combine the corresponding factors for each view together using product of experts, which integrates prior distribution along with the different views such that views with low variance in component strongly inform that component.
For given support set, we compute product of experts distribution for the representation z"
p(z") I1 (x
[ dz' p(z')
( IT ((Z((x)))
where p(z) is prior density over the latent space. Now for a query point with a view that matches other views from e-g. data item m, we can use Bayes rule to compute the probability that the query point would be generated from the corresponding representation 2n by
p(x (m pzx p(x")((("")
p(zm)
where, again, p(z) f dx p(x))((2))) is the prior.
We put Eq 2 and Eq D together and marginalise over zm to get the marginal predictive distribution
p(x" X") f fazm
( p(2") r== of (xm"
pp(x") (1xx")(("(x'')
dzz
(3)
f p(z!)
1 oe ((2()x)))
p(z
=p(x') ( (f "0z
x
"( (x
=p(x*) A(x*, x
(4)
I dz' p(z') I (zz (x))
X(X
where
X(y,X) / dz p(zly) II [(()x,),
and
(5)
V
X(X) = / dz p(z) II (zlxx.).
(6)
The marginal predictive p(x" " is used to form the training objective. In few shot task, we wish to maximize the likelihood for the correct match of query point to support set, accumu- lated across all support/query selections indexed with from the dataset. This provides a complete
4

---

Published as a conference paper at ICLR 2023
negative log marginal likelihood objective to be minimized, as derived in appendix A.2:
A(x",x'')
A(x", x()
((IS),(at))
Σ log
loo
X(X"*)
X(Xm)
Full pseudocode for training POEM with this objective is provided in appendix A.3
3.2 INTERPRETATION OF OBJECTIVE
While the normalised factors can be chosen from any distribution class, we take & to be Gaussian with parameterised mean and precision for the remainder of this paper, rendering the integral in
5q analytic. Approximating the prior p(z) by a Gaussian also renders Eq. 6 analytic. [ We note that other distributions with analytic products, such as Beta distributions, may also be of interest in certain applications, but we leave an investigation of other distributional forms for Φ to further work.
If the representations from each view for a support point are aligned with each other and the query view (the means of all the Gaussians are similar), they will have greater overlap and the integral of the resulting product of Gaussians will be larger, leading greater value of A(y, X). Furthermore, increasing the precisions for aligned Gaussian components leads to greater A(y, X), while, up to limit, decreasing the precisions for non-aligned Gaussian components leads to greater X).
While the numerator in Eq. My.X X). quantifies the overlap of the support set with the query, the denominator X' X(X) contrasts this with the overlap of the support set representation with the prior. Together, this factor is enhanced if it is beneficial in overlap terms to replace the prior with the query representation, and reduced if such a replacement is detrimental. A greater consistency between query and combined support set representations intuitively leads to greater probability that the query belongs to the class of the corresponding support set, effectively extending Prototypical Networks to probabilistic latent representation space (Snell et al. 2017).
As a result, this objective is a generalisation of Prototypical Network that allows for (a) learnable weighted averaging over support examples based their informativeness to given component; (b) learnable combinations of features from subsets of support examples (via differing relative preci- sions of components within support representations), and (c) partial comparison of the query sample with the support samples (via differing relative precisions within the query). With all precisions fixed to this approach reproduces Prototypical Networks, neglecting small differences in scaling factors that arise with varying numbers of views. This relationship is derived in Appendix A.4
4 EXPERIMENTAL EVALUATION
There is a lack established benchmarks specifically targeted at the evaluation of representation learning under partial observability. To design comprehensive benchmark for few-shot represen- tation learning under partial observability, we leverage Meta- Meta-Dataset (Triantafillou et al.] 2020), a recently proposed collection of few-shot learning benchmarks. We selected Mata-Dataset the ba- sis for our adapted benchmark as it consists of diverse datasets involving natural, human- mudan-made and text-based visual concepts, with variety of fine-grained classification tasks that require learning from varying and unbalanced numbers of samples and classes. As a result, our derived benchmark inherits these properties to provide robust measure of the ability of learning approach learn representations from partial observations.
To extend Meta-Dataset incorporate partial observability, we take multiple views of each sample and divide these views into support and query sets. Our adapted few-shot classification task is to predict which sample a query view comes from, given selection of support views of that sample, as demonstrated in Figure 2
In keeping with the spirit of Meta-Dataset, we vary the number of ways in the task (now the number of images) from 5 to 25, taken from between 1 to 5 classes. Views are generated by applying the standard augmentation operations used in SimCLR (Chen et al., (020) and most other self- supervised learning methods. However, to emphasise the focus on partial observability, the size of
iI reality, p(2) is typically flat over the region of non-negligible density of the product IV,,, o(2/x,) so does affect the value of X' in Eq, and he neglected, descrihed in appendix A.1 A.1
5

---

Published as a conference paper at ICLR 2023
Meta-Dataset
Satta-Dtasst
Dataset
VGG Flowers
Meta-Datasee Few-Shot Classification of Classes from Samplea
PO-Maete- Few-Shot
Classes
010. Globe Thistle
Samples
Samples
Views
Figure 2: Standard few-shot learning requires the prediction of an image class from a sample. Our adapted task evaluates representation learning under partial observability by instead requiring pre- diction of the underlying image from partial views. Views are generated with the standard contrastive augmentations, with stronger cropping. We call the resulting benchmark Partially Observable Meta- Dataset (PO-Meta-Datasse
the random crops and the number of views was fixed, such that the entire support set for sample contains a maximum of 50% of the image. We also maintain a constant number of query views per sample. Viewpoint information consisting of the coordinates of the view is provided to make possible for learners understand where view fits into representation even the absence of overlapping views. Full details of the definition of the task are provided in appendix A.5
We apply our proposed evaluation procedure to all datasets included Meta-Datase with a few exceptions. ILSVRC (ImageNet, [Russakovsky et al.] (2015)) was not included since our network backbones were pre-trained on this dataset, including the standard few-shot test classes (which is also why this dataset was subsequently removed from the updated benchmark, MetaDataset-v2 (Du- moulin et al. 202I)). Traffic Signs (Stalkkmmp et al. 2011]) and MSCOCO (Lin et al. 2015) were not included since these datasets are fully reserved for evaluation by Meta-Dataset Dataset and so do not have training set specified. Quick Draw Beemnandzz-Ferannde
et al., 2019) was also not included since this dataset found to be too large to within the memory constraints of standard RTX2080
GPUs. This leaves six diverse datasets: Aircraft (Maji et al. (203)) Birds (Wah et al.] [1022011] 1), Flow- ers (Nilsback & Zisserman 2008), Fungi (Schroeder, Brigit (018), Omniglot (Lake al.) 2015) and Textures (Cimpoi et al. 2014), on all of which our models were trained, validated and tested on according to the data partitions specified by the Meta-Dataset benchmark.
The resulting benchmark, Partially Observed that the learner coherently combine the information from the support views into consistent repre- sentation of the sample, such that the query view can be matched to the sample it originated from. Since a maximum of 50% of each sample is seen in the support set, the task also requires generali- sation to correctly match and classify query views.
(PO-Meta- (PO-Meta-Dataset) therefore requires
4.1 IMPLEMENTATION DETAILS
We utilise re-implementation of Meta-Dataset benchmarking in PyTorch (Paszke et al. 2019) which closely replicates the Meta-Dataset sampling procedure of uniformly sampling classes, fol- lowed by a balanced query set (since all classes considered equally important) and unbalanced support sets (to mirror realistic variations the appearances of classes). The experimental imple- mentation, including full open-source code and data will be available publication.
Following the MD-Transfer procedure used et al. 2015) classifier pre-trained ImageNet (Russakovsky et al. 2015) at 126 x 126 resolution. Since both a mean and precision must be learned to fully specify the model Φ (z x.)), we add two simple 3-yeyrr MIP heads onto this backbone for POEM, each maintaining embedding size of
Detas-a we leverage a single ResNet-13 (He

---

Published as a conference paper at ICLR 2023
512. For fair comparison, also add the same 3-layer MLP head onto the backbone for the base- lines. Using a larger embedding for the baselines was not found to be beneficial. During training, gradients are backpropagated through the entire network such that both the randomly initialised heads and pre-trained backbones are learned/fin-tuned.
We use a representative selection of meta-learning baselines utilised by Meta-Dataset for implementation. This includes a strong naive baseline (Finetuning, Nakamura & Harada (2019), ) an embedding- approach (Prototypical Network, [Snell et al. (2017)) and an optimisation-basee
approach (MAML, Finn et al. (2017). all modernised to use the ResNet-18 backbone as described above. Recent competitions, such as the NeurIPS 2021 MetaDL Challenge (Baz al., 2022 202T),
have demonstrated that these fundamental approaches, updated to use modern pre- pre-trained backbones that are finetuned on the meta-task (exactly as in our experiments below) are still generally state- of-the-art for novel datasets (Chen et al. 2021b), and so form strong baselines. In addition, our re-implementation enables us ensure that all learners are optimised for Meta-Dataset and that comparisons between learners are fair, utilising the same benchmark parameters, model architectures and where applicable, hyperparameters. Crucially, given the close connection between POEM and Prototypical Networks, we ensure that all hyperparameters, including learning rates, scheduling and architectures are identical for both methods.
4.2 RESULTS
Our results on this novel representation learning benchmark, PO-Meta-Dataset, are given in table
Test Source
Finetune
ProtoNet
MAML
POEM
Aircraft
46.5 + 0.6
48.5 1.0
37.5 4 0.3
55.3
0.7
Birds
62.6 + 0.7
67.4 1.2
52.5
71.1 0.1
Flowers
48.5 + 0.4
46.4 0.7
33.5 1.3 0.3
49.2 1.5
Fungi
61.0 0.2
61.4 0.4
4..1
64.8 0.3
Omniglot
71.3 3.1 0.1
87.8 0.1
4.0
89.2 - 0.7
Textures
83.2 1 0.4
76.7 + 1.6
73.140.4
81.4 + 0.6
Table 1: Few-shot classification accuracies on our adapted Meta-Dataset benchmark, PO-Meta- Dataset. All learners use ResNer-18 model pre-trained ImageNet, with MI.P heads incorpo- rate view information. POEM outperforms the baselines across the range of datasets, demonstrating the benefits of the approach to learn and match representations from partial observations.
The results show that POEM outperforms the baselines at identifying views of images across diverse range of datasets, demonstrating the benefits of the approach to learn and match representa- tions from partial observations. The only exception is the Textures dataset, for which the finetuning baseline performs particularly strongly. We hypothesise that this is because the images in the Tex- tures dataset are relatively uniform compared the other datasets, so capturing the relative location of views is less important than identifying very fine grained features that distinguish the samples, which batimi approaches are particularly effective at.
4.3 ABLATION: META-DATASET
To demonstrate that the observed benefit of POEM over the baselines is due to the requirement of the task to learn coherent representations from partial observations, we also evaluate our approach against the baselines on the established Meta-Datase benchmark. We now follow the standard few- shot learning procedure as described in the paper (Triantafillou et al., 2020), but keep all learners identical to those used in the evaluation above.
Our results on the standard Meta-Dataset benchmark are provided in table 2 As expected, we find that POEM performs comparably with the baselines. Although Meta-Dataset provides realistic few-shot learning tasks in terms of diversity of visual concepts, fine-grained classes and variable shots and ways, each sample generally contains complete information including all relevant features for the visual concept in question. Correctly classifying query samples does not generally require any unification of views from support examples, but simply the identification of common features. As result, see that the additional capacity of POEM to learn to weight support examples and
7

---

Published as a conference paper at ICLR 2023
Test Source
Finetune
ProtoNet
MAML
POEM
Aircraft
56.2 + 1.1
47.2 + 1.2
35.9 + 1.8
46.5 1.5
Birds
52.6 + 1.8
78.3 + 0.5
65.2 + 0.3
79.4 + 0.3
Flowers
80.1 + 2.0
84.2 1 0.7
70.4 1 0.4
83.6 1 1.3
Fungi
33.6 1.7
1.7
84.7 0.2
18.9 1 0.2
81.0 + 0.1
Omniglot
89.6 3.3
98.7 08.70... 0.1
94.7 + 0.1
98.6 - 0.1
Textures
60.4 + 1.0
65.3 + 1.2
56,1 = 0.3
65.7 = 0.8
Few-shot classification accuracies on Meta-Dataset, all using ResNet-18 backbone pre- trained on ImageNet, with 3 layer MLP head. POEM is comparable with the baselines.
Table 2:
combine partial features does not provide a significant performance improvement over the baselines at few-shot classification in this fully observable benchmark.
In support of our hypothesis that feature uncertainty is not useful on this benchmark, we find that the variance in the precisions relative to the means output by the POEM model generally decreases during training and becomes negligible for all datasets, indicating that the precisions are not be- ing utilised improve performance and that the POEM objective is reducing to the Prototypical Network objective, as discussed in section 3.2 This further evidenced by the very similar per- formances of POEM and the Prototypical Network across the entire set of datasets. However, PO-Meta-Dataset, we find that the relative variance in the precisions to the means much larger convergence, which leads the improved performance of POEM over the Prototypical Network observed in Table This is shown in appendix A.6
5 DEMONSTRATION OF LEARNING REPRESENTATIONS OF ENVIRONMENTS
We now apply POEM to the equivalent task of learming a representation of an environment from the partial observations collected by an agent exploring that environment.
To do so, we utilise the 2D gridworld environment, MiniGrid (ChevalierBoisveett consider the 11 11 Simple Crossing environment, which consists of procedurally generated maze where the agent is required to traverse from the top left corner to the goal in the bottom right corner. The MiniGrid environment provides an agent-centric viewpoint at each step in the trajectory, consisting of 7 x 7 window of the environment in front of the agent, taking into account the agent's current direction and where the line sight is blocked by walls.
Boisvert et al. 2018). We
5.1 META-LEARNING ENVIRONMENT REPRESENTATIONS VIA FEW-SHOT CLASSIFICATION
To generate few-shot episodes, we utilise two agents: an optimal agent that takes the optimal trajec- tory from the start to the goal, and an exploratory agent that is incentivised to explore all possible views in the environment. The support set for each environment is generated by running the optimal agent in the environment and collecting the partial observations of this agent at each step during its trajectory. The query set similarly generated by running the exploratory agent in the environment, filtering out any observations that are contained within the support set, and then randomly sampling the desired number of queries from the remaining observations.
We generate these few-shot episodes dynamically, and train POEM to combine the support samples (partial observations from the optimal trajectory) into representation of the environment, such that it classify which environment novel query observation has been collected from. A set of sample environments and observations from those environments are shown in figures 3 and
3
9
5
7
Figure 4: Sample queries labelled with targets correspond- ing to the environment which they were observed in.
Figure 3: Sample environments.
8

---

Published as a conference paper at ICLR 2023
All observations are provided as pixels standard convolutional backbone, with the corresponding agent location and direction appended to this representation and passed through an MLP head, equiv- experiments. As baseline comparison, we also train a Prototypical Network with an identical architecture on this task. Additionally, we train equivalent recurrent network architecture typically applied to POMDP tasks such this Haasskeech & Stone 2017), , by adding a GRU layer (Cho et al. 2014; Chung et al.) 2014) where the hidden state of the GRU is updated at each timestep and then extracted the unified represen- tation of the agent. We find that POEM trains more quickly and reaches almost 10% higher final environment recognition performance than both the Prototypical Network and GRU-based approach over 100 test episodes 181.15 1 % vs 72.4% and 72. 1%), as shown in appendix A.8 This is a result of POEM's capacity to associate each observation with only part of the representation.
alent to the procedure utilised for the adapted
5.2 RECONSTRUCTING ENVIRONMENTS FROM PARTIAL OBSERVATION TRAJECTORIES
Having learned an environment encoder using the few-shot learning procedure above, we now in- vestigate the extent to which our representations can used to reconstruct the environment. As above, we generate trajectories with the optimal agent and feed these through the encoder to gener- ate a representation of the environment. An MLP decoder is then trained to reconstruct the original environment layout from the learned environment representation. The decoder attempts to predict one-hot representation of each possible grid cell, with a mean squared error loss. Given the trained encoder and decoder, we now able generate map of the environment the optimal agent has traversed, solely from the agent's partial observations, and without ever having seen the environment whole. A sample of environments alongside their reconstructions are shown in figure 5
Figure 5: Left: Ground truth environments explored by the agent. Right: Reconstructions of the corresponding environments from POEM's unified representation, encoded from the partial obser- vations of the agent.
We see that the reconstructions clearly capture the approximate structure of each environment, demonstrating that the agent has been able to integrate its observations from along its trajectory into a single consistent representation. Since POEM enables the representation to updated incre- mentally with each partial observation of the environment at inference time, it would be possible for an agent to update an internal environment representation at each step in its trajectory. There is potential for utilising this approach for learning environment representations be beneficial in the context of exploration for reinforcement learning, but we leave such an investigation to future work.
CONCLUSION
In this work, we have introduced Partial Observation Experts Modelling (POEM), contrastive meta-learning approach for few-shot learning in partially-observable settings. Unlike other stan- dard contrastive and embedding-basse meta-learning approuches. POEM utilises representational uncertainty to enable observations to inform only part of representation vector. This probabilis- tie formalism enables consistent representation learning from multiple observations with few-shot learning objective. We have demonstrated that POEM is comparable to the state-of-the-art base- lines on a comprehensive few-shot learning benchmark, and outperforms these baselines when this benchmark is adapted to evaluate representation learning from partial observations. We have also demonstrated promising potential application for POEM to learn representations of an environ- ment from an agent's 's partial observations. We hope that this research inspires further work into the challenging task of learning representations under partial observability and the creation of more realistic partial observability benchmarks.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
Adam Jelley was kindly supported by Microsoft Research and EPSRC through Microsoft's PhD Scholarship Programme. Antreas Antoniou was supported by Huawei DDMPLab Innovation Re- search Grant. The Deta-Dataset experiments in this work were partly funded by Google Research Compute Credits, and we thank Hugo Larochelle for his support in acquiring these compute credits.
REPRODUCIBILITY STATEMENT
All results reported in this paper are reproducible. The implementation code is available at: https://q
ETHICS STATEMENT
This work considers the problem of learning representations from partial observations. The authors are not aware of any specific ethical concerns associated with improving the quality of learned representations from partial observations, other than those already associated with any work in the field of machine learning.
REFERENCES
Antreas Antoniou, Harrison Edwards, and Amos Storkey. How to train your MAML. October 2018. doi: 10.485500/rxiv.1810.0
URL http:/ttp///xxxx.oor//o////
09502v3
Adrian El Baz, Isabelle Guyon, Zhengying Liu, Jan N. van Rijn, Sebastien Treguer, and Joaquin Vanschoren. Advances MetaDL: AAAI 2021 Challenge and Workshop. In AAA/ Workshop
Meta-Learning and MetaDL Challenge, pp. 1-16. PMLR, August 2021. URL https:// himl
proceedings mlr.. wttpppeeeeeeeeeebaaaaaaaaaaa poocee/icc.......p2ee//////////2
ISSN: 2640-3498.
Adrian El Baz, Ihsan Ullah, Edesio Alcobaça, André C. P. L. F. Carvalho, Hong Chen, Fabio Fer- reira, Henry Gouk, Chaoyu Guan, Isabelle Guyon, Timothy Hospedales, Shell Hu, Mike Huis- man, Frank Hutter, Zhengying Liu, Felix Mohr, Ekrem Öztürk, Jan N. Rijn, Haozhe Sun, Xin Wang, and Wenwu Zhu. Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-leamiig dominates for few-shot learning image classification, July 2022. http::/ataaa:: org/ abs 2206.2222000 08138 arXiv:2206.08 138 [cs].
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002. 200709002.05709 [cs, stat), June 2020. URL halggg//b///// httt:://axiv.oort org/ abs abss/2022 05709
arXiv: 2002.05709.
Wentao Chen, Chenyang Si, Wei Wang, Liang Wang. Zilei Wang, and Tieniu Tan. Few-Shot Learning with Part Discovery and Augmentation from Unlabeled Images, May 2021a. URL http://arxiv. org/ abs /atps./215..20 11874 arXiv:2105.1188 11874 [cs].
Yudong Chen, Chaoyu Guan, Zhikun Wei, Xin Wang, and Wenwu Zhu. MetaDelta: A Meta- Learning System for Few-shot Image Classification, February 2021b. URL http:://aaa:
brgg/ba//100 10744 MrXiv:2102.10044 [cs].
Maxime Chevalier-Boiss Boisvert, Lucas Willems, and Suman Pal. Minimalistic gridworld environment for OpenAl gym, 2018.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the Properties of Neural Machine Translation: Encoder-Decodes Approaches, October 2014. URL http:// arXiv:1409.1B 1259 [cs, stat].
arxiv. hww.vvorg/aaa.. abrg//bss1445 1259
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling, December 2014. URL http: arXiv: 1422:141 [es].
3555
10

---

Published as a conference paper at ICLR 2023
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
Chuanqi Dong, Wenbin Li, Jing Huo, Zheng Gu, and Yang Gao. Learning Task-aware Lo- Representations for Few-shot Learning. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pp. 716-722, Yokohama. Japan, July 2020. Interna- tional Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411 16-5. doi: 000224963/jccai.2222/000 URLpttps: www htt.://pw.....o.....c..oa///////
Vincent Dumoulin, Neil Houlsby, Utku Evci, Xiaohua Zhai, Ross Goroshin, Sylvain Gelly, and Hugo Larochelle. Comparing Transfer and Meta Learning Approaches on Unified Few-Shot Classification Benchmark. Technical Report arXiv:2104.02633, arXiv, April 2021. URL htth::: type: article.
//arxiv.orr//ab//20000 02638 arXiv:2104.02638 [cs]
Harrison Edwards and Amos Storkey, Towards Neural Statistician, 1606:/6000002/88 [cs, stat], March 2017. URL http:://t.t...oo///////// abs 1606. 02185
arXiv: 1606.02185.
Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M. Hospedales. Self-Supervised Representation Learning: Introduction, Advances and Challenges. arXiv:21 [cs, stat), October 2021.
09327
arXiv: 2110.09327.
Raul Fermandez--Fernandez, Juan G. Victores, David Estevez, and Carlos Balaguer. Quick, Stat! A Statistical Analysis of the Quick, Draw! Dataset. In EUROSIM 2019 Abstract Volume, 2019. doi: 10.1 10.11128/arcp.58. URL cess].
wwwpxiiwwwr//ab//1/00.0 06417
arXiv:1907.06417 [cs,
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adapta- tion of Deep Networks. Technical report, 2017.
Andreas Fürst, Elisabeth Rumetshofer, Johannes Lehner, Viet Tran, Fei Tang, Hubert Ramsauer, David Kreil, Michael Kopp. Günter Klambauer, Angela Bitto-Nemling, and Sepp Hochreiter. CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP, November 2022. URL
wttp:/iwwwrgg/bb//1100/ 11316 1..i:21100111311 [cs].
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard Tumnr. Meta- Learning Probabilistic Inference For Prediction, August 2019. URL http://arxiv.org/
abs/1805 09921 arXiv:1805.0992 [cs, stat].
Matthew Hausknecht and Peter Stone. Deep Recurrent Q-Learning for Partially Observable MDPs Technical Report S0di::1500005553 arXiv, January 2017. URL [es] type: article.
/bs//
1507. 06527 107.vv1507.00527
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. a12.03151..03385 [cs]. December 2015. URL http::/ org/ abs/ 1512. 03385
arXiv: 1512.03385.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. arXiv: 1911.0522 [cs], March 2020. URLhttp:
arXiv: 1911.05722.
Markus Hiller, Rongkai Ma, Mehrtash Harandi, and Tom Drummond. Rethinking Generalization in Few-Shot Classification, October 2022. URL arXiv:2206.07267 [cs].
http://www....o.///////..../.... 0726.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 1771) 2002. Publisher: MIT Press.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-Learning in Neural Networks: A Survey. April 2020. doi: 10.48500\axx..20444554999 URL https: /arxiv.
hrg/abs/2004.. 05439v2
11

---

Published as a conference paper at ICLR 2023
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science (New York, N.Y.), 350(6266):1332-1255 -3388838, 2015.
Publisher: American Association for the Advancement of Science.
Phuc H. Le-Khac, Graham Healy, and Alan F. Smeaton. Contrastive Representation Learning: A Framework and Review. IEEE Access, 8:193907-193934, 2020. ISSN 2169-3536. doi: (TC.///AACCEESS22200111.... 2010.05113.
URL
wttp://///..////..1.10.... 05113
arXiv:
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li, Meta-sgd: Learning to learn quickly for few- shot learning. arXiv preprint arXiv:1707.09835, 2017.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft COCO: Com- mon Objects in Context, February 2015. URL http:://arxa org. wttp:ogggg..g......./oo arXiv:1405.0312 [cs].
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-Graines Visual Classification of Aircraft, June 2013. URL arXiv:1306.5151 [cs].
wttpp//wwwwwww/////.o/oooo/ 5151
Akihiro Nakamura and Tatsuya Harada. Revisiting Fine-tuning for Few-shot Learning, October 2019.
19201 1rXiv:1910.00216 [cs, stat].
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian conference on computer vision, graphics and image processing, December 2008.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Pre- dictive Coding. 1.XXiv1807.0311 [cs, stat], January 2019. URL http:://axxti orooo/g////oo /bb/ 1BC7.03745
arXiv: 1807.03748.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang. Zach Vetoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, Pirf-Peefeemance Deep Learning Library, December 2019. URL http:// httt:://axx.v.org//bbb. abs wttp:r/bbbb.1..111...../. arXiv:1912.01700 [cs, stat].
Massimiliano Patacchiola, Jack Turner, Elliot J. Crowley, Michael O'Boyle, and Amos Storkey. Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels, October 2020. URLhttp:
19t./9990.0000 05199 arXiv:1910.05199 [cs, stat].
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, February 2021. BtRp:tt:://a///
orq abs http//abb/222...3 2103.00022 arXiv:2103.00020 [cs].
Roweis, Sam. Gaussian Identities, 1999. URL https: wyw. wtw.....dd..o.oee////oooooo//
pd
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. ImageNet Large Scale Visual Recognition Challenge. arXiv: 1409.0575 (cs), January 2015. arXiv: 1409.0575.
URL httt:::/aww....gg/////o o100//119.../ 0575,
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-Leearning with Latent Embedding Optimization, March 2019. URL
http:://rxixl wwtp:/oggg//a//
00t.//66000000000 arXiv:1807.05960 [es. stat].
Schroeder, Brigit. FGVC5 Fungi 2018.
URL https: /ttp::///ssteeee google. wtw.
12

---

Published as a conference paper at ICLR 2023
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms. wwxivv/707.06344 [cs], August 2017. URL /tLLtttt///xxxi.oret
abs A.bs/170..660 06347
arXiv: 1707.06347.
Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical Networks for Few-shot Learning. arXiv:1703.05175 [es, stat], June 2017. URL
arXiv: 1703.05175.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. The German traffic sign recognition benchmark: a multi-class classification competition. The 2011 international joint conference on neural networks, pp. 1453-1460 2011, rex.organization: IEEE.
Zhuo Sun, Jijie Wu, Xiaoxu Li, Wenming Yang, and Jing-Hao Xue. Amortized Bayesian Proto- type Meta-learning. A New Probabilistic Meta-learning Approach to Few-shot Image Classifica- tion. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, pp. 1414-1422, PMLR, March 2021. URL httttts://pocceeeeeeeeee aag.nngsll
mllr. press/v13//
Cun21a 1ann2la.tml html
ISSN: 2640-3498.
Flood Sung, Yongxin Yang. Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to Compare: Relation Network for Few-Shot Learning. Technical Re-
port ttrXiv:1711.06022 arXiv, March 2018. URLattp://axxc.oorrg http:://aariv..orrr/////....... arXiv:1711.0602. [cs] version: 2 type: article.
Yichuan Tang, Ruslan Salakhutdinov, and Geoffrey Hinton. Deep Mixtures of Factor Analysers, June 2012.
/20.. 4635 1XXv:1200.4633. [cs, stat].
Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evei, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. Meta- Dataset: A Dataset of Datasets for Learning to Learn from Few Examples. Technical Re- port (tEXXi:19933000999 arXiv, April 2020. URL httt:://taaxatt wtwpg//aa//00.0000000 arXiv:1903.0399 [es, stat] type: article.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. Match- ing Networks for One Shot Learning. In Advances Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https: http::::s///ocoeeddige www.sneerriii.omm htwhaaaaeeeee1133333333399999990 (ac.aa/618....ffooeeeeeeaa11-1e0
paper/2011 www.sh/aaaeeeeeeee 90e 10e00ee15133330
.html
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. Publisher: California Institute of Technology.
Christopher Williams, Felix Agakov, and Stephen Felderhof. Products of gaussians. In T. Dietterich, S. Becker, and Z. Ghahramani (eds.), Advances in Neural Information Processing Systems, vol- ume 14. MIT Press, 2001. URL https: https::/
nwwrnnuuun...m wtwp:/pppps.cppppe////0..// .pff
File/8232e 9d8f 8 30 500
Chi Zhang, Yujun Cai. Guosheng Lin, and Chunhua Shen. DeepEMD: Diffecenntiable Earth Mover's Mover's Distance for Few-Shot Learning, January 2022. URL http::/arrx.w.or//bb/////
06777 arXiv:2003.06777 [cs, eess].
Luisa M. Zintgraf. Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast Context Adaptation via Meta-Learning arXiv: /1/0.03// 0364422 [cs, stat], June 2019. URL (
03642 arXiv: 1810.03642.
A APPENDIX
A.1 GAUSSIAN PRODUCT RULES
Assuming the latent variable model d(z(x) to be diagonal covariance multivariate Gaussian, the resulting integrals over latent variables become integrals over Gaussian products. This allows both X X) (Equation 5) and (XX (Equation 6) in the marginal predictive distribution (Equation 4 to
13

---

Published as a conference paper at ICLR 2023
be evaluated analytically using the following univariate Gaussian product rules each independent dimension (Roweis, Sam 1999).
Since product of Gaussians II, (PETTT)) is itself Gaussian, have II, (ps, T4
SN (1, F 1), where
Σ T
(8)
" - Σ
1
Tifli
(2m)
T4 71/2
exp
1 2 - Tupu
Σ
(10)
Therefore the integral of Gaussian product is given by the resulting normalisation S.
In the case of evaluating the marginal predictive distribution p(x*|X (equation 4). this gives S'S
S'S Sr where S is the normalisation constant of the support product. S* is the normalisation constant of the product of the query and normalised support product, and 5" is the normalisation constant of the product of the prior p(=) and normalised support product. In reality, p(=) generally has little impact as it is typically flat (r + 0) over the region of non-negligible density of the product II_- 1 approximated by 5", in the simplified pseduocode in appendix A.3
( and so S' = and we find S
x,)
Sr R S* so the ratios A in the objective can be X7
A.2 DERIVATION OF OBJECTIVE FROM MARGINAL PREDICTIVE DISTRIBUTION
In section 3 we derived the marginal predictive distribution:
p ( x = p(x*) A(x*, X
(11)
where
dim X))
X(yX)) dz d(zly)
II v=1
o(z(x,),
and
(12)
dim(X)
X(X) fds p(z)
II dlz(x.).
(13)
In our few shot task, the support data is chosen and then the query view is chosen uniformly at random to match the views of one of the support data items. Let the hypothesis Hm indicate the event that the query view x" comes from support point m. Then
P(Hm) x" H..))
(1/M)p(x* (S, Hm)
P(Hm|S.x') = En P( P(S, x Hm))
(14)
En (1/M)p((x11)) )(x Hm))
(1/M)p(x*) X²)
p(x" x
(15)
Σm
En p(x X
From this we can formulate the training task: we wish to maximize the likelihood for the correct match of query point to support set, accumulated across all support/query selections from the dataset. Denote the ith support set by S., the 7th query point by -;, and let m, denote the support point with views that match the view of the query point. Then the complete negative log marginal likelihood objective to be minimized is:
C(SS,), {x1}) =
Σ log P(Hm, [S,,x.) x")
(16)
p(x" X
Σ log
(17)
Em p(x* (Xm²'
X(x", X
X(x*, Xm)
log
log
(18)
(X
X(X
14

---

Published as a conference paper at ICLR 2023
A.3 PSEUDOCODE
Algorithm Pytorch-Style Pscudocode: Gaussian Partial Observation Experts Modelling
phi: qual-headed encoder network with shared backbone and output heads for mean and
precision of Gaussian embedding
Number itens/cl clsses in
3 V: Number of views of each item/class (in general can vary with m in range (8)
Number of query
D: Embudding dimension
Lead augmented partial views for ssupport_viewe query_views, targeta) in loader: support_viewa shape (M, ...)
information
(N, ...) query_targets shape (N,)
Rncade each support and query views meppert_means, support quezy_means, query_precisione - phi (query_ (N, D)
phi (suppart_ views) (M,
Combine support views into unified repreaentation
each item
Gaussian products computed using equationa in appendix A.1
Optionally include prier Gaussian here (neglected for simplified implementation)
2og_environmeet_noen environment_meane environment pppirooment_preclsiooe
imerr_aassian__rrae
(ausport_mmans, support_precisions $
(M, D)
Combine each query view with each unified suppert representation.
env env_query_mean, env_query__reee precypprecsionnssse log_env www__uurr__orreeeee sstion
sater_gaussian_peee product (support__eann) means, support_ preciissss,,,, query_means,
quety_precisions) Outputa: X,
Predictions correspond unified support with maximum overlap with query predictions Log.enw.peey_nnnmalllatee sum max (1) 4 (K,)
Cross entropy loss normal with softmax computes nogative log-likel ihood query_targets, reduction-'menn
losa - F.cross_entropy
Opsimization step
bass.wackwadds) opt uutimmiee (t)
Algorithm Language Agnostic Pseudocode: Gaussian Partial Observation Experts Modelling
Require:
Training meta-set Darain T
Require: Learning rate
1: Initialise healed network (
2:
Heads correspond to mean f and precision of Gaussian embedding
3: while not converged do
4: Sample task instance T, x*) Droan
5:
>> Support set X consists of 1 views of item m E {l. M }.
6:
Query set x" consists of N queries, each one view from any one item. 7: Encode each view in support set X into Gaussian z using diz) ()
8: Encode each query view x" into Gaussian z" using Ø(z" xx*)
9:
for m {1,..., .) do
Compute Gaussian product over views I ((z x (using results in A.I b This gives unified support representation (global environment representation)
11:
for E fl....,N) N) do
13:
Compute Gaussian product of query with support product 6(z,, ((,)) II == oi d(z(x")
end for
15:
end for
S
16:
Normalise resulting query-support normalisation constants Stn E.. Sm across items
Compute negative log of Sy for correct support as loss ).) {xt1) (eq. 7
17:
18:
Negative log likelihood for correct support Perform gradient step w.r.t. #: 9 + & - avoL((D)) {xt3)
19:
20:
end while
15

---

Published as a conference paper at ICLR 2023
A.4 EQUIVALENCE PROTOTYPICAL NETWORK OBJECTIVE TO POEM OBJECTIVE WITH FIXED PRECISIONS
The probability of query x" belonging class using the POEM objective is given by:
X(x*;"")
X"*
P(HmlS,x*)= X(X
(19)
X(X*)
as defined in equation
15 where
X(y,X) / f p(zly) II ((z(x)),
and
(20)
X(X) / dz p(z) II (z|xx))
(21)
Taking the precisions of the all Gaussian factors e in and X to be we can apply the Gaussian product rules given in appendix A.J to calculate À and X analytically. We find that this gives:
(- Vn (-2(v=+1)) (u- Ens Mns
Vn Vi-1
exp
Pn
(22)
Σm
V=+1 exp
I1
E Mmmi Vm
2
where je is the representation mean of the query, and Pni is the representation mean of support sample i for class n, and V, is the number of support samples for class n.
Equivalently, the probability query with representation vector μ belonging class n using Prototypical Network objective is given by:
(
(-(p-
Σj Junt Vn
exp
11-
Pu
(23)
Σ, 2
Σm exp
11
Vm
Vn We find that these are equivalent aside from the scaling factors [2] which only have (sig- nificant) effect when there varying numbers of samples by class, and greater effect when the
number of samples is smaller. Experimentally, we find that these scaling factors make little differ- ence, demonstrated in table section 4.3
16

---

Published as a conference paper at ICLR 2023
A.5 PARTIALLY OBSERVABLE META-DATASER BENCHMARK ADDITIONAL DETAILS
Parameters used for adapted 20-Meta-Datasee are provided in Table A.5 All parameters not listed chosen to match Meta-Datased defaults. All augmentations are applied using Torchvision, with parameters specified.
Table 3: Partially Observable Meta-Datasel Parameters
PARAMETER
VALUE
Classes per Task
1, 5]
Samples per Task
[5, 25]
Support Views per Sample
18
Query Views per Sample
2
Image Size
(84, 84) (except Omniglot, (28, 28))
Crop Size
(14, 14) (1/6 in each dim, except Omniglot, random placement)
Color Jitter
(0.8, 0.8, 0.2). p(apply) = 0.3
Random Greyscale
0.2
Random Horizontal Flip
0.5
Gaussian Blur
((3,3), (1,0, p(apply) 0.2
All results computed over three runs. The Finetuning, Prototypical Network and POEM baselines were run on-premise RTX2080 GPUs. MAML required more memory and compute than avail- able, so was run on cloud A100s.
A.6 RELATIVE VARIANCE OF PRECISIONS DURING TRAINING ON META-DATASER AND PARTIALLY
The plot below shows the evolution of the variance in the representation precisions relative to the variance in the representation means learned by POEM two distinct datasets, Aircraft and VGG Flowers. We see that for standard few-shot learning on Meta-Dataset, the variance in precisions is negligible relative to the variance in the means, demonstrating that the representational uncertainty not useful in this task. Meanwhile, we see the variance in the precisions relative to the variance in the means becoming large before converging demonstrating that learning relative precisions useful in this setting since each support sample only informs part of the representation.
value of O(100) the PO-Meta-Dataset task,
Relative Variance Precisions Variance Means POEM Representation During Training
Vario) / Varip)
200
190
10k
Training Step
Ncto-Gataset:
P0-Ncta-Dataset: Aircraft Po-Hota-Datsset.
Neto-Dotasct: Flovers
17

---

Published as a conference paper at ICLR 2023
A.7 LEARNING REPRESENTATIONS OF ENVIRONMENTS ADDITIONAL DETAILS
Additional details about the parameters used for learning environment representations from agent observations provided in T
Table 4: Environment Representation Learning Parameters
PARAMETER
VALUE
Agent Training Algorithm
PPO (Schulman al. 2017) (default hyperparameters)
Optimal Agent Reward
for reaching goal, -0.01 per timestep
Exploratory Agent Reward
1/N count exploration bonus (state defined by agent location and direction)
Encoder Conv Backhone Layers
5
Encoder MLP Head Layers
Encoder Embedding Dim
128 (corresponding 11 11 environment size)
Decoder MLP Layers
A.8 ENVIRONMENT RECOGNITION ACCURACY DURING TRAINING
POEM trains more quickly the environment recognition task and reaches higher final perfor- mance than an equivalent Prototypical Network or Recurrent Network (GRU) (81.1 % vs 72.4% and 72.1% 1% over subsequent 100 test episodes.
Environment Recognition Accuracy
0.7
Accuracy
0.6
0.5
0.4
POEM
0.2
Prototypical Network
Recurrent Network (GRU)
1000
2000
4000
Episodes
18


Reviews:
[{"review_id": "jTzeq7h9QF", "reviewer": "Reviewer_YAXs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes POEM, a generalization of prototypical networks designed to handle few-shot learning when each support or query example is only a partial view/observation of the underlying concept. The derivation follows from modeling the joint distribution of support representations as the product of conditionally independent Gaussians. Prediction for a query example is done via Bayes's rule and marginalizing over the representation. For their experiments, the authors adapt a subset of Meta-Dataset so that each sample exhibits substantial partial observability. POEM is shown to improve upon baselines that do not model the partial observability. The authors also compare POEM and prototypical networks on a map reconstruction task based on the MiniGrid environment, finding that POEM enables better reconstructions faster.", "summary_of_the_review": "Overall, the work is interesting and of good quality, but is currently mired by a few individually minor weaknesses as detailed above. As such, I currently recommend borderline rejection.\n\n\n\n\nReferences\n\n[A] Rusu et al., Meta-Learning with Latent Embedding Optimization, ICLR 2019.\n\n[B] Sun et al., Amortized Bayesian Prototype Meta-learning: A New Probabilistic Meta-learning Approach to Few-shot Image Classification, AISTATS 2021.\n\n[C] Ren et al., Wandering Within a World: Online Contextualized Few-Shot Learning, ICLR 2021.\n", "strength_and_weaknesses": "Strengths\n\nThe work studies partial observability, a realistic property of high-dimensional embodied observations, in the context of few-shot learning. The proposed method is reasonable and generalizes a standard method from prior work. Steps are taken to ensure fair empirical comparisons. The authors also contribute a new benchmark based on adapting an existing dataset, and experiment with an embodied toy mapping task.\n\nWeaknesses\n\nThe baseline algorithms used in the PO-Meta-Dataset are rather weak, and it would be interesting to see to what extent more expressive models such as latent embedding optimization [A] are able to meta-learn strategies to handle the partial observability in comparison to the more explicit modeling done in POEM. Some relevant references also seem to be missing; it would also be good to discuss POEM in relation to amortized Bayesian prototype meta-learning [B], which also considers a probabilistic model of semi-parametric few-shot learning for classification. The authors also don't discuss whether the RoamingX datasets proposed in [C] could be suitable for evaluation, given the close relationship between embodiment and partial observability (as evidenced by the MiniGrid experiment).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ePTsoBRh830", "reviewer": "Reviewer_crwa", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a contrastive training approach for few-shot learning. The main benefit of the proposed approach is that it can perform few-shot learning in partially-observable settings.\n\nThe paper describes a formal framework for learning in partially-observable settings based on the product of experts model. Specifically, a potential function is associated with the representation learned for each view.  The factors over views are combined using a product of mixtures. To make the learning objective tractable the potential functions are assumed to be Gaussians though other types of distributions may be explored in future.\n\nA new dataset is developed that augments an existing dataset for few-shot learning (META-DATASET). The idea is to incorporate different views with partial observability in the dataset. The proposed approach is compared on all tasks available in the META-DATASET and compared with state-of-the-art for few-shot classification. Results show that under partial observability the proposed approach outperforms existing methods while being close to state-of-the-art on datasets that o not require reasoning over partially observable views. Further, the same approach is applied to learning the representation of a grid environment based on partial observations and it is shown that the proposed approach performs well in this case.\n", "summary_of_the_review": "Overall, the paper seems to have a clear motivation backed by a nice formulation. However, since the task and the dataset is new, the significance of the comparison studies is one possible issue.", "strength_and_weaknesses": "Strengths\nSeems to be an intuitive idea with a nice formalism of connecting uncertainty in views to the training function\nThe new dataset seems to be a good contribution for others working in this area\nShows empirical performance that is quite good in different settings\n\nWeakness\nIs there any bias in the creation of PO Meta-datasets. Specifically, since the other state-of-the-art models were not designed to solve the task in PO-Meta (if I understood it correctly), then are the results fair towards these models?\n\nFor the agent environment, it seems like this type of representation learning is bit more forced as compared to using other techniques such as POMDPs. Is the Prototypical network the state-of-the-art for this type of learning? Is there a stronger, more realistic case for this representational learning approach in partially observable environments", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HYTnGVA1rV", "reviewer": "Reviewer_EVnw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "+++++After Rebuttal+++++\n\nI would like to thank the author(s) for the rebuttal and revisions made. I believe the my main concern, which is the clarity of the presentation, has been greatly improved. In response to the improvements made, I have adjusted my score. Note that I am still not fully convinced that $P(X^*|X_m)$ is comparable for different classes. \n\n++++++++++++++++++++\n\nThis paper develops a Bayesian formalism for few-shot learning that captures representations with different levels of uncertainty in different components. In particular, this work is motivated  by the current gap in handling views that only contains partial observations, which violates the assumption made by most existing few-shot solutions. The proposed method, Partial Observation Experts Modelling (POEM), efficiently integrates different but overlapping information from different pieces to enable better global decision making. The proposed method is extensively benchmarked on the large-scale few-shot benchmark datasets, reporting encouraging results in partial observation settings. The author also demonstrated a synthetic experiment where POEM is used to facilitate an agent exploring the environment using only partial observations. ", "summary_of_the_review": "The current presentation lacks clarity and the claimed novelty failed to set it apart from prior arts, the experimental evidence also needs to be strengthened. Consequently, I am recommending reject at this moment. I will be happy to reconsider my evaluation if the author(s) can present strong supporting arguments or significantly improve their work during the rebuttal phase. ", "strength_and_weaknesses": "### Strength\n* This work is very-well motivated. Given the scarcity of support examples, it is desirable to quantify the relative similarity between support and query examples with the notation of uncertainty for robust inference and better generalization. \n* In the experiment results, the proposed PEOM showed significant performance gains over the baseline solutions under the same benchmark parameters and model architectures in most cases. \n\n### Weakness\nThe main weakness with this submission is the lack of clarity. I am unable to establish a mental model of how everything works together after a few attempts with the paper. The mathematical notations are not intuitively mapped to intuitive concepts/examples in few-shot learning. The local expert $\\phi(z|x)$ is not defined when it appears in the text, and I can not find a final training objective in the main text. In Fig 1 & 2, it seems to suggest that each view is a partial observation of the object, potentially augmented with some transformations. And it seems to suggest each sample in the support represent one particular class. So are the views for each X_m extracted from patches of all supporting images of class-m? Also $V^m = dim(X^m)$ seems to suggest each view is one dimension of the vector X. Additionally, is the query a view or a collection of views? These are all not clear from the text. If the size of support views differ for each class m, then is P(x^*|X^m) still comparable? p(z) seems to be a mixture of Gaussian on page 4, how is that tractable (are you just summing over the finite support set)? Then on page 5 the author(s) claimed that p(z) is approximated with a Gaussian, which again does not give any detail how the approximation is implemented. Eq 18 and Eq 19 (from the Appendix) are introduced on page 5 without any context, I guess that’s a typo in referencing equations. While the author(s) criticized prior work for using diagonal covariance (sec 2.2), base on the notations it seems that the this work also relies on diagonal covariance. Finally, there is a lot of marginalization going on, and for real image data the probability might vary considerably in scale, causing concerns for numerical overflow issues. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-WuGiFwp83_", "reviewer": "Reviewer_fe5b", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to tackle the partially observable few-shot learning problem where useful features are only contained in some of the views of a data instance. It develops a product of experts model to map different views to the representations with different uncertainty, which can capture the importance of different features. The proposed method is demonstrated to outperform previous few-shot learning methods on the newly created Partially Observable MetaDataset and MiniGrid environment.", "summary_of_the_review": "This paper studies an interesting problem as partial observation in few-shot learning. It derives a novel method to tackle this problem and gives an intuitive interpretation. The method is proven to outperform previous baselines on two problems. But generally, the baselines chosen in the experiment are too old and the problems designed are not realistic enough, which makes the significance of the proposed method in doubt. I think this paper is on the borderline and authors should try to add additional experiments on more realistic problems/datasets to further demonstrate the effectiveness of POEM. ", "strength_and_weaknesses": "Strengths:\n1. This paper identifies a novel problem that is also very important to the community, partial observation is very common in the reinforcement learning problem.\n2. The proposed method is well-formulated and the authors also provide an intuitive interpretation of the training objective.\n3. The experimental shows the advantage of the proposed POEM method to some extent and the ablation study further verifies the source of improvement comes from the feature uncertainty.\n4. The newly created PO-Meta-Dataset could be useful for future studies on this problem.\n\nWeaknesses:\n1. The baselines compared with POEM are all too old, the authors have mentioned many recent strong meta-learning methods in the related work part, I think the authors should also compare POEM with some of them. DeepEMD [1] is also a strong baseline.\n2. I do not see any point to mention contrastive learning in this paper, although there are some weak connections, like the objective in embedding-based few-shot learning and multi-views used by SimCLR, POEM is in general not strongly connected with the contrastive representation learning methods and I feel the title is kind of confusing.\n3. Although partial observation is an interesting problem, it is not commonly seen in the way PO-Meta-Dataset developed. The two experiments are more like toy problems where the feature uncertainty is manually inserted, I do not think creating the views through data augmentations used by SimCLR is the correct way to generate a realistic partial observation problem.\n\n   The most common partial observation is in reinforcement learning, known as POMDP, but the authors do not explore this problem enough in this paper. Another way I can think of is to make use of the annotation in the CUB Bird dataset, create a support set with only some parts of a bird observed, and the query set with other parts appearing. It could be more close to the real-world scenario where objects are not always observed from the same angle, so the parts observed in the support set may be different from the parts observed in the query set for the object from the same class.\n\n[1] DeepEMD: Few-Shot Image Classification with Differentiable Earth Mover's Distance and Structured Classifiers. Zhang et al., CVPR 2020.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster