Zero-shot task adaptation by homoiconic meta-mapping
Authors: Andrew K. Lampinen
James L. McClelland
Venue:  (2020)
Keywords: Meta-mapping
zero-shot
task adaptation
task representation
meta-learning

Abstract: How can deep learning systems flexibly reuse their knowledge? Toward this goal, we propose a new class of challenges, and a class of architectures that can solve them. The challenges are meta-mappings, which involve systematically transforming task behaviors to adapt to new tasks zero-shot. We suggest that the key to achieving these challenges is representing the task being performed in such a way that this task representation is itself transformable. We therefore draw inspiration from functional programming and recent work in meta-learning to propose a class of Homoiconic Meta-Mapping (HoMM) approaches that represent data points and tasks in a shared latent space, and learn to infer transformations of that space. HoMM approaches can be applied to any type of machine learning task, including supervised learning and reinforcement learning. We demonstrate the utility of this perspective by exhibiting zero-shot remapping of behavior to adapt to new tasks.

Under review as a conference paper at ICLR 2020
ZERO-SHOT TASK ADAPTATION BY HOMOICONIC META-MAPPING
Anonymous authors Paper under double-bling review
ABSTRACT
How can deep learning systems flexibly reuse their knowledge? Toward this goal, we propose a new class of challenges, and class of architectures that can solve them. The challenges are meta mappings, which involve systematically transforming task behaviors to adapt to new tasks zero-shot. The key to achieving these challenges is representing the task being performed in such way that this task representation is itself transformable. We therefore draw inspiration from functional programming and recent work in meta-learning to propose class of Homoiconic Meta-Mapping (HoMM) approaches that represent data points and tasks in a shared latent space, and learn to infer transformations of that space. HoMM approaches can be applied to any type of machine learning task, including supervised learning and reinforcement learning. We demonstrate the utility of this perspective by exhibiting zero-shot remapping of behavior to adapt to new tasks.
INTRODUCTION
Humans able to and reuse knowledge more flexibly than most deep learning models can (Lake et al., 2017; Marcus, 2018). The problem of rapid learning has been partially addressed by meta-learning systems (Santoro et al., 2016; Finn et al., 2017: 2018; Stadie al., 2018; Botvinick et al., 2019, see also Section 7). However, humans can use our knowledge of a task to fiexibly adapt when the task changes. In particular, we can often perform an altered task zero-shot, that is, without seeing any data at all. For example, once we learn to play game, we can immediately switch to playing in order to lose, and can perform reasonably on our first attempt.
One fundamental reason for this is that humans aware of what This allows us to adapt our task representations to perform an altered task. By contrast, most prior work zero-shot learning relies generating completely new representation for new task, e.g. from a natural language description (Socher et al., 2013; Oh et al., 2017, c.g.). These systems must therefore throw away much of their prior knowledge of the task and regenerate it from scratch, rather than just transforming what needs to be transformed. Other approaches use relationships between tasks as an auxiliary learning signal (Yao al., 2019; Pal and Balasubramanian. 2019, e.g.). but cannot adapt to new task by transforming their task representations. To address this, it is necessary to represent tasks in a transformable space. This can grant the ability to rapidly and precisely adapt behavior to new task.
trying to compute and why.
In this paper, we propose a new class of tasks based on this idea: meta-mappings, i.e. mappings between tasks (see below). Meta-mappings allow zero-shot performance of new task based on its relationship to old tasks. To address the challenge of performing meta-mappings, we propose using architectures which essentially take functional perspective on meta-learning, and exploit the idea of homoiconicity. (A homoiconic programming language is one in which programs in the language can be manipulated by programs in the language, just as data can.) By treating both data and task behaviors as functions, we can conceptually think of both data and learned task behaviors as transformable. This yields the ability to not only learn to solve new tasks, but to learn how to transform these solutions in response to changing task demands. We demonstrate that architectures can flexibly remap their behavior to address the meta- mepa-mapping challenge. By allowing the network to recursively treat its task representations data points, and transform them to produce task

---

Under review as a conference paper at ICLR 2020
representations, approach is able to achieve this flexibility parsimoniously. We suggest that approaches like ours will be key to building more intelligent and flexible deep learning systems.
2 META-MAPPING
We propose the mepp-ng challenge. We define meta-mapping as task, or mapping, that takes task as an input, output, or both. These include mapping from tasks to language (explaining), mapping from language to tasks (following instructions), and mapping from tasks to tasks (adapting behavior). While the first two categories have been partially addressed in prior work (e.g. Hermann et al., 2017; Co-Reyes al., 2019), the latter is more novel. (We discuss the relationship between our work and prior work in Section 7.) This adaptation can be cued in several ways, including examples of the mapping (after winning and losing at poker, try to lose at blackjack) or natural- -aturallanguage instructions ("try to lose at blackjack").
We argue that tosk-to-task meta meta-mappings are a useful way to think about human-like flexibility, because a great deal of our rapid adaptation is from task to some variation on that task. For example, the task of playing go on large board is closely related to the task of playing go on a small board. Humans can exploit this to immediately play well on a different board, but deep learning models generally have way to achieve this. We can also adapt in much deeper ways, for example fundamentally altering our value function on task, such as trying to lose, or trying to achieve some orthogonal goal. While meta-learning systems can rapidly learn distribution tasks they have experience with, this does not fully capture human flexibility. Given appropriate conditioning (see below), our architecture can use meta-mappings adapt to substantial task alterations zero-shot, that is, without seeing a single example from the new task (Lake et al., 2017). Achieving this flexibility to map new tasks will be an important step toward more general intelligence intelligence that is not limited to precisely the training tasks it has seen.
task from
HOMOICONIC META (HoMM) ARCHITECTURE
To address these challenges, we propose HoMM architectures, composed of two components:
1. Input/output systems: domain specific encoders and decoders (vision, language, etc.) that map into a shared embedding space Z.
2. A meta-learning system that a) learns embed tasks into the shared embedding space Z, b) learns to use these task embeddings to perform task-appropriate behavior, c) learns to embed meta-mappings into the same space, and d) learns use these meta-mapping embeddings to transform basic task embeddings in a meta-mapping appropriate way.
These architectures are homoiconic because they have a completely shared Z for individual data points, tasks, and meta-mappings. This parsimoniously allows for arbitrary mappings between these entities. In addition to basic tasks, the system can learn to perform meta- weta-mappings to systematically alter task behavior based on relationships to other tasks. That is, can transform task representations using the same components it uses to transform basic data points. (See also Appendix E.
Without training on meta-mappings, of course, the system will not be able to execute them well. However, as we will show, if it is trained on broad enough of such mappings, will be able to generalize to new instances drawn from the same meta-mapping distribution. For instances that fall outside its data distribution, or for optimal performance, may require some retraining. however. This reflects the structure of human behavior we are able to adapt rapidly when new knowledge relatively consistent with our prior knowledge, but learning an entirely new paradigm (such as calculus for a new student) can be quite slow (cf. Kumaran et al., 2016; Botvinick et al., 2019).
More formally, we treat functions and data as entities of the same type. From this perspective, the data points that one function receives can themselves be functions' The key insight is that then our architecture can transform data points² to perform basic tasks (as is standard in machine learning),
Indeed, any data point can be represented as a constant function that outputs the data point. Where "data" is a quite flexible term. The approach is agnostic to whether the learning is supervised or reinforcement learning, whether inputs are images natural language, etc.
2

---

Under review as a conference paper at ICLR 2020
(b) Meta-mapping
Loss (train)
Probe targets
Predictions
(a) Basic meta- meta-learning
June funeeprod prod
(c) Meta-mapping evaluation
Loss
Loss (eval)
Prohe targets
Predictions
Probe targets
Predictions
prr 0
pobs Darg.
pred
H
M
Pazuneaadd
June FnnD
funnpphh
-Largia a
M
:
Examples
Probes
(Zin, Sturgoo
Prb
prbb
Examples
Probes
Probes
Figure I: The HoMM architecture allows for transformations at different levels of abstraction. (a) For basic meta-learning a dataset consisting of (input embedding, output embedding) tuples is processed by the meta-network M to produce function embedding 2 fune, which is processed by the hyper network H. to parameterize function Fg/ene, which attempts to compute the transformation on probe inputs (used to encourage the system to generalize). However, our approach goes beyond basic meta-learning. The function embedding zJune can then be seen as a single input or output at the next level of abstraction, when the same networks .M and H are used to transform function embeddings based on examples of a meta-mapping (b). To evaluate meta-mapping performance, probe embedding of held-out function is transformed by the architecture to yield predicted embedding for the transformed task. The performance of this predicted embedding is evaluated by moving back down a level of abstraction and evaluating the actual target task (c). Because the function embedding is predicted by transformation rather than from examples, new tasks can be performed zero-shot. (M and H are learnable deep networks, and F, is deep network parameterized by H conditioned on function embedding z. Input and output encoders/decoders are omitted for simplicity. See the text and Appendix F22 for details.)
but can also transform these task functions to adapt to new tasks. This is related to the concepts of homoiconicity, defined above, and higher-order functions. Under this perspective, basic tasks and meta-mappings from task to task are really the same type of problem. The functions at one level of abstraction (the basic tasks) become inputs and outputs for higher-level functions the next level of abstraction (meta-mapping between tasks).
Specifically, we embed each input, target, or mapping into a shared representational space Z. This means that single data points are embedded in the same space the representation of function or an entire dataset. Inputs are embedded by a deep network I input + Z. Model outputs are decoded from Z by O Z - output. Target outputs encoded by T targets Z.
Given this, the task of mapping inputs to outputs can be framed as trying to find transformation of the representational space that takes the (embedded) inputs from the training set embeddings that will decode to the target outputs. These transformations are performed by a system with the following components (see Fig. 1): M ((Z, + Z the meta network, which collapses dataset of (input embedding, target embedding) pairs to produce a single function embedding. H. Z - parameters - the hyper network, which maps a function embedding to parameters. F Z Z - the transformation. implemented by deep network parameterized by H.
To perform basic task, input and target encoders (I and T) are used to embed individual pairs from an example dataset D, to form a dataset of example (input, output) tuples (Fig. la). These examples are fed M, which produces function embedding (via deep
Basic meta-learning:
3

---

Under review as a conference paper at ICLR 2020
neural network, with several layers of parallel processing across examples, followed by element- wise max across examples, and several more layers). This function embedding is mapped through the hyper network H. to parameterize F, and then F is used to process dataset of embedded probe inputs, and O to map the resultant embeddings to outputs. This system can be trained end-to-end on target outputs for the probes. Having two distinct datasets forces generalization at the meta-learning level, in exactly the same way as standard meta-learning algorithms learn to generalize.
More explicitly, suppose we have a dataset of example input, target pairs (D1 = {( yo), and some input from a probe dataset D2. The system would predict a corresponding output as:
ý = O (F2 (I (z)))
where F Fzfune is the meta-learner's 's representation for the function underlying the examples in Dj:
F Fglune is parameterized by H (=funo), where /unc .M (L(Z (zo), T (yo)) ....)
Then, given some loss function (y. 9) defined on single target output y and an actual model output g. we define our total loss computed on the probe dataset D2 as:
(...) 1 (y, O (FD, (I))))))
The system can then be trained end-to-end on this loss to adjust the weights of T, H. M, O, and I. That is, the meta-learning training procedure optimizes the system for generalizing the function implied by the examples it sees, in order to respond appropriately to probe inputs. As in training a standard meta-learning model, these probes are held-out from the meta-network, but the labels still used to encourage the system to meta-learn to respond appropriately to new inputs, Other data points are held-out entirely during training, and only used for evaluation, see Appendix A. for some detailed description. See Appendix F.2 for further details on the architecture and seyper-parameten
The fundamental insight of our paper is to show how basic tasks and meta- mappings can be treated homogenously, by allowing the network to transform its task representations like data (see Fig. lb,c). That is. propose to transform prior task representation in order to perform a related new task zero-shot. The relationship by which the prior task embedding should be transformed to perform the new task is specified by example tuples of (input, output) task embeddings. This is precisely analogous to how a basic task is specified in terms of (input, output) embeddings of data points, except that the embeddings now represent tasks that are being transformed, rather than individual data points.
Meta-mapping:
Thus from the perspective of our architecture, learning a meta-mapping between tasks is exactly analogous to learning a basic task. Anything that is embedded in Z can be transformed using the same system. Because tasks are embedded in for basic meta-learning, this allows for meta- peta-mappings using exactly the same M and H that use for basic tasks. Just as we would take set of paired embeddings of data points for a basic task, and use them to compute a function embedding for that task, we can take set of paired function embeddings representing a cett-mapping, and use them to create an embedding for that meta- meta-mapping. We can then use this meta-mapping embedding to transform the embedding of another basic task. We can thus behave zero-shot on a novel task based on its relationship to prior task.
For example, suppose we have an embedding Zgamel E Z for the task of playing some game, and we want to switch trying lose this game. We can generate meta-mapping embedding zmetn E Z from examples of embeddings generated by the system when it is trying to win and lose various M (f((2game2+ Zgame2,loes) ....).... We can generate new task embedding
other games: mmeta-
&game E Z:
ggamel .lose Faam (Zgamet)
where Pr... is parameterized by H (Zmeta)
This &gameel.lose can be interpreted as the system's guess at a losing strategy for game 1. To train a meta-mapping, we minimize the &g loss in the latent space betwen this guessed embedding and the embedding of the target task?. Whether or not we have such a target embedding, we can
The gradients do not update the example function embeddings, only the weights of M and H. due to memory contraints, Allowing this might be useful in more complex applications.
4

---

Under review as a conference paper at ICLR 2020
Random 50% holdout
largeted 10% holdout
Average Reward
Average loss
Trained
Trahed
Holc out
Held
Blackjack
High card
Match
Gtragght fuair
Parr
Blaokjack
High card
Matoh
Straght fithh
Pairo
Trained
Hell
Held out
Game
(a) The polynomials domain, Section 4.
(b) The cards domain, Section 5.
Figure 2: The HoMM system succeeds basic veta-learning, which is a necessary prerequisite for meta- meta-mappings. (a) The polynomials domain, Section 4. The system successfully generalizes to held out polynomials. The solid line indicates optimal performance; the dashed line indicates untrained model performance. (b) The card games domain, Section 5. The system successfully generalizes to held out games, both when trained on a random sample of half the tasks, or when a targeted subset is held out. The gray dashed line indicates chance performance (otuputting the mean across all polynomials, or playing randomly), while the solid lines are optimal performance. The orange dashed lines shows performance on held-out tasks of playing the strategy from the most correlated trained task. The system generally exceeds this difficult baseline, thus showing deeper generalization than just memorizing strategies and picking the closest. Error-bars bootstrap 95 -Is, numerical values for plots can be found in Appendix G.
evaluate how well the system loses with this &gamel, Jose strategy, by stepping back down level of abstraction and actually having it play the game via this embedding (Fig. Ic). This is how we evaluate meta-mapping performance evaluating the loss of transformed task embeddings on the respective target tasks. Note that this allows for zero-shot performance of the new task, because it simply requires transforming a prior task embedding.
Alternatively, we could map from language to a mepp-ng embedding, rather than inducing from examples of the meta-mapping. This corresponds the human ability change behavior in response to instructions. The key feature of our architecture - the fact that tasks, data, and language are all embedded in a shared space allows for substantial flexibility within unified framework. Furthermore, our approach is parsimonious. Because it uses the same meta-learner for both basic tasks and meta-mappingg, this increased flexibility does not require any added parameters."
4 LEARNING MULTIVARIATE POLYNOMIALS
As proof of concept, we first evaluated the system on the task of learning polynomials of degree
< 2 in 4 variables (i.e. the task was to regress functions of the form p IR - IR where p E P2 (B), though the model was given no prior inductive bias toward polynomial forms). For example, p(u, x. y, 2) Ir, the model might be evaluated on its output for points like (-1, 1.3, 0.5, 0.3). This yields an infinite family of base- well many families of meta-mappings
examples like (-1, 1, 1, 1) and (0.7, 2.1, 1.3, 2.1. and
level tasks (the vector space of all such polynomials), tasks (for example, multiplying polynomials by constant, squaring them, or permuting their input variables). This allows us to not only examine the ability of the system to learn learn polynomials from data, but also to adapt its learned representations in accordance with these meta-tasks. Details of the architecture and training can be found in Appendix
At least in principle, in practice of course increasing network size might he more heneficial for HoMM architectures performing meta-mappings as well as hasic tasks, compared to those performing only hasic tasks.
5

---

Under review as a conference paper at ICLR 2020
Random 50% holdout
Targeted 10% holdout
Loss on targett task
0.50
Average rewarrd
0.00
Held out
-0.25
Trainesd masppng trained polynomili
Tralredd helihoaz pllnamiai
Heid-out mapping. farred polymaiiai
Held mapping held held-ad pelymaid
-0.50
Lower
Bwitch autso
Lowera
Gwitch aalla
Hold-out type
Meta task
(a) The polynomials domain, Section 4.
(b) The cards domain, Scction 5.
Figure The HoMM architecture performs tasks zero-shot by meta- veta-mappings. (a) The system generalizes to apply learned meta- mappings to new polynomials, and even to apply unseen meta- mappings. The plots show the loss produced when evaluating the mapped embedding on the target task. For example, if the initial polynomial is p(x) = x + 1, and the meta-task is "square," the loss would be evaluated by transforming the embedding of p(z) and evaluating how well the mapped embedding regresses
to p(x)) x² + 2x 1. The results show that the system succeeds at applying meta-mappings it is trained on to held-out polynomials, as well as applying held-out meta- mappings to either trained held-out polynomials. The solid line indicates optimal performance; the dashed line is untrained model performance. (b) The system generalizes to meta-mapping new tasks in the cards domain. The system is trained do the meta-mappings shown here on a subset of its basic tasks, and is able to generalize these mappings to perform novel tasks zero-shot. For example, for the "losers" mapping, the sytem is trained to map games to their losers variants. When given held-out game, it is able to apply the mapping to guess how to play the losing variation. This plot shows the reward produced by taking the mapped embedding and playing the targeted game. The gray dashed line indicates random performance, while the colored dashed lines indicate performance if the system did not alter its behavior in response to the meta-mapping. The system generally exceeds these baselines, although the switch-suits baseline is more difficult with the targeted holdout. Error-bars are bootstrap 95%-CIs.
First, we show that the system is able achieve the basic goal of learning a held-out polynomial from few data points in Fig. 2a (with good sample-efficiency see Supp. Fig. 7).
Basic meta-learning:
Zero-shot adaptation by meta-mapping (task task): Furthermore, the system is able per- form meta-mappings over polynomials in order to flexibly reconfigure its behavior (Fig. 3a). We train the system to perform variety mappings, for example switch the first two inputs of the polynomial, add 3 to the polynomial, or square the polynomial. We then test its ability to generalize held-out mappings from examples, for example a held-out input permutation, or an unseen additive shift. The system is both able to apply learned meta-mappings to held-out polynomials, and to apply held-out meta- mappings has not been trained on, simply by seeing examples the mapping.
A STOCHASTIC LEARNING SETTING: SIMPLE CARD GAMES
We next explored the setting of simple card games, where the agent is dealt a hand and must bet. There are three possible bets (including "don't bet"), and depending the opponent's hand the agent either wins or loses the amount bet. This task doesn't require long term planning, but does incorporate some aspects of reinforcement learning, namely stochastic feedback on only the action chosen. We considered five games that are simplified analogs of various real card games (see Appendix F.1.2). We also considered several binary options that could be applied the games, including trying to lose instead of trying to win, or switching which suit was more valuable. These are challenging manipulations, for instance trying to lose requires completely inverting learned Q-function.

---

Under review as a conference paper at ICLR 2020
Random 50% holdout
Targeted 10% holdout
Loss on targett task
Average reward
0.25
0.00
Held out
-3.25
Traines masppng trained polynomili
Tralredd helihoor pelnamiai
Heid-out mapping. farred polymaiiai
Held mapping held held-ad pelymaidd
-0.50
Lower
Bwitch autso
Losera
Gaitch aalla
Hold-out type
Meta task
(a) The polynomials domain.
(b) The cards domain.
Figure 4: The HoMM system can perform meta-map mepaamap from language cues rather than meta- mapping examples. Compare to Fig. 3, which shows the same results when using examples instead of language. (a) In the polynomials domain, language cues still lead to good performance, even on held-out tasks or held-out meta-mappings, although examples perform slightly better (Fig. 3a). (b) Similarly, in the cards domain, language cues perform well. Error-bars are bootstrap 9ls-CI..
In order to adapt the HoMM architecture, we made very simple change. Instead of providing the system with (input, target) tuples to embed, we provided it with (state, action, reward) tuples, and trained it to predict rewards for each bet each state. full RL framework is not strictly necessary here because there is no temporal aspect to the tasks; however, because the outcome is only observed for the action you take, is not a standard supervised task.) The hand explicitly provided to the network for each example, but which game is being played is implicitly captured in the training examples, without any explicit cues. That is, the system must learn to play directly from sceing a set of (state, action, reward) tuples which implicitly capture the structure and stochasticity of the game. We also trained the system to make meta-mappings, for example switching from trying to win a game to trying lose. Details of the architecture and training can be found in Appendix F.
Basic meta-learning: First, we show that the system is able to play held-out game from examples in Fig. 2b. We compare two different hold-out sets: 1) train on half the tasks at random, or 2) specifically hold out all the "losers" variations of the "straight flush" game. In either of these cases, the meta-learning system achieves well above chance performance (0) at the held out tasks, although it is slightly worse at generalizing to the targeted hold out, despite having more training tasks in that case. Note that the sample complexity in terms of number trained tasks not that high, even training 20 randomly selected tasks leads to good generalization to the held-out tasks. Furthermore, the task embeddings generated by the system are semantically organized, see Appendix D.
Zero-shot adaptation by meta-mapping (task task): Furthermore, the system is able per- form meta-mapping (mappings over tasks) in order to flexibly reconfigure its behavior. For example, the system is trained to map games to their losers variations, can generalize this mapping to a game it has not been trained to map, even if the source or target of that mapping is held out from training. Fig. 3b we demonstrate this by taking the mapped embedding and evaluating the reward received by playing the targeted game with it. This task is more difficult than simply learning to play a held out game from examples, because the system will actually receive no examples of the target game (when it is held out). Furthermore, in the case of the losers mapping, leaving the strategy unchanged would produce a large negative reward, and chance performance would produce 0 reward, so the results are quite good.
AN EXTENSION VIA LANGUAGE
Language is fundamental to human flexibility. Often the examples of the meta-mapping are implicit in prior knowledge about the world that is cued by language. For example, "try to lose go" does not give explicit examples of the "lose" meta- meta-mapping. but rather relies on prior knowledge of what losing means. This is much efficient way cue known meta-mapping. In order to replicate this,
7

---

Under review as a conference paper at ICLR 2020
trained the HoMM system with both meta- meta-mappings based examples, and meta- meta-mappings based on language. In the language-basse based meta-mappinn mappinap a language input identifying the meta- mapping (but not the basic task to apply it to) is encoded by language encoder, and then provided as the input to H (instead of an output from M). The meta-mapping then proceeds as normal - H parameterizes F, which is used to transform the embedding of the input task produce an embedding for the target.
This language-cued meta-mapping approach also yields good performance (Fig. 4). However, examples of the meta-mapping are slightly better, especially for meta-mappings not seen during training, presumably because examples provide a richer description. In the next section we show that using language to specify a meta-mapping performs better than using language to directly specify the target task.
6.1 WHY META-MAP FROM TASKS TO TASKS?
Why are meta- mappingss between tasks useful? To answer this, consider various ways of adapting to a new task in Figure 5 (based on results from the cards domain, Section 5). The standard meta- learning approach would be adapt from seeing examples of the task, but this requires going and collecting data, which may be expensive and does not allow zero-shot adaptation. Alternatively, the system could perform the new task via a meta-mapping from prior learned task, where the meta-mapping is either induced from examples of the meta-mapping, or from language. Finally, the system could perform a new task from language alone, it is trained to map instructions to tasks. This is the more typical zero-shot paradigm (e.g. Romera-Paree
and TorI, 2015).
To address this latter possibility, we trained a version of the model where we included training the language system produce embeddings for the basic tasks (while simultaneously training the system on all the other objectives, such as performing the tasks from examples, in order to provide the strongest possible structuring of the system's 's knowledge for the strongest possible comparison). We compare this model's performance at held-out tasks to that of systems learning from examples of the new task directly, or from meta-mapping, see Fig. 5.
These results demonstrate the advantage of meta- meta- meappma While learning from examples is still better given enough data, it requires potentially-expensii
data collection and does not allow zero- shot adaptation. Performing the new task from a language description alone uses only the implicit knowledge in the model's weights, and likely because of this it does not generalize well to the difficult held-out tasks. Meta-mapping performs substantially better, while relying only on cached prior knowledge, viz. prior task-ember embedddnedds and description of the meta-napping (either in the form of examples or natural language). That is, mecaa-mppinng allows zero-shot adaptation, like performing from language alone, but results in much better performance than language alone, by leveraging richer description of the new task constructed using the system's knowledge of a prior task and the new task's 's relationship to it.
7 DISCUSSION
Related work: There has been variety of past work zero-shot learning, mostly based on natural-language descriptions of tasks. For example, Larochelle et al. (2008) considered the general problem of behaving in accordance with language instructions as simply asking a model to adapt its response when conditioned on different "instruction" " inputs. Later work explored zero-shot classification based on only natural language description of the target class (Socher et al., 2013; Nomera-Paredee and Torr, 2015; Xian et al.. 2018), or of a novel task in a tunguage-conditionn RL system (Hermann et al., 2017). Some of this work has even exploited relationships between tasks as a learning signal (Oh et al., 2017). Other work has considered how similarity between tasks can be useful for generating representations for a new task (Pal and Balasubramanian, 2019), but without transforming task representations to do so. (Furthermore, similarity is less specific than an onput-output mapping, since it does not specify along which dimensions two tasks are similar.) To our knowledge none of the prior work has proposed using meta- aata-mapiing approaches to adapt to new tasks by transforming task representations, nor has the prior work proposed a parsimonious homoiconic model which can perform these mappings.
8

---

Under review as a conference paper at ICLR 2020
Average reward on held-out tasks
0.4
T
0.2
0.1-
Examples
mappl-m Meta mappima (langurgge oue) Larguage directl
(eosmpple
Performing from
Figure 5: Comparison of a variety of methods for performing one of the 10% held-out tasks in the more difficult hold-out in the cards domain. There are a number of ways the system could adapt to a new task: from seeing example of the new task, from hearing the new task described from language alone, or from leveraging its knowledge about prior tasks via meta-mappings (in this case, from the non-losers variations of the same games). The meta-mappings offer happy medium between the other two alternatives - they only require cached knowledge of prior tasks, rather than needing to collect experience on the task before policy can be derived, but they outperform a system that simply tries to construct the task embedding from a description alone. Language alone not nearly as rich cue knowledge of how a new task relates to prior tasks.
Our work an extrapolation from the rapidly-growing literature on meta-learning (e.g. Vinyals et al., 2016; Santoro et al., 60 Finn et al., 2017: 2018; Stadie et al., 2018; Botvinick et al., 2019). It is also related the literature on continual learning, or more generally tools for avoiding catastrophic interference based on changes to the architecture (e.g. Fernando et al., 2017; Rusu et al., 2016), loss (e.g. Kirkpatrick et al., 2016; Zenke et al., 2017; Aljundi al., 2019), external memory (e.g. Sprechmann et al., 2018). We also connect to different perspective on continual learning in Appendix B. Recent work has also begun to blur the separation between these approaches, for example by meta-learning in online setting (Finn et al., 2019). Our work is specifically inspired by the algorithms that attempt to have the system learn to adapt to a new task via activations rather than weight updates, either from examples (e.g. Wang et al., 2016; Duan al., 2016), task input (e.g. Borsa et al., 2019).
Our architecture builds directly off of prior work on HyperNetworks (Ha al., 2016) networks which parameterize other networks and other recent applications thereof, such as guessing pa- rameters for a model to accelerate model search (e.g. Brock al., 2018; Zhang et al., 2019), and meta-learning (Li et al., 2019; Rusu et al., 2019, e.g.). In particular, recent work in natural language processing has shown that having contextually generated parameters can allow for zero-shot task performance, assuming that good representation for the novel task is given (Platanios al., 2017) in their work this representation was evident from the factorial structure of translating between many pairs of languages. Our work is also related to the longer history of work on different time-scales of weight adaptation (Hinton and Plaut, 1982; Kumaran et al., 2016) that has more recently been applied meta-learning contexts (e.g. Ba et al., 2016; Munkhdalai and Yu, 2017: Garnelo al., 2018) and continual learning (Hu et al., 2019, 9019, e.g.). is more abstractly related to work on learning to propose architectures (c.g. Zoph and Le, 2016: Cao et al.. 2019), and to models that learn to select and compose skills to apply to new tasks (c.g. Andreas et al., 2016b;a; Tessler et al., 2016; Reed and de Freitas, 2015: 5;; Chang et al., 2019). In particular, some of the work domains like visual question answering has explicitly explored the idea of building a classifier conditioned on a question (Andreas et al., 2016b; 2017), which is related to one the possible computational paths through our architecture. Work in model-based reinforcement learning has also partly addressed how transfer knowledge between different reward functions (e.g. Laroche and Barlier, 2017): our approach is more general. Indeed, our insights could be combined with model-based approaches, for example approach could be used to adapt a task embedding, which would then be used by learned planning model.

---

Under review as a conference paper at ICLR 2020
There has also been other recent interest in task (or function) embeddings. Achille et al. (Achille et al., 2019) recently proposed computing embeddings for visual tasks from the Fisher information of the parameters in model partly tuned on the task. They show that this captures some interesting properties of the tasks, including some types of semantic relationships, and can help identify models that can perform well framework where latent codes are computed for a task which can be decoded to distribution over parameters (Rusu al., 2019). Other recent work has tried learn representations for skills (e.g. Eysenbach et al., 2019) or tasks (Hsu et al., 2019, e.g.) for exploration and representation learning. Our perspective can be seen as a generalization of these that allows for remapping of behavior via meta-tasks. While there have been other approaches to zero-shot task performance, to the best of our knowledge none of the prior work has explored zero-shot performance of a task via meta-mappings.
task. Rusu and colleagues recently suggested similar meta-learning
We think that the general perspective of considering meta-mappings will yield many fruitful future directions. We hope that our work will inspire more exploration of behavioral adaptation, in areas beyond the simple domains we considered here. To this end, we suggest the creation of meta-learning datasets which include information not only about tasks, but about the relationships between them. For example, reinforcement learning tasks which involve executing instructions (e.g. Hermann et al., 2017; Co-Reyes al., 2019) can be usefully interpreted from this perspective. Furthermore, we think our work provides a novel perspective on the types of flexibility that human intelligence exhibits, and thus hope that may have implications for cognitive science.
Future Directions:
We do not necessarily believe that the particular architecture we have suggested is the best architecture for addressing these problems, although it has a number of desirable characteristics. However, the modularization of the architecture makes it easy to modify. (We compare some variations in Appendix E.) For example, although we only considered task networks F that are feed-forward and of fixed depth, this could be replaced with recurrent architecture to allow more adaptive computation, or even a more complex architecture (e.g. Reed and de Freitas, 2015; Graves et al., 2016). Our work also opens the possibility of doing unsupervised learning over function representations for further learning, which relates to long-standing ideas in cognitive science about how humans represent knowledge (Clark and Karmiloff-Smite 1993).
CONCLUSIONS
We've highlighted new type of flexibility in the form of meta- veea-ma between tasks. Meta-mapping can produce zero-shot performance on novel tasks, based on their relationship to old tasks. This is key aspect of human flexibility. The meta- veta-mapping perspective suggests that this flexibility requires representing tasks in a transformable way. To address this, our Homoiconic Meta Meta-Mapping (HoMM) approach explicitly represents tasks by function embeddings, and derives the computation from these embeddings via a HyperNetwork. This allows for our key insight: by embedding tasks and individual data points in a shared latent space. the same meta-lcarning architecture can be used both to transform data for basic tasks, and to transform task embeddings to adapt to task variations. This approach parsiomious, because it uses the same networks for basic- and meta-mappings. Perhaps because of this, the requisite meta-task sample complexity is small; we showed generalization to unseen meta-mappings after training on only 20 meta- meta-mappings. That is, we are able to achieve zero-shot performance on novel task based on held-out relationship between tasks. Furthermore, the meta- mapping approach yields better task performance than zero-shot approach based on performing the task based on just natural language description. This is step closer to human-leved flexibility.
We see our proposal as a logical progression from the fundamental idea of meta-learning that there is a continuum between data and tasks. This naturally leads to the idea of manipulating task representations just like we manipulate data. We've shown that this approach yields considrarbe flexibility, most importantly the mapa-na ability to adapt zero-shot to a new task. We hope that these results will lead to the development of more powerful and flexible deep-learning models.
REFERENCES
Achille, A., Lam, M., Tewari, R., Ravichandran, A., Maji, S., Fowlkes, C., Soatto, S., and Perona, P. (2019). Tes: Task Embedding for Meta-Leamning arXiv preprint.
10

---

Under review as a conference paper at ICLR 2020
Aljundi, R., Rohrbach, M., and Tuytelaars, T. (2019). Selfless sequential learning. In International Conference on Learning Representations, pages 1-17.
Andreas, J., Klein, D., and Levine, S. (2016a). Modular Multitask Reinforcement Learning with Policy Sketches. arXiv preprint.
Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. (2016b). Learning to Compose Neural Networks for Question Answering. arXiv preprint.
Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. (2017). Deep Compositional Question Answer- ing with Neural Module Networks.
Ba, J., Hinton, G., Mnih, V., Leibo, J. Z., and Ionescu, C. (2016), Using Fast Weights to Attend to the Recent Past. In Advances in Neural Information Processing Systems, pages 1-10.
Baars, B. J. (2005). Global workspace theory of consciousness: Toward cognitive neuroscience of human experience. Progress Brain Research, 150:45-53.
Borsa, D., Quan, J., Mankowitz, D., Hasselt, H. V., Silver, D., and Schaul, T. (2019). Universal Successor Features Approximators. In International Conference Learning Representations, number 2017, pages 1-24.
Botvinick, M., Ritter, S., Wang, J. X., Kurth-nelson, Z., Blundell, C., and Hassabis, D. (2019). Reinforcement Learning, Fast and Slow. Trends in Cognitive Sciences, pages 1-15.
Brock, A., Lim, T., Ritchie, J. M., and Weston, N. (2018). SMASH: One-Shot Model Architecture Search through HyperNetworks. In International Conference on Learning Representations.
Cao, S., Wang, X., and Kitani, K. M. (2019). Learnable Embedding Space for Efficient Neural Architecture Compression. In International Conference on Learning Representations, pages 1-17.
Chang, M. B., Gupta, A., Levine, S., and Griffiths, T. L. (2019). Automatically Composing Represen- tation Transformations as Means for Generalization. International Conference on Learning Representations, pages 1-23.
Clark, A. and Karmiloff- Smiih, A. (1993). The Cognizer's Innards: A Psychological and Philosophi- Perspective on the Development of Thought. Mind & Language, 8(4):487-519
Co-Reyes, D., Gupta, A., Suvansh, S., Altieri, N., Andreas, J., DeNero, J., Abbeel, P., and Levine, S. (2019). Guiding policies with language via meta-learning. In International Conference on Learning Representations, pages 1-17.
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P. (2016). RLS*2S: Fast Reinforcement Learning via Slow Reinforcement Learning. arXiv preprint, pages 1-14.
Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. (2019). Diversity all you need: learning skills without reward function. In International Conference Learning Representations, pages 1-22.
Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A. A., Pritzel, A., and Wierstra, D. (2017). PathNet: Evolution Channels Gradient Descent in Super Neural Networks. arXiv.
Finn, C., Abbeel, P., and Levine, S. (2017). Model-Agnostic Letannn for Fast Adaptation of Deep Networks, In Proceedings of the 34th Annual Conference on Machine Learning.
Finn, C., Rajeswaran, A., Kakade, S., and Levine, S. (2019). Online Meta-Learning, arXiv preprint.
Finn, C., Xu, K., and Levine, S. (2018). Probabilistic Model-Agnostic Meta-Learning, arXiv preprint.
Gareel,, M., Rosenbaum, D., Maddison, C. J., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D. J., and Eslami, S. M. A. (2018). Conditional Neural Processes. arXiv preprint.
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I.,
A., Gómez Colmenarejo, S., Grefenstette, E., Ramalho, T., Agapiou, J., Badia, A. P., Moritz Hermann, K., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., and Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature Publishing Group, 53877666)):7114477
11

---

Under review as a conference paper at ICLR 2020
Ha, D., Dai, A., and Le, Q. V. (2016). HyperNetworks. arXiv.
Hermann, K. M., Hill, F., Green, S., Wang, F., Faulkner, R., Soyer, H., Szepesvari, D., Czarnecki,
W. M., Jaderberg, M., Teplyashin, D., Wainwright, M., Apps, C., and Hassabis, D. (2017). Grounded Language Learning in Simulated 3D World. arXiv preprint, pages 1-22.
Hinton, E. and Plaut, C. (1982). Using Fast Weights to Deblur Old Memories. Proceedings of the 9th Annual Conference of the Cognitive Science Society, (1987).
Hlavac, M. (2018). stargazer: Well- Well-Formatted Regression and Summary Statistics Tables.
Hsu, K., Levine, S., and Finn, C. (2019). Unsupervised Learning Via Meta-Leaming, In International Conference on Learning Representations.
Hu, W., Lin, Z., Liu, B., Tao, C., Tao, Z., Zhao, D., and Yan, R. (2019). Overcoming catastrophic forgetting for continual learning via model adaptation. In International Conference on Learning Representations, pages 1-13.
Johnson, M., Schuster, M., Le, Q V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenherg, M., Corrado, G., Hughes, M., and Dean, J. (2016). Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. arXiv, pages 1-16.
Kirkpatrick. J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. (2016). Overcoming catastrophic forgetting in neural networks. arXiv preprint.
J., Ramalho, T.,
Kumaran, D., Hassabis, D., and McClelland, J. L. (2016). What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated. Trends Cognitive Sciences, 20(7):512-534
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2017). Building Machines that learn and think like people. Behavioral and Brain Sciences, pages 1-55.
Lampinen, A. K. and Ganguli, S. (2019). An analytic theory generalization dynamics and transfer learning in deep linear networks. In International Conference on Learning Representations, pages 1-20.
Lampinen, A. K. and McClelland, J. L. (2018). One-shot and few-shot learning of word embeddings. arXiv preprint.
Laroche, R. and Barlier, M. (2017). Transfer Reinforcement Learning with Shared Dynamics. In Proceedings of the Thirty First AAAI Conference on Artificial Intelligence, pages 2147-2153
Larochelle, H., Erhan, D., and Bengio, Y. (2008). Zero-data learning of new tasks. Proceedings of the Twenty- -hind AAAI Conference on Artificial Intelligence, (Miller 2002):646-651.
Laurens van der Maaten and Hinton, G. (2008). Visualizing Data using t-SNE. Journal of Machine Learning Research, 9:2579- 2605.
Li, H., Dong, W., Mei, X., Ma, C., Huang, F., and Hu, B.-G. (2019). LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning. Proceedings of the 36th International Conference on Machine Learning.
Marcus, G. (2018). Deep Learning: Critical Appraisal. arXiv preprint, pages 1-27.
McCloskey, M. and Cohen, N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and molivation, 24.
Munkhdalai, T. and Yu, H. (2017). Meta Networks. arXiv preprint.
Oh, J., Singh, S., Lee, H.. and Kohli, P. (2017). Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning. Proceedings of the 34th International Conference on Machine Learning.
Pal, A. and Balasubramanian, N. (2019). Zero-Shot Task Transfer. Proceedings of the IEEE Conference Computer Vision and Patlern Recognition.
12

---

Under review as a conference paper at ICLR 2020
Platanios, E. A., Sachan, M., Neubig, G., and Mitchell, T. M. (2017). Contextual Parameter Generation for Universal Neural Machine Translation. arXiv preprint.
Reed, S. and de Freitas, N. (2015). Neural Programmer-mneerpeeerrs arXiv preprint, pages 1-12.
Comera-Paredess, B. and Torr, P. H. (2015). An embarrassingly simple approach to zero-shot learning. 32nd International Conference on Machine Learning, ICML 2015, 3:2142-2151.
Rumelhart, and Todd, P. M. (1993). Learning and connectionist representations. Attention and performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience, pages 3-30.
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. (2016). Progressive neural networks, arXiv preprint.
Rusu, A. A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., and Hadsell, R. (2019). Meta-Learning with Latent Embedding Optimization. International Conference on Learning Representations, pages 1-17.
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. (2016). Meta-Learning with Mlemory-Augmente Neural Networks. In Proceedings of the 33rd International Conference Machine Learning, volume 48.
Socher, R., Ganjoo, M., Manning, D., and Ng, A. Y. (2013). Zero-shot learning through cross- modal transfer. Advances in Neural Information Processing Systems.
Sprechmann, P., Jayakumar, M., Rae, J. W., Pritzel, A., Uria, B., Vinyals, O., Hassabis, D., Pascanu, R., and Blundell, C. (2018). Memory-basee parameter Adaptation. In International Conference on Learning Representations.
Stadie, B. C., Yang, G., Houthooft, R., Chen, X., Duan, Y., Wu, Y., Abbeel, P., and Sutskever, I. (2018). Some Considerations Learning to Explore via Meta- Rent-Reeiforre Learning. arXiv preprint.
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J., and Mannor, S. (2016). A Deep Hierarchical Approach Lifelong Learning in Minecraft. arXiv preprint.
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. (2016). Matching Networks for One Shot Learning. Advances in Neural Information Processing Systems.
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M. (2016). Learning to reinforcement learn. arXiv preprint, pages 1-17.
Xian, Y., Lampert, C. H., Schiele, B., and Akata, Z. (2018). Zero-Shot Learning- Comprchensive Evaluation of the Good, the Bad and the Ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-14.
Yao, H., Wei, Y., Huang. J., and Li, Z. (2019). Hierarchically Structured Meta-leamnig, Proceedings of the 36th International Conference on Machine Learning.
Zenke, F., Poole, B., and Ganguli, S. (2017). Continual Learning Through Synaptic Intelligence. In Proceedings of the 34th International Conference on Machine Learning.
Zhang, C., Ren. M., Urtasun, R., Advanced, U., and Group, T. (2019). Graph HyperNetworks for neural architecture search. In International Conference on Learning Representations, number 2018, pages 1-17.
Zoph, B. and Le, Q. V. (2016). Neural Architecture Search with Reinforcement Leaming, arXiv preprint, pages 1-16.
13

---

Under review as a conference paper at ICLR 2020
The supplemental material is organized as follows: In Section A clarify some definitional details. In Section we describe a continual-learing like perspective based on our approach. In Section C provide supplemental figures. In Section D show t-sne results for the cards domain. In Section we provide some lesion studies. Section F we list details fo the datasets and architectures we used, well as providing links the source code for all models, experiments, and analyses. In Section G we provide means and bootstrap CIs corresponding to the major figures in the paper.
A CLARIFYING MEPA-MGPPING
A.1 CLARIFYING HOLD-OUTS
There are several distinct types of hold-outs in the basic training of our architecture:
1. On each basic task, some of the data (D1) is fed the meta-network examples M while some probes (D2) held out. This encourages the model to actually infer the underlying function, rather than just memorizing the examples. However, the probe labels still used in training the system end-to-end, in order to encourage it to generalize.
2. There are also truly held-out tasks that the system has never seen in training. These are the held-out tasks that we evaluate on the end of training and that are plotted the "Held out" sections in the main plots.
This applies analogously to the meta- mappings: each time meta-mapping is trained, some basic tasks are used as examples while others are held out to encourage generalization. There are also meta-mappings which have never been encountered during training, which we evaluate at the end of training. those are the meta-mappings which are plotted in the "held out" section in the relevant plots. We also evaluate the old (and new) meta-mappings on the new basic tasks that have never been trained.
A.2 A DEFINITIONAL NOTE
When we discussed meta-mappings in the main task, we equivocated between tasks and behaviors for the sake of brevity. For perfect model, this somewhat justifiable, because each task will have corresponding optimal behavior, and the sytem's 's embedding of the task will be precisely the embedding which produces this optimal behavior. However, behavior-irrelevan details of the task, like the color of the board, may not be embedded, so this should not really be thought of as task-to-task mapping. This problem is exacerbated when the system is imperfect, e.g. during learning. It is thus more precise to distinguish between ground-trui meta- mapping,,, which maps tasks to tasks, and the computational approach to achieving that meta-mapping, which really maps between representations which combine both task and behavior.
14

---

Under review as a conference paper at ICLR 2020
B CONTINUAL LEARNING
training new task ambeddings
from random ambeddlnngs
In an untrained network
10
Loss
Prior tasks
5-
New tasks
250
500
750
1000
0
250
500
750
1000
250
500
750
1000
Epoch
Figure 6: Once the meta-learning system has been trained on distribution of prior tasks, its performance on new tasks can tuned by caching its guessed embeddings for the tasks and then optimizing those, thus avoiding any possibility of interfering with performance on prior tasks. Starting from random embeddings in the trained model results in slower convergence, while in an untrained model the embeddings cannot be optimized well. Error-bars bootstrap 95%-CIs.
Continual learning: Although the meta-learning approach is effective for rapidly adapting to new task, it is unreasonable think that system must consider every example it has seen each inference step. We would like to be able to store our knowledge more efficiently, and allow for further refinement. Furthermore, guessed solution isn't t perfect) without catastrophically interfering with prior tasks (McCloskey and Cohen, 1989).
would like the system to be able adapt to new tasks (for which its
A very simple solution these problems is naturally suggested by architecture. Specifically, task embeddings can be cached so that they don't I have to be regenerated at each inference step. This also allows optimization of these embeddings without altering the other parameters the architecture, thus allowing fine-tuning on a task without seeing more examples, and without interfering with performance on any other task (cf. Rumelhart and Todd, 1993; Lampinen and McClelland, 2018). This is like the procedure of Rusu et al. (2019). except considered across episodes. That is, we can see the meta-learning step as "warm start" for an optimization procedure over embeddings that are cached in memory (cf. Kumaran et al., 2016). While this is not traditional continual learning perspective, we think it provides an interesting perspective on the issue. might in fact be much more memory-efficient to store embedding per task, compared to storing extra "importance" parameter for every parameter in our model, as in e.g. elastic weight consolidation (Kirkpatrick et al., 2016). It also provides stronger guarantee of -n-interference.
To test this idea, we pre-trained the system on 100 polynomial tasks, and then introduced 100 new tasks. We trained on these new tasks by starting from the meta-network's "guess" at the correct task embedding. and then optimizing this embedding without altering the other parameters. The results are shown Fig. The meta-network embeddings offer good immediate performance, accelcrate the optimization process, compared to -nnioml--nitializzed embedding (see supp. Fig. 10 for more direct comparison). Furthermore, this ability to learn is due training, not simply the expressiveness of the architecture, is shown by attempting the same with untrained network.
substantially
15

---

Under review as a conference paper at ICLR 2020
C
SUPPLEMENTAL FIGURES
6
Minimum necessary for perfect performance
Average loss
4
Trained
Held out
0
5
10
15
20
30
40
Number of points provided to meta-network
Figure 7: The system is able to infer polynomials from only sceing a few data points (i.c. evaluations of the polynomial), despite the fact that during training it always saw 50. A minimum of 15 random points is needed to correctly infer polynomials without prior knowledge of the polynomial distribution, but the system is performing well below this value, and quite well above it, although it continues to refine its estimates slightly when given more data.
Average training loss (basic)
15
10
HoMM
2000
4000
6000
8000
Epoch
Figure 8: Learning curves for basic regression in the polynomials domain.
16

---

Under review as a conference paper at ICLR 2020
15
Loss on target task (train only)
10
HoMM
2000
4000
6000
8000
Epoch
Figure 9: Learning curves for meta-mappings in the polynomials domain. Although the results seem to be leveling off at the end, we found that generalization performance slightly increasing or stable in this region, which may have interesting implications about the structure of these tasks (Lampinen and Ganguli, 2019).
Loss on new tasks
Training new task embeddings
from random embeddings
In untrained network
250
500
750
1000
Epoch
Figure 10: Continual learning in the polynomials domain: more direct comparison. Once the meta-learning system has been trained on distribution of prior tasks, performance on new tasks can be tuned by caching its guessed embeddings for the tasks and then optimizing those, thus avoiding any possibility interfering with performance on prior tasks. Starting with the guessed embedding substantially speeds-up the process compared to randomly -intiaiiittiilizedd embedding. Furthermore, this ability to learn due to training, not simply the expressiveness of the architecture, as is shown by attempting the same with untrained network.
17

---

Under review as a conference paper at ICLR 2020
Random 50% holdout
largeted 10% holdout
Average loss
0.2
Average Reward
0.4
Trahed
Held
Hold
0.2
0.0
Epoch after introducing new tasks
Epoch after introducing new tasks
(a) The polynomials domain, Section 4.
(b) The cards domain, Scction 5.
Figure 11: Integrating new tasks into the system by training all parameters results in some initial interference with prior tasks (even with replay), suggesting that an approach like the continual aearning-approach may useful.
D CARD GAME 1-SNE
We performed 1-SNE (Laurens der Maaten and Hinton, 2008) the task embeddings of the system at the end of learning the card game tasks, to evaluate the organization of knowledge in the network. In Fig. 12 show these embeddings for just the basic tasks. The embeddings show systematic grouping by game attributes. In Fig. 13 we show the embeddings of the meta and basic tasks, showing the organization of the meta-tasks by type.
800
Game
Blackjack
High card
400
Match
Pairs
Dimension
Straight flush
Switch suits
FALSE
TRUE
Losers
-400
TRUE
-800
-300
300
Dimension
Figure 12: t-SNE embedding of the function embeddings the system learned for the basic card game tasks. (Note that the pairs nearby embeddings differ in the "suits rule" attribute, discussed in Appendix F.1.2.)
18

---

Under review as a conference paper at ICLR 2020
Task
400 switch suits
pers
Basic
is fs_high_carre
ls_ |a_poker
Dimension
is match
1 maich Islllss is losers is switche Suits
is_ is_pairs
-400
blackjack
8
is losers
switched sults
Doggle_losers
loggle_switch_see sults
-1200
-1000
1000
Dimension
Figure 13: t-SNE embedding of the function embeddings the system learned for the meta tasks (basic tasks are included in the background).
ARCHITECTURE EXPERIMENTS
In this section we consider few variations of the architecture, to justify the choices made in the paper.
E.I SHARED Z VS. SEPARATE TASK-EMBEDDING AND DATA-EMBEDDING SPACE
Instead of having a shared where data and tasks are embedded, why not have a separate embedding space for data, tasks, and so on? There are a few conceptual reason why we chose to have shared 2, including its greater parameter efficiency, the fact that humans seem represent our conscious knowledge of different kinds in shared space (Baars, 2005), and the fact that this representation could allow for zero-shot adaptation to new computational pathways through the latent space, analogously to the zero-shot language translation results reported by Johnson and colleagues (Johnson al., 2016). this section, we further show that training with a separate task encoding space worsens performance, Fig. 14. This seems to primarily be due to the fact that learning in the shared Z accelerates and de-noises the learning process, see Fig. 15. (It's therefore worth noting that running this model for longer could result in convergence to the same asymptotic generalization performance.)
E.2 HYPER NETWORK VS. CONDITIONED TASK NETWORK
Instead of having the task network F parameterized by the hyper network H. we could simply have task network with learned weights which takes a task embedding as another input. Here, we show that this architecture fails to learn the meta-mapping tasks, although it can successfully perform the basic tasks. We suggest that this because is harder for this architecture to prevent interference between the comparatively larger number of basic tasks and the smaller number of meta-tasks. While might be possible to succeed with this architecture, it was more difficult in the hyper-pun peree-p space searched.
19

---

Under review as a conference paper at ICLR 2020
Loss on target task
2-
Trained mapping. trained polynomial
Trained mappling held-out polynomial
Held-out mapping, trained polynomial
Held-out mapping, held-out polynomial
0-
HoMM
HoMM (unshared Z)
Run type
Figure 14: Having a separate embedding space for tasks results in worse performance on meta- mappings. (Results are from only run.)
Loss on target task (train only)
15
10
HoMM
HoMM (unshared
0
2000
4000
6000
8000
Epoch
Figure 15: Having a separate embedding space for tasks results in noisier, slower learning of meta-mappings. (Results are from only run.)
20

---

Under review as a conference paper at ICLR 2020
Loss target task
Trained mapping, trained polynomial
10
Trained mapping, held-out polynomial
Held-out mapping, trained polynomial
Held-oo out mapping, held-out polynomial
EML
EML (non (non-hypef) hyper)
Run type
Figure 16: Conditioning the task network on the task embedding, rather than parameterizing via hyper network causes it to fail at the meta-mapping tasks. Results are from only 2 runs.
F DETAILED METHODS
F.1 DATASETS
F.1.1 POLYNOMIALS
We randomly sampled the train and test polynomials as follows:
Sample the number of relevant variables (k) uniformly random from (i.e. a constant) to the total number of variables.
2. Sample the subset variables that are relevant from all the variables.
3. For each term combining the relevant variables (including the intercept), include the term with probability 0.5. If so give it random coefficient drawn from (0, 2.5).
The data points on which these polynomials were evaluated were sampled uniformly from [-1, 1] independently for each variable, and for each polynomial. The datasets were resampled every 50 epochs training.
Meta-tasks: For meta-tasks, we trained the network on task-embedding classification tasks:
Classifying polynomials constan//non-constan..
Classifying polynomials zero/non-zero intercept.
For each variable, identifying whether that variable was relevant to the polynomial.
We trained 20 meta-mapping tasks, and held 16 related meta-mappings.
Squaring polynomials (where applicable).
Adding a constant (trained: -3, -1, 3, held-out: 2, -2).
Multiplying by a constant (trained: -3, 3, held-out: -2).
Permuting inputs (trained: 1320, 3201, 2103, 3102, 0132, 2031, 2301, 1203, 1023, 2310, held-out: 0312, 0213, 0321, 3012, 1230, 1032, 3021, 0231, 0123, 3120, 2130, 2013).
21

---

Under review as a conference paper at ICLR 2020
Language: We encoded the meta-tasks in language by sequences as follows:
Classifying polynomials constant/non-constantt ('is' *constant"")
Classifying polynomials
zero/non-zero
intercept:
('s''', "intercapt_ponzeroo"
For each variable, identifying whether that variable was relevant the polynomial: ('i''', <variable-name: 'relevant'')
Squaring polynomials: "square'
Adding a constant: [ 'add'' <value>]
Multiplying by a constant: [ ('multiply'' <value>]
Permuting inputs:
'permute'', <variable-name>, <variable- <variable- evariable-names
All sequences were front-padded with "<PAD>" to the length of the longest sequence.
F.1.2 CARD GAMES
Our card games were played with two suits, and values per suit. In our setup, each hand in game has win probability (proportional to how it ranks against all other possible hands). The agent is dealt hand, and then has to choose bet 0, 1, (the three actions it has available). We considered a variety of games which depend on different features of the hand:
High card: Highest card wins.
Pairs Same as high card, except pairs are more valuable, and same suit pairs are even more valuable.
Straight flush: Most valuable is adjacent numbers in same suit, i.e. 4 and 3 in most valuable suit wins every time (royal flush).
Match: the hand with cards that differ least in value (suit counts as 0.5 pt difference) wins.
Blackjack: The hand's value increases with the sum of the cards until it crosses which point the player "goes bust," and the value becomes negative.
We also considered three binary attributes that could be altered to produce variants of these games:
Losers:
Try to lose instead of winning! Reverses the ranking of hands.
Suits rule: Instead of suits being less important than values, they are more important (essentially flipping the role of suit and value in most games).
Switch suit: Switches which of the suits is more valuable.
Any combination of these options can be applied any of the games, yielding 40 possible games. The systems were trained with the full 40 possible games, but after training we discovered that the "suits rule" option does substantially alter the games we chose (in the sense that the probability of a hand winning in the two variants of a game is very highly correlated), so we have omitted it from our analyses.
Meta-tasks: For meta-tasks, we gave the network task-embedding classification tasks (one-vs-all classification of each of the 5 game types, and of each of the 3 attributes), and 3 meta-mapping tasks (each of the 3 attributes).
Language: We encoded the meta-tasks in language by sequences of the form [' "toggle'', <attribute-name-1 for the meta-mapping tasks, and (iig'', <attribbute-or-gam---r
22

---

Under review as a conference paper at ICLR 2020
F.2
MODEL & TRAINING
Basic task operation:
1. A training dataset Dj of (input, target) pairs is embedded by I and TT to produce paired embeddings. Another set of (possibly unlabeled) inputs D2 provided and embedded.
2. The meta network M maps the set of embedded (input, target) pairs to function embedding.
3. The hyper network H. maps the function embedding parameters for F, which is used to transform the second set of inputs to set of output embeddings.
4. The output embeddings are decoded by O to produce a set outputs.
5. The system trained end-to-end to minimize the loss on these outputs.
The model is trained to minimize
(a [ (y, O (Fn, (I()))))
where FD, the transformation the meta-learner guesses for the training dataset D1:
FD. parameterized by H (M (1(I (z+). T (y:)) for (aa y1) D13))
Meta-task operation:
1. A meta-datasct of (source-task-embeddnng targel-task-embedding) pairs, D1, collected. Another dataset D2 (possibly only source tasks) is provided. (All embeddings included in D2 during training are for basic tasks that have themselves been trained, to ensure that there is useful signal. During evaluation, the embeddings in Dj are for tasks that have been trained on, but those in D2 may be new.
2. The meta network M maps this of (source, target) eabeednbg pairs to a function embedding.
3. The hyper network H. maps the function embedding to parameters for F, which is used to transform the second set of inputs set of output embeddings.
4. The system trained minimize & loss between these mapped embeddings and the target embeddings.
The model is trained to minimize
E E(Eeearaa [S(zaarges (Ztarget, FD, (I (Esource)))
where & is &g loss, and FD, is the transformation the meta-learner guesses for the training dataset Dj:
FD, is parameterized by H. (M ([((Zeource Starget) E D1}))
Note that there three kinds of hold-out in the training of this system, see Section A.I.
Language-cued meta-tasks: The procedure is analogous to the meta-tasks from examples, except that the input to is the embedding the language input, rather than the output of M. The systems that were trained from language were also trained with the example-based meta-tasks.
F.2.1 DETAILED HYPER-PARAMETERS
See table for detailed architectural description and hyperparameters for each experiment. Hyper- parameters were generally found by heuristic search, where mostly only the optimizer, learning rate annealing schedule, and number of training epochs were varied, not the architectural parameters. Some of the parameters take the values they do for fairly arbitrary reasons, e.g. the continual learning experiments were run with the current polynomial hyperparameters before the hyperparameter search for the polynomial data was complete, so some parameters altered between these.
23

---

Under review as a conference paper at ICLR 2020
Polynomials
Continual learning
Cards
Z Z-dimension
512
512
512
Z num. layers I num. hidden units architecture
64
2-layer LSTM +2 fully- cullycconnected
I-layer LSTM + 2 fully-connected
C num. hidden units
512
512
O num. layers
O num. hidden units
512
T num. layers
M architecture
2 layers per-datum, max pool across, layers
H architecture
4 layers
M, H. num. hidden units
512
I' architecture
4 layers
F num. hidden units Nonlincarities
64
Leaky ReLU in most places, except no non- non-linearity at final
layer of T, M. C, F, and sigmoid for meta-clasiiiccaton outputs.
Main loss
l2 for main task meta-mapping, cross-entropy for meta- classification.
Optimizer
Adam
RMSProp
RMSProp
Learning rate (base)
3 10-b
1.10-
1 10
Learning rate (meta)
1 10-5
1 10-
L.R. decay rate (base)
*0.85
#0.85
*0.85
L.R. decay rate (meta)
*0.85
#0.9
L.R. min (base)
3 10-8
L.R. min (meta)
1 10-0
3 10-
L.R. decays every
100 epochs above min.
Cached embedding L.R.
10-3
Num. training epochs
4000
3000
40000
Num. runs
5
5
10
Num. base tasks (eval)
60
100
36 or 20
Num. base tasks (training)
1200
(= 60 20)
100
36 or 20
Num. meta classifications
Num. meta mappings
20
Num. new base tasks
40
30
4 or 20
Num. new meta mappings
16
Num. meta classifications
Base dataset size
1024
Base datasets refreshed
Every 50 epochs
M batch size
50
128
768
Table 1: Detailed hyperparameter specification for different experiments. A "_" indicates parameter that does not apply to that experiment. Where only one value is given, it applied to all the experiments discussed. As reminder: the shared representational space is denoted by Z. Input encoder:
I: input > Z. Output decoder O >> output. Target encoder T targets - Z. Meta-network M: ( ... + Z takes set of (input embedding, target embedding) pairs and produces function embedding. Hyper-network (: Z + parameters takes a function embedding and produces of parameters. Task network F + Z the transformation that executes the task mapping, implemented by a deep network with parameters specified by H. Where language was used to cue meta-mappings, it was processed by language encoder: natural language - Z.
24

---

Under review as a conference paper at ICLR 2020
Each epoch consisted of separate learning step each task (both base and meta), in random order. In each task, the meta-learner would receive only a subset (the "batch size" above) the examples to generate function embedding, and would have to generalize to the remainder of the examples in the dataset. The embeddings of the tasks for the meta-learner were computed once per epoch, so as the network learned over the course of the epoch, these embeddings would get "stale," but this did not seem to be too detrimental.
The results reported in the figures in this paper are averages across multiple runs, with different trained and held-out tasks (in the polynomial case) and different network initializations (in all cases), to ensure the robustness of the findings.
F.3 SOURCE REPOSITORIES
(Will be included in de-anonymized version.)
G NUMERICAL RESULTS
In this section provide the mean values and bootstrap confidence intervals corresponding the major figures in the paper, as well as the baseline results in those figures. Tables were generated with stargazer (Hlavac, 2018).
G.1 POLYNOMIALS
namod_runt type
is_now
mean_Joss
boot_Cl_Jww
boot_C CL_ggh
HoMM HoMM
Trained
0.015
0.012
0.018
0.246
0.188
0.308
Untrained HoMM network
Trained
5.735
4.823
6.74
Untrained HoMM network
Held out
5.968
4.984
6.991
Table 2: Table for basic meta-learning, Figure 2a
named_run rype
result_Iype
mean_ Joss
bootC'L_oo
HoMM
Trained mapping. on trained
0.098
HoMM
Trained mapping, on held-oat lask
1.721
1.419
2.115
Held-oot mapping, trained
HoMM
Held-out mapping, on held-out task
1,775
1.706
1,846
Untrained HoMM network
Trained mapping. on trained task
12.998
11.689
14.381
Untrained HoMM network
Trained mapping, held-out task
15.002
13.39
16.83
Untrained HoMM network
Held-out mapping. on trained task
8,36
7,898
8,833
Untrained network Held-out mapping. on held-out
Table 3: Table for meta- eeta-mapping results from examples, Figure
_umeed_uuu_type
result_type
mean_Joss
booLCLooww
Loot_CL_high
Language
Trained mapping. on trained task
0.515
0.483
0.552
Language
Trained mapping, on held-out task
2.244
1.921
2.623
Langunge
Held-out mapping, trained task
2.072
1.958
2.19
Langunge
Held-out mapping, on held-out task
2.35
2.254
2.447
Untrained HoMM network
Trained mapping. on trained
13.328
11.977
14.823
Untrained HoMM network
Trained mapping. on held-out tusk
15.313
13.602
17.354
Untrained network
Held mapping, trained
8.662
Untrained HoMM network
Held-out mapping, held-out task
8.625
8.131
9.104
Table 4: Table for meta- meta-mapping results from language, Figure 4a
25

---

Under review as a conference paper at ICLR 2020
G.2 CARDS
namned run_uunnnnn
named. gamne_ type
is new gurne
average_ rewwrr
wgg_rwd_Cr [__oow
avg_rwd CC high
Random holdout
High
Trained
0.521
Random 50% holdout
High card
Held out
0.441
0.42
0.462
Random
Match
Trained
0.537
0,524
Random 50% holdout
Match
Held
0.539
0.523
0.556
Random 50% holdout
Pairs
Trained
0.521
0.504
0.536
Random 50% holdout
Pairs
Held
0.453
0,434
0.47
Random, 50% holdout
Straight Dush
Trained
0.525
0.508
0.54
Random holdout
Straight Ilush
Random 50% holdout
Blackjack
Trained
0.582
0.557
0.603
Random holdout
Blackjack
Held
0.492
0,468
0.513
Targeted 10% holdout
High card
Traind
0.527
0.518
0.536
Targeted 10% holdout
Match
Trained
0.536
0,526
0,546
Targeted holdout
Pairs
Trained
0.522
Targeted 10% holdout
Straight Dush
Trained
0.524
0,509
0.538
Targeted holdout
Straight flush
Targeted 10% holdout
Blackjack
Trained
0.586
0.575
0.598
Table 5: Table for basic meta-learning, Figure 2b
Bon game
named_rum _tpe type
named game_type
expected roward
Trained
Targeted 10% holdout
High card
0.531
Trained
Targeted 10% holdout
Match
0.541
Trained
Targeted 10% holdout
Pains
0,532
Trained
Targeted
Straight
Held
Targeted 10% holdout
Straight flush
0.274
Trained
Targeted 10% holdout
Blackjack
0.592
Trained
Random 50% holdout
High card
0.531
Held out
Random 50% holdout
High card
0.396
Trained
50% holdout
Match
Held cut
Random 50% holdout
Match
0,54
Trained
Random 50% holdout
Pairs
0.532
Held
Random 50% holdout
Pairs
0.37
Trained
Random 50% holdout
Straight flush
0.536
Held
Random 50% holdout
Straight flush
0.452
Trained
Random 50% holdout
Blackjack
0,595
50% holdout
Blackjack
Table Table for playing most correlated learned strategy for basic meta-learning, dashed colored lines in Figure 2b
named game_type type
expected_ reward
reward
Blackjack
0.592
High card
0.531
Match
0.541
Pairs
Straght flush
U.536
Table 7: Table for playing optimal rewards for basic meta-learning, solid lines in Figure 2b
26

---

Under review as a conference paper at ICLR 2020
game
named. type
named. gamn type
expected reward
Trained
Targeted 10% holdout
High card
0.531
Trained
Targeted 10% holdout
Match
0.54
Trained
Targeted 10% holdout
Pairs
0.532
Truined
Targeted 10% holdout
Straight
Held out
Targeied 10% holdout
Straight Bush
0.274
Trained
Targeted
Blackjack
Trained
Random 50% holdout
High card
0,53
Held out
Random 50% holdout
High card
0.396
Trained
Random 50% holdout
Match
0.54
Held out
Random 50% holdout
Match
0.54]
Truined
holdout
Held out
Random 50% holdout
Pains
0.37
Trained
Random 50% holdout
Straight
0.536
Held
Random 50% holdout
Straight flush
0.452
Trained
Random 50% holdout
Blackjack
0.595
Held
Random 50% holdout
Blackjack
0.456
Table Table for most correlated baselines for basic meta-learnin, dashed colored lines in Figure 2b
namedd type
named, meta Lask
is_new
average_
avg avg_rwd_
avg_rwd_ Ch_hhhhh
Targeted 10% holdout
Switch suits
Trained
0.523
0.512
0.534
Targeted 10%
boldout
Switch
Held
0.273
Targeted 10%
holdout
Losers
Trained
0.532
0.51
0.546
Targeted 10%
boldour
Losers
Held our
0.289
0.241
0.322
Random 50% holdout
Switch suits
Trained
0.528
0.521
0.533
Random 50%;
holdout
Switch suits
Held out
0.375
0.368
0.382
Random 50% holdout
Losers
Trained
0.531
0.523
0.538
Random 50% holdout
Losers
Held our
0.427
0.417
0.436
Table 9: Table for meta-mapping from examples, Figure 3b
named_rum gamee__rn__tyee
metaatskk
is_new
average
avg Ang_rwd_Cl MLLCCLLR
avg_rwd_ CL CL_highh
Language (random 50%
Switch suits
Trained
0.525
0.52
0.534
Language frandom
Switch suils
0.353
Language (random 50%)
Lnsees
Trained
0.524
0.521
0.527
Language (random 50%)
Held
0.426
0.413
Language (targeted 10%)
Switch suits
Trained
0,531
0.52
0,542
Langunge (largeted 10%)
Switch suits
Held out
0.225
0.146
0.305
Langunge (largeted 10%)
I Losers
Trained
0.539
0.533
0.544
Language (langeted 105))
Losers
Held out
0.341
0,308
0.367
Table 10: Table for meta-mapping from language, Figure 4b
named_ _e___________________ type
named
expocted_ reward
Targeted 10% boldout
Switch suits
Trained
0.298
Targted 10% holdout
Held
Targeted 10% boldout
Losers
Trined
-0.46
Targeted 10% holdout
Losers
Held out
-0.463
Random 50% holdout
Switch suits
Trained
0.37
Random 50% holdout
Switch suits
Held out
0.278
Random 50% holdout
Losers
Trained
-0.465
Random 50% holdout
Losers
Held out
-0.444
Table 11: Table of rewards if system ignored meta-mapping, colored dashed lines in Figure 3b
27


Reviews:
[{"review_id": "rJg8esmZcB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a zero-shot task adaptation methodology by learning a meta-learner that reuses a latent representation of data points and tasks (Homoiconic Meta-Mapping). The proposed approach is applied to learn multivariate polynomial, a card game and the paper claims that such a method can apply in both supervised and RL settings.\n\n+ves:\n\n+ Task-to-task mapping is a relevant topic in the contemporary context, and the paper seeks to address this relevant problem.\n+ The overall idea of considering winning-losing strategies in meta-learning is interesting.\n+ The experimental settings of card games and polynomials are interesting to study the proposed setting, and the results are promising.\n\nConcerns:\n\n- A key issue is that the paper claims to be the first to propose a method for zero-shot task adaptation (pg 8, para before “Future Directions”). The paper seems to have missed papers such as (1) Zero-Shot Task Transfer, Pal et. al, CVPR 2019, and, (2) Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning, Junhyuk Oh et al, ICML 2017. In particular, the CVPR 2019 paper proposes zero-shot task transfer using meta-mappings. Citing these papers and comparing against them is essential, considering this paper addresses the same objective.\n\n- The proposed methodology does not clearly detail how the zero-shot problem setting is used. Section F.2 (Appendix) gives the formulation of a meta-learner and zero-shot task a bit. However, how the meta-learner performs zero-shot task parameter regression is not clear. The notations used in mathematical equations are also not very clear, and a reader needs to swap between different sections to understand the proposed method. (Moving Sec F.2 to the main paper would be very useful. Similarly, moving Figs 5, 7, 8 etc. to the main paper would have helped greatly too.)\n\n- How is the notion of a “task” defined? Are different classes of a dataset tasks (already done in [3]), or are different vision-related objectives as tasks (reference [1])? Are language-related objectives (such as: document classification, sentiment analysis, Named-entity recognition etc.) tasks? In an RL setting, do zero-shot tasks deal with a new environment? It would have been nice to see clearer definitions for the kind of tasks that this method is suited to.\n\n- It was not clear on how the relationship between source tasks and zero-shot tasks are modeled or computed. Clearly, the performance of such a system depends on the correlations between the tasks, and these details are not clear, especially in generalizing the proposed method. The paper seems to correlate tasks and language-related hints; however getting a language hint is not always available, especially for datasets such as Taskonomy [1, see Refs below] - which is a dataset for multiple tasks.\n\n- The zero-shot part is described using a winning-losing scenario, where we have a partial/side knowledge of winning of task_1 and the model can predict losing strategy. Now, what if:\nWe do not have any knowledge of the winning strategy (or losing strategy) of a zero-shot task?\nCan we predict the losing strategy of a game, let’s say card game, from the knowledge of other games like: soccer, tennis, badminton etc. (the initial motivation of the paper). This is quite possible in the continual learning scenario (the paper mentioned some studies on continual learning). If so, then please comment on the negative transfers in that scenario and how it will affect the inference of the meta-learner.\nThe proposed method also mentioned language descriptions. In case of tasks that are not easy to describe (say, a vision task such as segmentation), how will the proposed method will compute the task-prior of seen-tasks and zero-shot tasks?\nThe paper mentions “winning-losing” strategy and the metanetwork gives losing strategy parameters. What if the scenario is the otehr way - given a losing strategy, we now have to perform the winning strategy (IMHO this has more value in the real world). Does the paper show the result on this, or am I missing something?\n\n- Experiments are limited to synthetic data and card games, and it would be nice to see the validation of the proposed method on different domains. The notion of zero-shot tasks are also not clear for multivariate polynomial experiment and card-game experiment.\n\n- Figure 1: please clearly mention in the caption what is the meaning and purpose of probes. It is not completely clear whether probes (also referred as held-out set In figure 1 caption) are taking part in training, or only at the time of validation?\n\n- Shared Z vs. Separate task embedding: reuse of M (or, H network) network will only be possible if the input and output dimensions are same - i.e. the dimension of Z_output of M is the same as the dimension of Z_input of the M. What if  input and output dimensions are not the same?\n\nOther minor comments:\n\n* One general comment is: please mention the “Zero-shot” experiments more clearly (may be a separate paragraph or subsection), otherwise it is hard to follow.\n* Figure 2(a) please remove the question mark (?) after Held-out. Similarly, Figure 5: Please complete the sentence “ Performing from …”. \n* Figure 1: (a)  Basic meta-learning: Please use some other color to highlight “(a)  Basic meta-learning”, the grey gives an illusion that the step ((a)  Basic meta-learning is not important.\n* A term such as “Homoiconic” would have benefited from a formal definition, especially considering it’s not a commonly used term.\n* The presentation is in general not well-polished, and needs revision.\n\nReferences:\n[1] Taskonomy, CVPR 2018, Zamir et al\n[2] Zero-shot Task Transfer, CVPR 2019, Pal et al\n[3] Learning To Detect Unseen Object Classes by Between-Class Attribute Transfer, CVPR 2009, Lampert et al.\n[4] Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning, ICML 2017, Junhyuk Oh et al.\n\n======= POST-REBUTTAL COMMENTS=========\nI thank the authors for the response. Some of the queries/concerns in the review comments above were clarified. I am updating my rating to Weak Reject, but unfortunately am not completely convinced about accepting the paper in its current form for the following reasons:\n\n- Thank you for pointing out the difference between this work and other recent efforts (Oh et al, 2017 and Pal et al, 2019), and updating the manuscript with this content. However, considering these methods have been proposed, it is only natural to ask that the proposed method needs to be compared against them, to judge the usefulness of meta-mappings as against other ways to achieve zero-shot task adaptation, at least on some common datasets/settings. The paper indeed does propose a new idea, but without a comprehensive comparison/evaluation, the inferences on its usefulness may not be conclusive without such studies.\n\n- Considering methods solving the same/similar problem using other approaches have tested on real-world datasets, experiments on synthetic datasets alone seems limiting.  I agree and appreciate the amount of material that is already in the paper, but it is also important to have the correct material to make conclusive inferences that benefit the community.\n\nI once again thank the authors for the efforts in the response, but just think the paper may benefit from revisions based on the comments shared here, to be more impactful/useful.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "BkerfQqaFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors proposed to address the zero-shot adaptation of tasks by the defined and learned “meta-mappings”. Technically speaking, this work is built upon HyperNetworks, while its major contribution lies in the homoiconic embedding/treatment of a data point, a task, and a meta-mapping. Despite this fascinating and intriguing idea itself, I am still in doubt about its “real” power, considering that a few important references/baselines are missing.\n\nPros:\n-\tThe ideas of introducing a meta-mapping and treating it similarly or the same as a learning a task itself are novel to me.\n-\tThis work innovatively tackles the problem of zero-shot task adaptation, which is indeed challenging.\n-\tThe case study of the proposed framework in the card game is interesting.\n\nCons:\n-\tThe major concern for this work is its lack of discussion and comparison with state-of-the-art meta-learning baselines. \n    o\tIn fact, the meta-mapping this work learns is the relationship between tasks, and the meta-task the authors mentioned is a group of tasks. In this sense, I strongly suggest the authors to go through the recent work [1], and compare with it. Even the datasets constructed in that paper can be used as a benchmark to validate the proposed method in image classification.\n    o\tMore recent work on explicitly learning the embedding of a task should be noted and compared, including [1][2]. The meta-mapping in this work is still based on embedding of a task, i.e., z_func, which is obtained by the function network. So is it possible to replace the embedding with recent SOTA algorithms?\n    o\tOther basic meta-learning algorithms like MAML[3] and many others should still be compared and discussed, especially in the basic meta-learning setting in Figure 2.\n-\tMany important details and ablation studies are missing, making the work less convincing.\n    o\tWhy don’t merge the results Fig. 2 and Fig. 3 and compare them in a single figure, so that the contribution of meta-mappings can be shown. \n    o\tHow do you train the domain-specific encoder and decoder? Are they generalizable to a wide range of more complicated tasks like image classification? \n    o\tIt would be great to analyze the limitations of the proposed algorithm, especially for the case where a newly coming task belonging to an novel and unseen meta-task arrives. How insightful or predictable is the proposed zero-shot adaptation method in these cases?\n    o\tHow do you obtain the lower-bound (solid line) and the upper-bound (dashed line) of the performance in the figures?\n    o\tWill the dimension of Z influence the performance? Since data points, tasks, and meta-mappings contain different amount of information, investigating the effect of the dimension of Z is key for the robustness of the method.\n-\tThe paper is too expatiatory with many confusion notations and redundant parts, which make it quite difficult to follow.\n\nReferences:\n[1] Hierarchically Structured Meta-learning, ICML19\n[2] TADAM: Task dependent adaptive metric for improved few-shot learning, NeurIPS18\n[3] Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "B1ez1LvJcB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method for adapting a model that has been trained to perform one task, so that it can perform a new task (potentially without using any new training data at all—i.e., zero-shot learning). In some ways the presented work is a form of meta-learning or *meta-mapping* as the authors refer to it. The premise of the paper is very interesting and the overall problem is definitely of high interest and high potential impact.\n\nI believe that the presentation of the proposed method can be significantly improved. The method description was a bit confusing and unclear to me. The experimental results presented were all done on small synthetic datasets and it’s hard to evaluate whether the method is practically useful. Furthermore, no comparisons were provided to any baselines/alternative methods. For example, in Sections 4 and 5 I was hoping to see comparisons to methods like MAML. Also, I felt that the proposed approach in Section 5 is very similar to MAML intuitively. This makes a comparison with MAML even more desirable. Without any comparisons it’s hard to tell how difficult the tasks under consideration are and what would amount to good performance on the held-out tasks.\n\nIn summary, I feel the paper tackles an interesting problem with an interesting approach, but the content could be organized much better. Also, this work would benefit significantly from a better experimental evaluation. For these reasons I lean towards rejecting this paper for now, but would love to see it refined for a future machine learning conference.\n\nAlso, the work by Platanios, et al. on contextual parameter generation is very relevant to this work as it tackles multi-task learning using HyperNetworks. It may be worth adding a short discussion/comparison to that work as it also considers zero-shot learning.\n\nMinor comments:\n- Capitalize: “section” -> “Section”, “appendix” -> “Appendix”, “fig.” -> “Figure”. Sometimes these are capitalized, but the use is inconsistent throughout the paper.\n- “Hold-out” vs “held-out”. Be consistent and use “held-out” throughout.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors presents a method for adapting models to new tasks in a zero shot manner using learned meta-mappings.  The reviewers largely agreed that this is an interesting and creative research direction.  However, there was also agreement that the writing was unclear in many sections, that the appropriate metalearning baselines were not compared to, and that the power of the method was unclear due to overly simplistic domains.  While the baseline issue was mostly cleared up in rebuttal and discussion, the other issues remain.  Thus, I recommend rejection at this time.
Decision: Reject