Zero-shot task adaptation by homoiconic meta-mapping

How can deep learning systems flexibly reuse their knowledge ? Toward this goal , we propose a new class of challenges , and a class of architectures that can solve them . The challenges are meta-mappings , which involve systematically transforming task behaviors to adapt to new tasks zero-shot . The key to achieving these challenges is representing the task being performed in such a way that this task representation is itself transformable . We therefore draw inspiration from functional programming and recent work in meta-learning to propose a class of Homoiconic Meta-Mapping ( HoMM ) approaches that represent data points and tasks in a shared latent space , and learn to infer transformations of that space . HoMM approaches can be applied to any type of machine learning task , including supervised learning and reinforcement learning . We demonstrate the utility of this perspective by exhibiting zero-shot remapping of behavior to adapt to new tasks . 1 INTRODUCTION . Humans are able to use and reuse knowledge more flexibly than most deep learning models can ( Lake et al. , 2017 ; Marcus , 2018 ) . The problem of rapid learning has been partially addressed by meta-learning systems ( Santoro et al. , 2016 ; Finn et al. , 2017 ; 2018 ; Stadie et al. , 2018 ; Botvinick et al. , 2019 , see also Section 7 ) . However , humans can use our knowledge of a task to flexibly adapt when the task changes . In particular , we can often perform an altered task zero-shot , that is , without seeing any data at all . For example , once we learn to play a game , we can immediately switch to playing in order to lose , and can perform reasonably on our first attempt . One fundamental reason for this is that humans are aware of what we are trying to compute and why . This allows us to adapt our task representations to perform an altered task . By contrast , most prior work on zero-shot learning relies on generating a completely new representation for a new task , e.g . from a natural language description ( Socher et al. , 2013 ; Oh et al. , 2017 , e.g. ) . These systems must therefore throw away much of their prior knowledge of the task and regenerate it from scratch , rather than just transforming what needs to be transformed . Other approaches use relationships between tasks as an auxiliary learning signal ( Yao et al. , 2019 ; Pal and Balasubramanian , 2019 , e.g . ) , but can not adapt to a new task by transforming their task representations . To address this , it is necessary to represent tasks in a transformable space . This can grant the ability to rapidly and precisely adapt behavior to a new task . In this paper , we propose a new class of tasks based on this idea : meta-mappings , i.e . mappings between tasks ( see below ) . Meta-mappings allow zero-shot performance of a new task based on its relationship to old tasks . To address the challenge of performing meta-mappings , we propose using architectures which essentially take a functional perspective on meta-learning , and exploit the idea of homoiconicity . ( A homoiconic programming language is one in which programs in the language can be manipulated by programs in the language , just as data can . ) By treating both data and task behaviors as functions , we can conceptually think of both data and learned task behaviors as transformable . This yields the ability to not only learn to solve new tasks , but to learn how to transform these solutions in response to changing task demands . We demonstrate that our architectures can flexibly remap their behavior to address the meta-mapping challenge . By allowing the network to recursively treat its task representations as data points , and transform them to produce new task representations , our approach is able to achieve this flexibility parsimoniously . We suggest that approaches like ours will be key to building more intelligent and flexible deep learning systems . 2 META-MAPPING . We propose the meta-mapping challenge . We define a meta-mapping as a task , or mapping , that takes a task as an input , output , or both . These include mapping from tasks to language ( explaining ) , mapping from language to tasks ( following instructions ) , and mapping from tasks to tasks ( adapting behavior ) . While the first two categories have been partially addressed in prior work ( e.g . Hermann et al. , 2017 ; Co-Reyes et al. , 2019 ) , the latter is more novel . ( We discuss the relationship between our work and prior work in Section 7 . ) This adaptation can be cued in several ways , including examples of the mapping ( after winning and losing at poker , try to lose at blackjack ) or natural-language instructions ( “ try to lose at blackjack ” ) . We argue that task-to-task meta-mappings are a useful way to think about human-like flexibility , because a great deal of our rapid adaptation is from a task to some variation on that task . For example , the task of playing go on a large board is closely related to the task of playing go on a small board . Humans can exploit this to immediately play well on a different board , but deep learning models generally have no way to achieve this . We can also adapt in much deeper ways , for example fundamentally altering our value function on a task , such as trying to lose , or trying to achieve some orthogonal goal . While meta-learning systems can rapidly learn a new task from a distribution of tasks they have experience with , this does not fully capture human flexibility . Given appropriate conditioning ( see below ) , our architecture can use meta-mappings to adapt to substantial task alterations zero-shot , that is , without seeing a single example from the new task ( Lake et al. , 2017 ) . Achieving this flexibility to meta-map to new tasks will be an important step toward more general intelligence – intelligence that is not limited to precisely the training tasks it has seen . 3 HOMOICONIC META-MAPPING ( HOMM ) ARCHITECTURE . To address these challenges , we propose HoMM architectures , composed of two components : 1 . Input/output systems : domain specific encoders and decoders ( vision , language , etc . ) that map into a shared embedding space Z . 2 . A meta-learning system that a ) learns to embed tasks into the shared embedding space Z , b ) learns to use these task embeddings to perform task-appropriate behavior , c ) learns to embed meta-mappings into the same space , and d ) learns to use these meta-mapping embeddings to transform basic task embeddings in a meta-mapping appropriate way . These architectures are homoiconic because they have a completely shared Z for individual data points , tasks , and meta-mappings . This parsimoniously allows for arbitrary mappings between these entities . In addition to basic tasks , the system can learn to perform meta-mappings to systematically alter task behavior based on relationships to other tasks . That is , it can transform task representations using the same components it uses to transform basic data points . ( See also Appendix E.1 . ) Without training on meta-mappings , of course , the system will not be able to execute them well . However , as we will show , if it is trained on a broad enough set of such mappings , it will be able to generalize to new instances drawn from the same meta-mapping distribution . For instances that fall outside its data distribution , or for optimal performance , it may require some retraining , however . This reflects the structure of human behavior – we are able to adapt rapidly when new knowledge is relatively consistent with our prior knowledge , but learning an entirely new paradigm ( such as calculus for a new student ) can be quite slow ( cf . Kumaran et al. , 2016 ; Botvinick et al. , 2019 ) . More formally , we treat functions and data as entities of the same type . From this perspective , the data points that one function receives can themselves be functions1 . The key insight is that then our architecture can transform data points2 to perform basic tasks ( as is standard in machine learning ) , 1Indeed , any data point can be represented as a constant function that outputs the data point . 2Where “ data ” is a quite flexible term . The approach is agnostic to whether the learning is supervised or reinforcement learning , whether inputs are images or natural language , etc . but it can also transform these task functions to adapt to new tasks . This is related to the concepts of homoiconicity , defined above , and higher-order functions . Under this perspective , basic tasks and meta-mappings from task to task are really the same type of problem . The functions at one level of abstraction ( the basic tasks ) become inputs and outputs for higher-level functions at the next level of abstraction ( meta-mapping between tasks ) . Specifically , we embed each input , target , or mapping into a shared representational space Z . This means that single data points are embedded in the same space as the representation of a function or an entire dataset . Inputs are embedded by a deep network I : input→ Z . Model outputs are decoded from Z by O : Z → output . Target outputs are encoded by T : targets→ Z . Given this , the task of mapping inputs to outputs can be framed as trying to find a transformation of the representational space that takes the ( embedded ) inputs from the training set to embeddings that will decode to the target outputs . These transformations are performed by a system with the following components ( see Fig . 1 ) : M : { ( Z , Z ) , ... } → Z – the meta network , which collapses a dataset of ( input embedding , target embedding ) pairs to produce a single function embedding . H : Z → parameters – the hyper network , which maps a function embedding to parameters . F : Z → Z – the transformation , implemented by a deep network parameterized byH . Basic meta-learning : To perform a basic task , input and target encoders ( I and T ) are used to embed individual pairs from an example dataset D1 , to form a dataset of example ( input , output ) tuples ( Fig . 1a ) . These examples are fed toM , which produces a function embedding ( via a deep neural network , with several layers of parallel processing across examples , followed by an elementwise max across examples , and several more layers ) . This function embedding is mapped through the hyper networkH to parameterize F , and then F is used to process a dataset of embedded probe inputs , and O to map the resultant embeddings to outputs . This system can be trained end-to-end on target outputs for the probes . Having two distinct datasets forces generalization at the meta-learning level , in exactly the same way as standard meta-learning algorithms learn to generalize . More explicitly , suppose we have a dataset of example input , target pairs ( D1 = { ( x0 , y0 ) , ... } ) , and some input x from a probe dataset D2 . The system would predict a corresponding output ŷ as : ŷ = O ( Fzfunc ( I ( x ) ) ) where Fzfunc is the meta-learner ’ s representation for the function underlying the examples in D1 : Fzfunc is parameterized byH ( zfunc ) , where zfunc =M ( { ( I ( x0 ) , T ( y0 ) ) , ... } ) Then , given some loss function L ( y , ŷ ) defined on a single target output y and an actual model output ŷ , we define our total loss computed on the probe dataset D2 as : E ( x , y ) ∈D2 [ L ( y , O ( FD1 ( I ( x ) ) ) ) ] The system can then be trained end-to-end on this loss to adjust the weights of T , H , M , O , and I . That is , the meta-learning training procedure optimizes the system for generalizing the function implied by the examples it sees , in order to respond appropriately to probe inputs . As in training a standard meta-learning model , these probes are held-out from the meta-network , but the labels are still used to encourage the system to meta-learn to respond appropriately to new inputs . Other data points are held-out entirely during training , and only used for evaluation , see Appendix A.1 for some detailed description . See Appendix F.2 for further details on the architecture and hyper-parameters . Meta-mapping : The fundamental insight of our paper is to show how basic tasks and metamappings can be treated homogenously , by allowing the network to transform its task representations like data ( see Fig . 1b , c ) . That is , we propose to transform a prior task representation in order to perform a related new task zero-shot . The relationship by which the prior task embedding should be transformed to perform the new task is specified by example tuples of ( input , output ) task embeddings . This is precisely analogous to how a basic task is specified in terms of ( input , output ) embeddings of data points , except that the embeddings now represent tasks that are being transformed , rather than individual data points . Thus from the perspective of our architecture , learning a meta-mapping between tasks is exactly analogous to learning a basic task . Anything that is embedded in Z can be transformed using the same system . Because tasks are embedded in Z for basic meta-learning , this allows for meta-mappings using exactly the sameM andH that we use for basic tasks . Just as we would take a set of paired embeddings of data points for a basic task , and use them to compute a function embedding for that task , we can take a set of paired function embeddings representing a meta-mapping , and use them to create an embedding for that meta-mapping . We can then use this meta-mapping embedding to transform the embedding of another basic task . We can thus behave zero-shot on a novel task based on its relationship to a prior task . For example , suppose we have an embedding zgame1 ∈ Z for the task of playing some game , and we want to switch to trying to lose this game . We can generate a meta-mapping embedding zmeta ∈ Z from examples of embeddings generated by the system when it is trying to win and lose various other games : zmeta = M ( { ( ( zgame2 , zgame2 , lose ) , ... } ) . We can generate a new task embedding ẑgame1 , lose ∈ Z : ẑgame1 , lose = Fzmeta ( zgame1 ) where Fzmeta is parameterized byH ( zmeta ) This ẑgames1 , lose can be interpreted as the system ’ s guess at a losing strategy for game 1 . To train a meta-mapping , we minimize the ` 2 loss in the latent space betwen this guessed embedding and the embedding of the target task3 . Whether or not we have such a target embedding , we can 3The gradients do not update the example function embeddings , only the weights of M and H , due to memory contraints . Allowing this might be useful in more complex applications . evaluate how well the system loses with this ẑgame1 , lose strategy , by stepping back down a level of abstraction and actually having it play the game via this embedding ( Fig . 1c ) . This is how we evaluate meta-mapping performance – evaluating the loss of transformed task embeddings on the respective target tasks . Note that this allows for zero-shot performance of the new task , because it simply requires transforming a prior task embedding . Alternatively , we could map from language to a meta-mapping embedding , rather than inducing it from examples of the meta-mapping . This corresponds to the human ability to change behavior in response to instructions . The key feature of our architecture – the fact that tasks , data , and language are all embedded in a shared space – allows for substantial flexibility within a unified framework . Furthermore , our approach is parsimonious . Because it uses the same meta-learner for both basic tasks and meta-mappings , this increased flexibility does not require any added parameters.4

Summary: This paper presents a method for adapting a model that has been trained to perform one task, so that it can perform a new task (potentially without using any new training data at all—i.e., zero-shot learning). In some ways the presented work is a form of meta-learning or *meta-mapping* as the authors refer to it. The premise of the paper is very interesting and the overall problem is definitely of high interest and high potential impact.