Meta-Learning Update Rules for Unsupervised Representation Learning

1 INTRODUCTION . Supervised learning has proven extremely effective for many problems where large amounts of labeled training data are available . There is a common hope that unsupervised learning will prove similarly powerful in situations where labels are expensive , impractical to collect , or where the prediction target is unknown during training . Unsupervised learning however has yet to fulfill this promise . One explanation for this failure is that unsupervised representation learning algorithms are typically mismatched to the target task . Ideally , learned representations should linearly expose high level attributes of data ( e.g . object identity ) and perform well in semi-supervised settings . Many current unsupervised objectives , however , optimize for objectives such as log-likelihood of a generative model or reconstruction error , producing useful representations only as a side effect . Unsupervised representation learning seems uniquely suited for meta-learning ( Hochreiter et al. , 2001 ; Schmidhuber , 1995 ) . Unlike most tasks where meta-learning is applied , unsupervised learning does not define an explicit objective , which makes it impossible to phrase the task as a standard optimization problem . It is possible , however , to directly express a meta-objective that captures the quality of representations produced by an unsupervised update rule by evaluating the usefulness of the representation for candidate tasks . In this work , we propose to meta-learn an unsupervised update rule by meta-training on a meta-objective that directly optimizes the utility of the unsupervised representation . Unlike hand-designed unsupervised learning rules , this meta-objective directly targets the usefulness of a representation generated from unlabeled data for later supervised tasks . By recasting unsupervised representation learning as meta-learning , we treat the creation of the unsupervised update rule as a transfer learning problem . Instead of learning transferable features , we learn a transferable learning rule which does not require access to labels and generalizes across both data domains and neural network architectures . Although we focus on the meta-objective of semi-supervised classification here , in principle a learning rule could be optimized to generate representations for any subsequent task . 2 RELATED WORK . 2.1 UNSUPERVISED REPRESENTATION LEARNING . Unsupervised learning is a topic of broad and diverse interest . Here we briefly review several techniques that can lead to a useful latent representation of a dataset . In contrast to our work , each method imposes a manually defined training algorithm or loss function whereas we learn the algorithm that creates useful representations as determined by a meta-objective . Autoencoders ( Hinton and Salakhutdinov , 2006 ) work by first compressing and optimizing reconstruction loss . Extensions have been made to de-noise data ( Vincent et al. , 2008 ; 2010 ) , as well as compress information in an information theoretic way ( Kingma and Welling , 2013 ) . Le et al . ( 2011 ) further explored scaling up these unsupervised methods to large image datasets . Generative adversarial networks ( Goodfellow et al. , 2014 ) take another approach to unsupervised feature learning . Instead of a loss function , an explicit min-max optimization is defined to learn a generative model of a data distribution . Recent work has shown that this training procedure can learn unsupervised features useful for few shot learning ( Radford et al. , 2015 ; Donahue et al. , 2016 ; Dumoulin et al. , 2016 ) . Other techniques rely on self-supervision where labels are easily generated to create a non-trivial ‘ supervised ’ loss . Domain knowledge of the input is often necessary to define these losses . Noroozi and Favaro ( 2016 ) use unscrambling jigsaw-like crops of an image . Techniques used by Misra et al . ( 2016 ) and Sermanet et al . ( 2017 ) rely on using temporal ordering from videos . Another approach to unsupervised learning relies on feature space design such as clustering . Coates and Ng ( 2012 ) showed that k-means can be used for feature learning . Xie et al . ( 2016 ) jointly learn features and cluster assignments . Bojanowski and Joulin ( 2017 ) develop a scalable technique to cluster by predicting noise . Other techniques such as Schmidhuber ( 1992 ) , Hochreiter and Schmidhuber ( 1999 ) , and Olshausen and Field ( 1997 ) define various desirable properties about the latent representation of the input , such as predictability , complexity of encoding mapping , independence , or sparsity , and optimize to achieve these properties . 2.2 META LEARNING . Most meta-learning algorithms consist of two levels of learning , or ‘ loops ’ of computation : an inner loop , where some form of learning occurs ( e.g . an optimization process ) , and an outer loop or metatraining loop , which optimizes some aspect of the inner loop , parameterized by meta-parameters . The performance of the inner loop computation for a given set of meta-parameters is quantified by a meta-objective . Meta-training is then the process of adjusting the meta-parameters so that the inner loop performs well on this meta-objective . Meta-learning approaches differ by the computation performed in the inner loop , the domain , the choice of meta-parameters , and the method of optimizing the outer loop . Some of the earliest work in meta-learning includes work by Schmidhuber ( 1987 ) , which explores a variety of meta-learning and self-referential algorithms . Similarly to our algorithm , Bengio et al . ( 1990 ; 1992 ) propose to learn a neuron local learning rule , though their approach differs in task and problem formulation . Additionally , Runarsson and Jonsson ( 2000 ) meta-learn supervised learning rules which mix local and global network information . A number of papers propose meta-learning for few shot learning ( Vinyals et al. , 2016 ; Ravi and Larochelle , 2016 ; Mishra et al. , 2017 ; Finn et al. , 2017 ; Snell et al. , 2017 ) , though these do not take advantage of unlabeled data . Others make use of both labeled and unlabeld data ( Ren et al. , 2018 ) . Hsu et al . ( 2018 ) uses a task created with no supervision to then train few-shot detectors . Garg ( 2018 ) use meta-learning for unsupervised learning , primarily in the context of clustering and with a small number of meta-parameters . To allow easy comparison against other existing approaches , we present a more extensive survey of previous work in meta-learning in table form in Table 1 , highlighting differences in choice of task , structure of the meta-learning problem , choice of meta-architecture , and choice of domain . To our knowledge , we are the first meta-learning approach to tackle the problem of unsupervised representation learning , where the inner loop consists of unsupervised learning . This contrasts with transfer learning , where a neural network is instead trained on a similar dataset , and then fine tuned or otherwise post-processed on the target dataset . We additionally believe we are the first representation meta-learning approach to generalize across input data modalities as well as datasets , the first to generalize across permutation of the input dimensions , and the first to generalize across neural network architectures ( e.g . layer width , network depth , activation function ) . 3 MODEL DESIGN . We consider a multilayer perceptron ( MLP ) with parameters φt as the base model . The inner loop of our meta-learning process trains this base model via iterative application of our learned update rule . See Figure 1 for a schematic illustration and Appendix A for a more detailed diagram . In standard supervised learning , the ‘ learned ’ optimizer is stochastic gradient descent ( SGD ) . A supervised loss l ( x , y ) is associated with this model , where x is a minibatch of inputs , and y are the corresponding labels . The parameters φt of the base model are then updated iteratively by performing SGD using the gradient ∂l ( x , y ) ∂φt . This supervised update rule can be written as φt+1 = SupervisedUpdate ( φt , xt , yt ; θ ) , where t denotes the inner-loop iteration or step . Here θ are the meta-parameters of the optimizer , which consist of hyper-parameters such as learning rate and momentum . In this work , our learned update is a parametric function which does not depend on label information , φt+1 = UnsupervisedUpdate ( φt , xt ; θ ) . This form of the update rule is general , it encompasses many unsupervised learning algorithms and all methods in Section 2.1 . In traditional learning algorithms , expert knowledge or a simple hyper-parameter search determines θ , which consists of a handful of meta-parameters such as learning rate and regularization constants . In contrast , our update rule will have orders of magnitude more meta-parameters , including the weights of a neural network . We train these meta-parameters by performing SGD on the sum of the MetaObjective over the course of ( inner loop ) training in order to find optimal parameters θ∗ , θ∗ = argmin θ Etask [ ∑ t MetaObjective ( φt ) ] , ( 1 ) that minimize the meta-objective over a distribution of training tasks . Note that φt is a function of θ since θ affects the optimization trajectory . In the following sections , we briefly review the main components of this model : the base model , the UnsupervisedUpdate , and the MetaObjective . See the Appendix for a complete specification . Additionally , code and meta-trained parameters θ for our meta-learned UnsupervisedUpdate is available1 . 3.1 BASE MODEL . Our base model consists of a standard fully connected multi-layer perceptron ( MLP ) , with batch normalization ( Ioffe and Szegedy , 2015 ) , and ReLU nonlinearities . We chose this as opposed to a convolutional model to limit the inductive bias of convolutions in favor of learned behavior from the UnsupervisedUpdate . We call the pre-nonlinearity activations z1 , · · · , zL , and post-nonlinearity activations x0 , · · · , xL , where L is the total number of layers , and x0 ≡ x is the network input ( raw data ) . The parameters are φ = { W 1 , b1 , V 1 , · · · , WL , bL , V L } , where W l and bl are the weights and biases ( applied after batch norm ) for layer l , and V l are the corresponding weights used in the backward pass . 3.2 LEARNED UPDATE RULE . We wish for our update rule to generalize across architectures with different widths , depths , or even network topologies . To achieve this , we design our update rule to be neuron-local , so that updates are a function of pre- and post- synaptic neurons in the base model , and are defined for any base model architecture . This has the added benefit that it makes the weight updates more similar to synaptic updates in biological neurons , which depend almost exclusively on local pre- and post-synaptic neuronal activity ( Whittington and Bogacz , 2017 ) . In practice , we relax this constraint and incorporate some cross neuron information to decorrelate neurons ( see Appendix G.5 for more information ) . To build these updates , each neuron i in every layer l in the base model has an MLP , referred to as an update network , associated with it , with output hlbi = MLP ( xlbi , zbi l , V l+1 , δl+1 ; θ ) where b indexes the training minibatch . The inputs to the MLP are the feedforward activations ( xl & zl ) defined above , and feedback weights and an error signal ( V l and δl , respectively ) which are defined below . All update networks share meta-parameters θ . Evaluating the statistics of unit activation over a batch of data has proven helpful in supervised learning ( Ioffe and Szegedy , 2015 ) . It has similarly proven helpful in hand-designed unsupervised learning rules , such as sparse coding and clustering . We therefore allow hlbi to accumulate statistics across examples in each training minibatch . During an unsupervised training step , the base model is first run in a standard feed-forward fashion , populating xlbi , z l bi . As in supervised learning , an error signal δ l bi is then propagated backwards through the network . Unlike in supervised backprop , however , this error signal is generated by the corresponding update network for each unit . It is read out by linear projection of the per-neuron hidden state h , δlbi = lin ( hlbi ) , and propogated backward using a set of learned ‘ backward weights ’ ( V l ) T , rather than the transpose of the forward weights ( W l ) T as would be the case in backprop ( diagrammed in Figure 1 ) . This is done to be more biologically plausible ( Lillicrap et al. , 2016 ) . Again as in supervised learning , the weight updates ( ∆W l ) are a product of pre- and post-synaptic signals . Unlike in supervised learning however , these signals are generated using the per-neuron update 1https : //github.com/tensorflow/models/tree/master/research/learning_ unsupervised_learning networks : ∆W lij = func ( hlbi , h l−1 bj , Wij ) . The full weight update ( which involves normalization and decorrelation across neurons ) is defined in Appendix G.5 .

Summary: This paper introduces a novel meta-learning approach to unsupervised representation learning where an update rule for a base model (i.e., an MLP) is meta-learned using a supervised meta-objective (i.e., a few-shot linear regression from the learned representation to classification GTs). Unlike previous approaches, it meta-learns an update rule by directly optimizing the utility of the unsupervised representation using the meta-objective. In the phase of unsupervised representation learning, the learned update rule is used for optimizing a base model without using any other base model objective. Experimental evaluations on few-shot classification demonstrate its generalization performance over different base architectures, datasets, and even domains.