Meta-Learning Neural Procedural Biases

## Abstract

Abstract The goal of few-shot learning is to generalize and achieve high performance on new unseen learning tasks, where each task has only a limited number of examples available. Gradient-based meta-learning attempts to address this challenging task by learning how to learn new tasks by embedding inductive biases informed by prior learning experiences into the components of the learning algorithm. In this work, we build upon prior research and propose Neural Procedural Bias Meta-Learning (NPBML), a novel framework designed to meta-learn task-adaptive procedural biases. Our approach aims to consolidate recent advancements in meta-learned initializations, optimizers, and loss functions by learning them simultaneously and making them adapt to each individual task to maximize the strength of the learned inductive biases. This imbues each learning task with a unique set of procedural biases which is specifically designed and selected to attain strong learning performance in only a few gradient steps. The experimental results show that by meta-learning the procedural biases of a neural network, we can induce strong inductive biases towards a distribution of learning tasks, enabling robust learning performance across many well-established few-shot learning benchmarks.

## 1 Introduction

Humans have an exceptional ability to learn new tasks from only a few examples instances. We can often quickly adapt to new domains effectively by building upon and utilizing past experiences of related tasks, leveraging only a small amount of information about the target domain. The field of meta-learning (Schmidhuber, ; Vanschoren, ; Peng, ; Hospedales et al., ) explores how deep learning techniques, which often require thousands or even millions of observations to achieve competitive performance, can acquire such a capability. In this line of research, the learning process is often framed as a bilevel optimization problem (Bard, ; Maclaurin et al., ; Grefenstette et al., ; Lorraine et al., ). The outer optimization aims to learn the underlying regularities across a distribution of related tasks and embed them into the inductive biases of a learning algorithm. Consequently, in the inner optimization, the learning algorithm is utilized to quickly adapt to new learning tasks using only a few example instances.

Model-Agnostic Meta-Learning (MAML) (Finn et al., ) and its variants (Nichol & Schulman, ; Rajeswaran et al., ; Song et al., ; Triantafillou et al., ), are a popular approach to meta-learning. In MAML, the outer optimization aims to learn the underlying regularities across a set of related tasks and embed them into a shared parameter initialization. This initialization is then used in the inner optimization’s learning algorithm to encourage fast adaptation to new tasks. While successful, these methods resort to simple gradient descent using the cross-entropy loss (for classification) or squared loss (for regression) for the inner learning algorithm. Consequently, subsequent research has extended MAML to meta-learn additional components, such as the learning rate (Behl et al., ; Baik et al., ), gradient-based optimizer (Li et al., ; Lee & Choi, ; Simon et al., ; Flennerhag et al., ; Kang et al., ), loss function (Antoniou & Storkey, ), and more (Antoniou et al., ; Baik et al., ). This enables the meta-learning algorithm to induce stronger inductive biases on the learning algorithm, further enhancing performance.

In this paper, we propose Neural Procedural Bias Meta-Learning (NPBML), a novel gradient-based framework for meta-learning task-adaptive procedural biases for deep neural networks. Procedural biases are the subset of inductive biases that determine the order of traversal over the search space (Gordon & Desjardins, ), they play a central determining role in the convergence, sample efficiency, and generalization of a learning algorithm. As we will show, the procedural biases (Figure 1) are primarily encoded into three fundamental components of a learning algorithm: the loss function, the optimizer, and the parameter initialization. Therefore, we aim to meta-learn these three components to maximize the learning performance when using only a few gradient steps.

To achieve this ambitious goal, we first combine three related research areas into one unified end-to-end framework: MAML-based learned initializations (Finn et al., ), preconditioned gradient descent methods (Lee & Choi, ; Flennerhag et al., ; Kang et al., ), and meta-learned loss functions (Antoniou & Storkey, ; Baik et al., ; Bechtle et al., ; Raymond et al., ; ). We then demonstrate how these meta-learned components can be made task-adaptive through feature-wise linear modulation (FiLM) (Perez et al., ) to facilitate downstream task-specific specialization towards each task. The proposed framework is highly flexible and general, and as we will show many existing gradient-based meta-learning approaches arise as special cases of NPBML. To validate the effectiveness of NPBML, we empirically evaluate our proposed algorithm on four well-established few-shot learning benchmarks. The results show that NPBML consistently outperforms many state-of-the-art gradient-based meta-learning algorithms.

Figure: Figure 1: In NPBML, the procedural biases of a deep neural network are meta-learned. This involves meta-learning three key components: the loss function (left), the parameter initialization (center), and the optimizer (right). By meta-learning these components, a strong inductive bias towards fast adaptation can be induced into the learning algorithm.
Refer to caption: extracted/5661349/figures/npbml-loss-landscapes.png

## 2 Background

In few-shot meta-learning, we are given access to a collection of tasks $\{\mathcal{T}_{1},\mathcal{T}_{2},\dots\}$, otherwise known as a meta-dataset, where each task is assumed to be drawn from a task distribution $p(\mathcal{T})$. Each task $\mathcal{T}_{i}$ contains a support set $\mathcal{D}^{S}$, and a query set $\mathcal{D}^{Q}$ (i.e. a training set and a testing set), where $\mathcal{D}^{S}\cap\mathcal{D}^{Q}=\emptyset$. Each of these sets contains a set of input-output pairs $\{(x_{1},y_{1}),(x_{2},y_{2}),\dots\}$. Let $x\in X$ and $y\in Y$ denote the inputs and outputs, respectively. In few-shot meta-learning, the goal is to learn a model of the form $f_{\theta}(x):X\rightarrow Y$, where $\theta$ are the model parameters. The primary challenge of few-shot learning is that $f_{\theta}$ must be able to quickly adapt to any new task $\mathcal{T}_{i}\sim p(\mathcal{T})$ given only a very limited number of instances. For example, in an $N$-way $K$-shot few-shot classification task, $f_{\theta}$ is only given access to $K$ labeled examples of $N$ distinct classes.

### 2.1 Model Agnostic Meta-Learning

MAML (Finn et al., ) is a highly influential and seminal method for gradient-based few-shot meta-learning. In MAML, the outer learning objective aims to meta-learn a shared parameter initialization $\bm{\theta}$ over a distribution of related tasks $p(\mathcal{T})$. This shared initialization embeds prior knowledge learned from past learning experiences into the learning algorithm such that when a new unseen task is sampled fast adaptation can occur. The outer optimization prototypically occurs by minimizing the sum of the final losses $\sum_{\mathcal{T}_{i}\sim p(\mathcal{T})}\mathcal{L}^{meta}$ on the query set at the end of each task’s learning trajectory using gradient descent as follows:

$$ $\bm{\theta}_{new}=\bm{\theta}-\eta\nabla_{\bm{\theta}}\sum_{\mathcal{T}_{i} \sim p(\mathcal{T})}\left[\mathcal{L}^{meta}\Big{(}\mathcal{D}^{Q}_{i},\theta_ {i,j}(\bm{\theta})\Big{)}\right]$ (1) $$

where $\theta_{i,j}(\bm{\theta})$ refers to the adapted model parameters on the $i^{th}$ task at the $j^{th}$ iterations of the inner update rule $\mathrm{U}^{MAML}$, and $\eta$ is the meta learning rate. The inner optimization for each task starts at the meta-learned parameter initialization $\bm{\theta}$ and applies $\mathrm{U}^{MAML}$ to the parameters $J$ times:

$$ $\theta_{i,j}(\bm{\theta})=\bm{\theta}-\alpha\sum_{j=0}^{J-1}\left[\mathrm{U}^{ MAML}\Big{(}\mathcal{T}_{i},\theta_{i,j}(\bm{\theta})\Big{)}\right].$ (2) $$

In the original version of MAML, the inner update rule $\mathrm{U}^{MAML}$ resorts to simple Stochastic Gradient Descent (SGD) minimizing the loss $\mathcal{L}^{base}$, typically set to the cross-entropy or squared loss, with a fixed learning rate $\alpha$ across all tasks

$$ $\mathrm{U}^{MAML}\Big{(}\mathcal{T}_{i},\theta_{i,j}(\bm{\theta})\Big{)}:= \theta_{i,j}-\alpha\nabla_{\theta_{i,j}}\left[\mathcal{L}^{base}\Big{(} \mathcal{D}^{S}_{i},\theta_{i,j}\Big{)}\right].$ (3) $$

This approach assumes that all tasks in $p(\mathcal{T})$ should use the same fixed learning rule $\mathrm{U}$ in the inner optimization. Consequently, this greatly limits the performance that can be achieved when taking only a small number of gradient steps with few labeled samples available.

## 3 Neural Procedural Bias Meta-Learning

In this work, we propose Neural Procedural Bias Meta-Learning (NPBML), a novel framework that replaces the fixed inner update rule in MAML, Equation (3), with a meta-learned task-adaptive learning rule. This modifies the inner update rule in three key ways:

- 1.
An optimizer is meta-learned by leveraging the paradigm of preconditioned gradient descent. This involves meta-learning a parameterized preconditioning matrix $P_{\bm{\omega}}$ with meta-parameters $\bm{\omega}$ to warp the gradients of SGD.
- 2.
The conventional loss function $\mathcal{L}^{base}$ (i.e. the standard cross-entropy or squared loss) used in the inner optimization is replaced with a meta-learned loss function $\mathcal{M}_{\phi}$, where $\bm{\phi}$ are the meta-parameters.
- 3.
The meta-learned initialization, optimizer, and loss function are adapted to each new task using Feature-Wise Linear Modulation $FiLM_{\psi}$, a general-purpose preconditioning method that has learnable meta-parameters $\bm{\psi}$.

For clarity, we provide a high-level overview of NPBML and contrast it to MAML, before expanding on each of the new components in Sections 3.2, 3.3, and 3.4, respectively. Additionally, pseudocode for the outer and inner optimizations is provided in Algorithm 1 and 2 in Appendix A.

### 3.1 Overview

The central goal of NPBML is to meta-learn a task-adaptive parameter initialization, optimizer, and loss function. This changes the outer optimization previously seen in Equation (1) to the following, where $\bm{\Phi}=\{\bm{\theta},\bm{\omega},\bm{\phi},\bm{\psi}\}$ refers to the set of meta-parameters:

$$ $\bm{\Phi}_{new}=\bm{\Phi}-\eta\nabla_{\Phi}\sum_{\mathcal{T}_{i}\sim p( \mathcal{T})}\left[\mathcal{L}^{meta}\Big{(}\mathcal{D}^{Q}_{i},\theta_{i,j}( \bm{\Phi})\Big{)}\right].$ (4) $$

Unlike MAML which employs a fixed update rule $\mathrm{U}^{MAML}$ for all tasks, simple SGD using $\mathcal{L}^{base}$, NPBML uses a fully meta-learned update rule

$$ $\theta_{i,j}(\bm{\Phi})=\bm{\theta}(\bm{\psi})-\alpha\sum_{j=0}^{J-1}\left[ \mathrm{U}^{NPBML}\Big{(}\mathcal{T}_{i},\theta_{i,j}(\bm{\Phi})\Big{)}\right]$ (5) $$

which adjusts $\theta_{i,j}$ in the direction of the negative gradient of a meta-learned loss function $\mathcal{M}_{\phi}$. Additionally, the gradient is warped via a meta-learned preconditioning matrix $P_{\bm{\omega}}$ as follows:

$$ $\mathrm{U}^{NPBML}\Big{(}\mathcal{T}_{i},\theta_{i,j}(\bm{\Phi})\Big{)}:= \theta_{i,j}-\alpha P_{(\bm{\omega},\bm{\psi})}\nabla_{\theta_{i,j}}\left[ \mathcal{M}_{(\bm{\phi},\bm{\psi})}\Big{(}\mathcal{D}^{S}_{i},\theta_{i,j}\Big {)}\right]$ (6) $$

where both $\mathcal{M}_{\phi}$ and $P_{\bm{\omega}}$ are adapted to each task using $FiLM_{\psi}$. This new task-adaptive learning rule empowers each task with a unique set of procedural biases enabling strong and robust learning performance on new unseen tasks in $p(\mathcal{T})$ using only a few gradient steps as depicted in Figure 2.

Figure: Figure 2: In MAML, the update rule $\mathrm{U}^{MAML}$ optimizes the base model parameters from a shared initialization using simple SGD minimizing $\mathcal{L}^{base}$. In contrast, NPBML adapts the model parameters from a task-adapted initialization using $\mathrm{U}^{NPBML}$, a task-adaptive update rule employing a meta-learned preconditioning matrix $P_{\bm{\omega}}$ and loss function $\mathcal{M}_{\phi}$.
Refer to caption: x1.png

### 3.2 Meta-Learned Optimizer

In NPBML, the paradigm of Preconditioned Gradient Descent (PGD) (Li et al., ; Lee & Choi, ; Park & Oliva, ; Flennerhag et al., ; Simon et al., ; Kang et al., ) is employed to meta-learn a gradient preconditioner $P_{\bm{\omega}}$ that rescales the geometry of the parameter space by modifying the gradient descent update rule as follows:

$$ $\theta_{new}=\theta-\alpha P_{\bm{\omega}}\nabla_{\theta}\mathcal{M}_{(\bm{ \phi},\bm{\psi})}.$ (7) $$

In this work, we take inspiration from T-Nets (Lee & Choi, ) and insert linear projection layers $\bm{\omega}$ into our model $f_{(\bm{\theta},\bm{\omega},\bm{\psi})}$. The model, composed of an encoder $z$ and a classification head $h$,

$$ $f_{(\bm{\theta,\bm{\omega},\psi})}=z_{(\bm{\theta,\bm{\omega},\psi})}\circ h_{\theta}$ (8) $$

is interleaved with linear projection layers $\bm{\omega}$ between each layer in the $L$ layer encoder (where $\sigma$ refers to the non-linear activation functions):

$$ $z_{(\bm{\theta},\bm{\omega},\bm{\psi})}(x_{i})=\sigma^{(L)}(\bm{\theta}^{(L)} \bm{\omega}^{(L)}(\dots\sigma^{(1)}(\bm{\omega}^{(1)}\bm{\theta}^{(1)}x)\dots)).$ (9) $$

As described in Equations (4)–(6), $\bm{\omega}$ is meta-learned in the outer loop and held fixed in the inner loop such that preconditioning of the gradients occurs. This form of precondition defines $P_{\bm{\omega}}$ as a block-diagonal matrix, where each block is defined by the expression $(\bm{\omega}\bm{\omega}^{\mathsf{T}})$, as shown in (Lee & Choi, ). For a simple model where $f_{(\theta,\bm{\omega})}(x)=\bm{\omega}\theta x$, the update rule becomes:

$$ $\theta_{new}=\theta-\alpha(\bm{\omega}\bm{\omega}^{\mathsf{T}})\nabla_{\theta} \mathcal{M}_{(\bm{\phi},\bm{\psi})}.$ (10) $$

We leverage this style of parameterization for $P_{\bm{\omega}}$ due to its relative simplicity and high expressive power. However, we emphasize that the NPBML framework is highly general, and other forms of preconditioning, such as those presented in (Lee & Choi, ; Park & Oliva, ; Flennerhag et al., ; Simon et al., ), could be used instead.

### 3.3 Meta-Learned Loss Function

Unlike MAML, which defines the inner optimization’s loss function $\mathcal{L}^{base}$ to be the cross-entropy or squared loss for all tasks, NPBML uses a meta-learned loss function $\mathcal{M}_{(\bm{\phi},\bm{\psi})}$ that is learned in the outer optimization. In contrast to handcrafted loss functions, which typically consider only the ground truth label $y$ and the model predictions $f_{(\theta,\bm{\omega},\bm{\psi})}(x)$, meta-learned loss functions can, in principle, be conditioned on any task-related information (Bechtle et al., ; Baik et al., ). In NPBML, the meta-learned loss function is conditioned on three distinct sources of task-related information, which are subsequently processed by three small feed-forward neural networks:

$$ $\mathcal{M}_{(\bm{\phi},\bm{\psi})}=\mathcal{L}^{S}_{(\bm{\phi},\bm{\psi})}+ \mathcal{L}^{Q}_{(\bm{\phi},\bm{\psi})}+\mathcal{R}_{(\bm{\phi},\bm{\psi})}.$ (11) $$

First, $\mathcal{L}^{S}:\mathbb{R}^{2N+1}\rightarrow\mathbb{R}^{1}$ is an inductive loss function conditioned on task-related information derived from the support set; namely, the one-hot encoded ground truth target and model predictions, and the corresponding loss calculated using $\mathcal{L}^{base}$. Next, $\mathcal{L}^{Q}:\mathbb{R}^{2N+1}\rightarrow\mathbb{R}^{1}$ is a transductive loss function conditioned on task-related information derived from the query set. Here, we give $\mathcal{L}^{Q}$ access to the model predictions on the query set, embeddings (i.e., relation scores) from a pre-trained relation network (Sung et al., ), and the corresponding loss between the model predictions and embeddings using $\mathcal{L}^{base}$. Note that similar embedding functions have previously been used in (Rusu et al., ; Antoniou & Storkey, ). Finally, the adapted model parameters $\theta_{i,j}$ are used as inputs to meta-learn a weight regularizer $\mathcal{R}:\mathbb{R}^{4L}\rightarrow\mathbb{R}^{1}$. To improve efficiency, we condition $\mathcal{R}$ on the mean, standard deviation, L$1$, and L$2$ norm of each layer’s weights, as opposed to $\theta_{i,j}$ directly.

### 3.4 Task-Adaptive Modulation

Figure: Figure 3: An example of a two-layer convolutional neural network in NPBML, where layers $\theta^{(1)}$ and $\theta^{(2)}$, are interleaved with warp preconditioning layers $\bm{\omega}^{(1)}$ and $\bm{\omega}^{(2)}$. Both types of layers are modulated in the inner loop using feature-wise linear modulation layers to induce task adaptation.
Refer to caption: x2.png

Figure: Figure 4: An example of one of the meta-learned loss functions in NPBML, where the loss function represented as a composition of feed-forward (linear) layers are modulated using feature-wise linear modulation, resulting in a task-adaptive meta-learned loss function.
Refer to caption: x3.png

Although all tasks in few-shot learning are assumed to be sampled from the same task distribution $p(\mathcal{T})$, the optimal parameter initialization, optimizer, and loss function may differ between tasks. Therefore, in NPBML the meta-learned components $\bm{\Phi}$ are modulated for each new task, providing each task with a unique set of task-adaptive procedural biases. To achieve this, Feature-wise Linear Modulation (FiLM) layers (Perez et al., ; Dumoulin et al., ) are inserted into both the encoder $z_{(\bm{\theta},\bm{\omega},\bm{\psi})}$ and the meta-learned loss function $M_{(\bm{\phi},\bm{\psi})}$ as shown in Figures 3 and 4. Where $FiLM_{\psi}$ is defined as follows, where $\gamma$ and $\beta$ are the scaling and shifting vectors, respectively, and $\psi$ are the meta-learned parameters:

$$ $FiLM_{\psi}(x)=(\gamma_{\psi}(x)+\bm{1})\odot x+\beta_{\psi}(x).$ (12) $$

Affine transformations conditioned on some input have become increasingly popular, being used by several few-shot learning works to make the learned component adaptive (Oreshkin et al., ; Jiang et al., ; Vuorio et al., ; Zintgraf et al., ; Baik et al., ; ). Furthermore, FiLM layers can help alleviate issues related to batch normalization (Ioffe & Szegedy, ), which have been empirically observed to cause training instability due to different distributions of features being passed through the same model in few-shot learning (De Vries et al., ; Antoniou et al., ). In our work, we have found that conditioning the FiLM on the output activations of the previous layers is an effective way to achieve task adaptability. This form of conditioning is in essence a simplified version of that used in CNAPs (Requeima et al., ); however, we have omitted the use of global embeddings as we found it was not necessary for our method.

### 3.5 Initialization

Due to the large number of learnable meta-parameters, initialization becomes an important and necessary aspect to consider. Here we detail how to initialize each of the meta-learned components, i.e., $\bm{\Phi}_{0}=\{\bm{\theta_{0}},\bm{\omega}_{0},\bm{\phi}_{0},\bm{\psi}_{0}\}$, in NPBML. Firstly, we pre-train the encoder weights $\bm{\theta}_{0}$ prior to meta-learning, following many recent methods in few-shot learning (Rusu et al., ; Qiao et al., ; Requeima et al., ; Ye et al., ; Ye & Chao, ), see Appendix A.3 for more details. For the linear projection layers $\bm{\omega}$, we leverage the fact that in PGD, setting $P_{\bm{\omega}}$ to the identity $\bm{I}$ recovers SGD. Therefore, we set $\smash{\forall l\in\{1,\dots,L\}:\bm{\omega}^{(l)}=\bm{I}}$; note for convolutional layers this corresponds to Dirac initialization. Regarding the meta-learned loss function $\mathcal{M}_{(\bm{\phi},\bm{\psi})}$, the weights $\bm{\phi}$ at the start of meta-training are randomly initialized $\smash{\bm{\phi}_{0}\sim\mathcal{N}(0,1e^{-2})}$; therefore, the $\mathbb{E}[\mathcal{M}_{(\bm{\phi}_{0},\bm{\psi}_{0})}]=0$, assuming an identity output activation. Consequently, the definition of the meta-learned loss function in Equation (11) can be modified to

$$ $\mathcal{M}_{(\bm{\phi},\bm{\psi})}=\mathcal{L}^{base}+\mathcal{L}^{S}_{(\bm{ \phi},\bm{\psi})}+\mathcal{L}^{Q}_{(\bm{\phi},\bm{\psi})}+\mathcal{R}_{(\bm{ \phi},\bm{\psi})}$ (13) $$

such that the meta-learned loss function approximately recovers the base loss function at the start of meta-training, i.e., $\mathcal{M}_{(\bm{\phi}_{0},\bm{\psi}_{0})}\approx\mathcal{L}^{base}$. Finally, the FiLM layers in NPBML are initialized using a similar strategy taking advantage of the fact that when $\bm{\psi}_{0}\sim\mathcal{N}(0,1e-2)$ the $\mathbb{E}[\gamma_{\bm{\psi}_{0}}(x)]=\mathbb{E}[\beta_{\bm{\psi}_{0}}(x)]=0$; consequently, $FiLM_{\bm{\psi}_{0}}(x)\approx x$. When initialized in this manner the update rule for NPBML at the start of meta-training closely approximates the update rule of MAML

$$ $\mathrm{U}^{MAML}\Big{(}\mathcal{T}_{i},\theta_{i,j}(\bm{\theta_{0}})\Big{)} \approx\mathbb{E}\left[\mathrm{U}^{NPBML}\Big{(}\mathcal{T}_{i},\theta_{i,j}( \bm{\Phi_{0}})\Big{)}\right].$ (14) $$

## 4 Implicit Meta-Learning

In NPBML, the parameter initialization, gradient-based optimizer, and loss function are explicitly meta-learned in the outer loop. Critically, we make a novel observation that many other key procedural biases are also implicitly learned by meta-learning these three fundamental components (hence the name given to our algorithm). For example, consider the scalar learning rate $\alpha$, which is implicitly meta-learned since the following equality holds:

$$ $\exists\alpha\exists\bm{\phi}:\theta_{i,j}-\alpha\nabla_{\theta_{i,j}}\mathcal {L}^{base}\approx\theta_{i,j}-\nabla_{\theta_{i,j}}\mathcal{M}_{\bm{\phi}},$ (15) $$

and if $\bm{\phi}$ is made to adapt on each inner step as done in (Baik et al., ; Raymond et al., ), then by extension NPBML also learns a learning rate schedule. Another straightforward related observation is that NPBML implicitly learns a layer-wise learning rate $\smash{\{\alpha^{(1)},\dots,\alpha^{(L)}\}}$, since for each block $\smash{\{\bm{\omega}^{(1)},\dots,\bm{\omega}^{(L)}\}}$ in the block diagonal preconditioning matrix $P_{\bm{\omega}}$ the following holds:

$$ $\forall l(\exists\bm{\omega}^{(l)}\exists\alpha^{(l)}):(\bm{\omega}^{(l)}(\bm{ \omega}^{(l)})^{\mathsf{T}})\nabla_{\theta^{(l)}}\mathcal{M}_{\bm{\phi}} \approx\alpha^{(l)}\nabla_{\theta^{(l)}}\mathcal{M}_{\bm{\phi}}.$ (16) $$

There are also less obvious connections that can be drawn. For example, by transitive relation, NPBML implicitly learns the regularization behavior of the batch size hyperparameter, since as shown in (Smith et al., ; Smith & Le, ; Goyal et al., ) there exists a linear scaling rule between the batch size and the learning rate with respect to the “noise scale”, which controls the magnitude of random fluctuations in the training dynamics of SGD. Another example is label smoothing regularization (Müller et al., ) which as proven in (Gonzalez & Miikkulainen, ), can be implicitly induced when meta-learning a loss function.

## 5 Related Work

Meta-learning approaches to few-shot learning aim to equip models with the ability to quickly adapt to new tasks given only a limited number of examples by leveraging prior learning experiences across a distribution of related tasks. These approaches are commonly partitioned into three categories: (1) metric-based methods, which aim to learn a similarity metric for efficient class differentiation (Koch et al., ; Vinyals et al., ; Sung et al., ; Snell et al., ); (2) memory-based methods, which utilize architectures that store training examples in memory or directly encode fast adaptation algorithms in the model weights (Santoro et al., ; Ravi & Larochelle, ); and (3) optimization-based methods, which aim to learn an optimization algorithm specifically designed for fast adaptation and few-shot learning (Finn et al., ). This work explored the latter approach by meta-learning a gradient update rule.

MAML (Finn et al., ), a highly flexible task and model-agnostic method for meta-learning a parameter initialization from which fast adaptation can occur. Many follow-up works have sought to enhance MAML’s performance by addressing limitations in the outer-optimization algorithm, such as the memory and compute efficiency (Nichol & Schulman, ; Rajeswaran et al., ; Raghu et al., ; Oh et al., ), or the meta-level overfitting (Rusu et al., ; Flennerhag et al., ). Relatively fewer works have focused on enhancing the inner-optimization update rule, as is done in our work. Most MAML variants continue to use an inner update rule consisting of SGD with a fixed learning rate minimizing a loss function such as the cross-entropy or squared loss.

Of the works that have explored improving the inner update rule, the vast majority focus on improving the optimizer. For example, early MAML-based methods such as (Behl et al., ; Antoniou et al., ; Li et al., ) explored meta-learning the scalar, layer-wise, and parameter-wise learning rates, respectively. More recent methods have explored more powerful parameterization for the meta-learned optimizer through the utilization of preconditioned gradient descent methods (Lee & Choi, ; Park & Oliva, ; Flennerhag et al., ; Simon et al., ; Kang et al., ), which rescale the geometry of the parameter space by modifying the update rule with a learned preconditioning matrix. While these methods have advanced MAML-based few-shot learning, they often lack a task-adaptive property, falsely assuming that all tasks should use the same optimizer.

A small number of recent works have also investigated replacing the inner loss function (e.g., cross-entropy loss) with a meta-learned loss function. In (Antoniou & Storkey, ), a fully transductive loss function represented as a dilated convolutional neural network is meta-learned. Meanwhile, in (Baik et al., ), a set of loss functions and task adapters are meta-learned for each step taken in the inner optimization. Although these approaches have shown a lot of promise, their potential has yet to be fully realized, as they have not yet been meta-learned in tandem with the optimizer as we have done in this work.

## 6 Experimental Evaluation

In this section, we evaluate the performance of the proposed method on a set of well-established few-shot learning benchmarks. The experimental evaluation aims to answer the following key questions: (1) Can NPBML perform well across a diverse range of few-shot learning tasks? (2) Do the novel components meta-learned in NPBML individually enhance performance? (3) To what extent does each component synergistically contribute to the overall performance of the proposed algorithm?

### 6.1 Results and Analysis

To evaluate the performance of NPBML, experiments are performed on four well-established few-shot learning datasets: mini-Imagenet (Ravi & Larochelle, ), tiered-ImageNet (Ren et al., ), CIFAR-FS (Bertinetto et al., ), and FC-100 (Oreshkin et al., ). For each dataset, experiments are performed using a 5-way 1-shot and 5-way 5-shot configuration. Results are also reported on both the 4-CONV (Finn et al., ; Zintgraf et al., ; Flennerhag et al., ; Kang et al., ) and ResNet-12 (He et al., ; Baik et al., ; ) network architectures. The full details of all experiments, including a comprehensive description of all datasets, models, and training hyperparameters, can be found in Appendix A. The code for our experiments can be found at [https://github.com/*redacted*](https://github.com/*redacted*)

#### 6.1.1 Mini-ImageNet and Tiered-ImageNet

**Table 1: Few-shot classification meta-testing accuracy on 5-way 1-shot and 5-way 5-shot mini-ImageNet and tiered-ImageNet where $\pm$ represents the 95% confidence intervals.**
| Method | Base Learner | mini-ImageNet (5-way) | tiered-ImageNet (5-way) |  |  |
| --- | --- | --- | --- | --- | --- |
| 1-shot | 5-shot | 1-shot | 5-shot |  |  |
| MAML^1 | 4-CONV | 48.70$\pm$1.84% | 63.11$\pm$0.92% | 50.98$\pm$0.26% | 66.25$\pm$0.19% |
| MetaSGD^2 | 4-CONV | 50.47$\pm$1.87% | 64.03$\pm$0.94% | - | - |
| T-Net^3 | 4-CONV | 50.86$\pm$1.82% | - | - | - |
| MAML++^4 | 4-CONV | 52.15$\pm$0.26% | 68.32$\pm$0.44% | - | - |
| SCA^5 | 4-CONV | 54.84$\pm$0.99% | 71.85$\pm$0.53% | - | - |
| WarpGrad^7 | 4-CONV | 52.30$\pm$0.80% | 68.40$\pm$0.60% | 57.20$\pm$0.90% | 74.10$\pm$0.70% |
| ModGrad^8 | 4-CONV | 53.20$\pm$0.86% | 69.17$\pm$0.69% | - | - |
| MeTAL^9 | 4-CONV | 52.63$\pm$0.37% | 70.52$\pm$0.29% | 54.34$\pm$0.31% | 70.40$\pm$0.21% |
| ALFA^10 | 4-CONV | 50.58$\pm$0.51% | 69.12$\pm$0.47% | 53.16$\pm$0.49% | 70.54$\pm$0.46% |
| GAP^11 | 4-CONV | 54.86$\pm$0.85% | 71.55$\pm$0.61% | 57.60$\pm$0.93% | 74.90$\pm$0.68% |
| \hdashline<br>NPBML | 4-CONV | 57.49$\pm$0.83% | 75.01$\pm$0.64% | 64.24$\pm$0.97% | 79.17$\pm$0.71% |
| MAML^1 | ResNet-12 | 58.60$\pm$0.42% | 69.54$\pm$0.38% | 59.82$\pm$0.41% | 73.17$\pm$0.32% |
| MC^6 | WRN-28-10 | - | - | 64.40$\pm$0.10% | 80.21$\pm$0.10% |
| ModGrad^8 | WRN-28-10 | - | - | 65.72$\pm$0.21% | 81.17$\pm$0.20% |
| MeTAL^9 | ResNet-12 | 59.64$\pm$0.38% | 76.20$\pm$0.19% | 63.89$\pm$0.43% | 80.14$\pm$0.40% |
| ALFA^10 | ResNet-12 | 59.74$\pm$0.49% | 77.96$\pm$0.41% | 64.62$\pm$0.49% | 82.48$\pm$0.38% |
| \hdashline<br>NPBML | ResNet-12 | 61.59$\pm$0.80% | 78.18$\pm$0.60% | 72.22$\pm$0.96% | 85.41$\pm$0.61% |

#### 6.1.2 CIFAR-FS and FC-100

**Table 2: Few-shot classification meta-testing accuracy on 5-way 1-shot and 5-way 5-shot CIFAR-FS and FC-100 where $\pm$ represents the 95% confidence intervals.**
| Method | Base Learner | CIFAR-FS (5-way) | FC-100 (5-way) |  |  |
| --- | --- | --- | --- | --- | --- |
| 1-shot | 5-shot | 1-shot | 5-shot |  |  |
| MAML^1 | 4-CONV | 57.63$\pm$0.73% | 73.95$\pm$0.84% | 35.89$\pm$0.72% | 49.31$\pm$0.47% |
| BOIL^2 | 4-CONV | 58.03$\pm$0.43% | 73.61$\pm$0.32% | 38.93$\pm$0.45% | 51.66$\pm$0.32% |
| MeTAL^3 | 4-CONV | 59.16$\pm$0.56% | 74.62$\pm$0.42% | 37.46$\pm$0.39% | 51.34$\pm$0.25% |
| ALFA^4 | 4-CONV | 59.96$\pm$0.49% | 76.79$\pm$0.42% | 37.99$\pm$0.48% | 53.01$\pm$0.49% |
| \hdashline<br>NPBML | 4-CONV | 64.90$\pm$0.94% | 79.24$\pm$0.69% | 40.56$\pm$0.76% | 53.48$\pm$0.68% |
| MAML^1 | ResNet-12 | 63.81$\pm$0.54% | 77.07$\pm$0.42% | 37.29$\pm$0.40% | 50.70$\pm$0.35% |
| MeTAL^3 | ResNet-12 | 67.97$\pm$0.47% | 82.17$\pm$0.38% | 39.98$\pm$0.39% | 53.85$\pm$0.36% |
| ALFA^4 | ResNet-12 | 66.79$\pm$0.47% | 83.62$\pm$0.37% | 41.46$\pm$0.49% | 55.82$\pm$0.50% |
| \hdashline<br>NPBML | ResNet-12 | 69.30$\pm$0.91% | 83.72$\pm$0.64% | 43.63$\pm$0.71% | 59.85$\pm$0.70% |

### 6.2 Ablation Studies

To further investigate the performance of the proposed method, we conduct two ablation studies to analyze the effectiveness of each component. All ablation experiments were performed using the 4-CONV network architecture in a 5-way 5-shot setting on the mini-ImageNet dataset.

#### 6.2.1 Meta-Learned Components

First, we examine the importance of the meta-learned optimizer $P_{\bm{\omega}}$, loss function $\mathcal{M}_{\bm{\phi}}$, and task-adaptive conditioning method $FILM_{\bm{\psi}}$. The results are presented in Table 6.2.2, and they demonstrate that each of the proposed components clearly and significantly contributes to the performance of NPBML. In (2) MAML is modified to include gradient preconditioning, which increases accuracy by $2.09\%$. Conversely in (3) we modify MAML with our meta-learned loss function, resulting in a $6.37\%$ performance increase. Interestingly, the meta-learned loss function enhances performance by a larger margin; however, this may be due to the relatively simple T-Net style optimizer used in NPBML. This suggests that a more powerful parameterization, such as (Flennerhag et al., ) or (Kang et al., ), may further improve performance. In (4), MAML is modified to include both the optimizer and loss function, resulting in a $7.41\%$ performance increase. This further supports our claim that meta-learning both an optimizer and a loss function are complementary and orthogonal approaches to improving MAML. Finally, in (5), we add our task-adaptive conditioning method, increasing performance by $2.22\%$ over the prior experiment and $9.63\%$ over MAML.

#### 6.2.2 Meta-Learned Loss Function

The prior ablation study shows that the meta-learned loss function $\smash{\mathcal{M}_{\bm{\phi}}}$ is a crucial component in NPBML. Therefore, we further investigate each of the components; namely, the meta-learned inductive and transductive loss functions, and weight regularizer. The results are presented in Table 6.2.2, and surprisingly, they show that each of the components in isolation (7), (8), and (9), improves performance by approximately $5\%$. However, when combined in (10), the total performance increase is $6.37\%$. We hypothesize that this result is a consequence of the implicit meta-learning of the learning rate identified in Equation (15), which not only holds for $\smash{\mathcal{M}_{\bm{\phi}}}$, but also for each of its components, i.e., the equality is also true when $\smash{\mathcal{M}_{\bm{\phi}}}$ is replaced with $\smash{\mathcal{L}^{S}_{\bm{\phi}}}$, $\smash{\mathcal{L}^{Q}_{\bm{\phi}}}$, or $\smash{\mathcal{R}_{\bm{\phi}}}$. Since all components share implicit learning rate tuning, the performance gains from this behavior do not accumulate; however, the improvement in (10) is better than each component in isolation indicating that each component provides additional unique benefits to the meta-learning process.

**Table 3: Ablation study of the meta-learned components in NPBML, reporting the meta-testing accuracy on mini-ImageNet 5-way 5-shot. A ✓denotes that the component is meta-learned, with variant (1) reducing to MAML, while variant (5) represents our final proposed algorithm.**
|  | Initialization | Optimizer | Loss Function | Task-Adaptive | Accuracy |
| --- | --- | --- | --- | --- | --- |
| (1) | ✓ |  |  |  | 65.38$\pm$0.67% |
| (2) | ✓ | ✓ |  |  | 67.47$\pm$0.68% |
| (3) | ✓ |  | ✓ |  | 71.75$\pm$0.69% |
| (4) | ✓ | ✓ | ✓ |  | 72.79$\pm$0.67% |
| \hdashline<br>(5) | ✓ | ✓ | ✓ | ✓ | 75.01$\pm$0.64% |

## 7 Conclusion

In this work, we propose a novel meta-learning framework for learning the procedural biases of a deep neural network. The proposed technique, Neural Procedural Bias Meta-Learning (NPBML), consolidates recent advancements in MAML-based few-shot learning methods by replacing the fixed inner update rule with a fully meta-learned update rule. This is achieved by meta-learning a task-adaptive loss function, optimizer, and parameter initialization. The experimental results confirm the effectiveness and scalability of the proposed approach, demonstrating strong few-shot learning performance across a range of popular benchmarks. We believe NPBML provides a principled framework for advancing general-purpose meta-learning in deep neural networks. Looking ahead, numerous compelling future research directions exist, such as developing more powerful parameterization for the meta-learned optimizer or introducing additional arguments to the meta-learned loss function. Furthermore, broadening the scope of the proposed framework to encompass the related domains of cross-domain few-shot learning and continual learning, may be a promising avenue for future exploration.

## Appendix A Experimental Setup

In this section, we provide a comprehensive description of the experimental settings used in this work. Section A.1 offers an overview of the datasets utilized, Section A.2 discusses the network architectures, and Section A.3 details the hyperparameters specific to our proposed method, NPBML. For further details, please refer to Algorithms 1 and 2, as well as our code made available at: [https://github.com/*redacted*](https://github.com/*redacted*)

Figure: Algorithm 1 Meta-Learning (Outer Loop)

Figure: Algorithm 2 Base-Learning (Inner Loop)

### A.1 Dataset Configurations

The few-shot learning experiments follow a prototypical episodic learning setup (Ravi & Larochelle, ), where each dataset contains a set of non-overlapping meta-training, meta-validation, and meta-testing tasks. Each task $\mathcal{T}_{i}$ has a support $\mathcal{D}^{S}$ and query $\mathcal{D}^{Q}$ set (i.e., training and testing set, respectively). For an $N$-way $K$-shot classification task, the support set contains $N\times K$ instances, and the query set contains $N\times 15$ instances, where $N$ refers to the number of randomly sampled classes, and $K$ refers to the number of instances (i.e., shots) available from each of those classes. For example, in a $5$-way $5$-shot classification task, the support set contains $5\times 5=|\mathcal{D}^{S}|$ instances to train on, and the query set contains $5\times 15=|\mathcal{D}^{Q}|$ instances to validate the performance on.

#### A.1.1 ImageNet Datasets

Two commonly used datasets for few-shot learning are two ImageNet derivatives (Deng et al., ): mini-ImageNet (Ravi & Larochelle, ) and tiered-ImageNet (Ren et al., ). Both datasets are composed of three subsets (training, validation, and testing), each of which consists of images with a size of $84\times 84$. The two datasets differ in regard to how the classes are partitioned into mutually exclusive subsets.

mini-ImageNet randomly samples $100$ classes from the $1000$ base classes in Imagenet. Following (Vinyals et al., ) the sampled classes are partitioned such that $64$ classes are allocated for meta-training, $16$ for meta-validation, and $20$ for meta-testing, where for each class $600$ images are available.

tiered-ImageNet as described in (Ren et al., ) alternatively stratifies $608$ classes into $34$ higher-level categories in the ImageNet human-curated hierarchy. The $34$ classes are partitioned into $20$ categories for meta-training, $6$ for meta-validation, and $8$ for meta-testing. Each class in tiered-ImageNet has a minimum of $732$ instances and a maximum of $1300$.

#### A.1.2 CIFAR-100 Datasets

Additional experiments are also conducted on CIFAR-FS (Bertinetto et al., ) and FC-100 (Oreshkin et al., ), which are few-shot learning datasets derived from the popular CIFAR-100 dataset (Krizhevsky & Hinton, ). The CIFAR100 dataset contains $100$ classes containing $600$ images each, each with a resolution of $32\times 32$.

CIFAR-FS uses a similar sampling procedure to mini-ImageNet (Ravi & Larochelle, ), CIFAR-FS is derived by randomly sampling $100$ classes from the $100$ base classes in CIFAR100. The sampled classes are partitioned such that $60$ classes are allocated for meta-training, $20$ for meta-validation, and $20$ for meta-testing.

FC-100 is obtained by using a dataset construction process similar to tiered-ImageNet (Ren et al., ), in which class hierarchies are used to partition the original dataset to simulate more challenging few-shot learning scenarios. In FC100 there are a total of $20$ high-level classes, where $12$ classes are allocated for meta-training, $4$ for meta-validation, and $4$ for meta-testing.

### A.2 Network Architectures

4-CONV: Following the encoder architecture settings in (Zintgraf et al., ; Flennerhag et al., ; Kang et al., ), the network consists of four modules. Each module contains a $3\times 3$ convolutional layer with 128 filters, followed by a batch normalization layer, a ReLU non-linearity, and a $2\times 2$ max-pooling downsampling layer. Following the four modules, we apply average pooling and flatten the embedding to a size of 128.

ResNet-12: The encoder $z_{\theta}$ follows the standard network architecture settings used in (Baik et al., ; ). The network, similar to 4-CONV, consists of four modules. Each module contains a stack of three $3\times 3$ convolutional layers, where each layer is followed by batch normalization and a leaky ReLU non-linearity. A skip convolutional over the convolutional stack is used in each module. Each skip connection contains a $1\times 1$ convolutional layer followed by batch normalization. Following the convolutional stack and skip connection, a leaky ReLU non-linearity and $2\times 2$ max-pooling downsampling layer are placed at the end of each module. The number of filters used in each module is $[64,128,256,512]$, respectively. Following the four modules, we apply average pooling and flatten the embedding to a size of 512.

Classifier: As MAML and its variants are sensitive to label permutation during meta-testing, we opt to use the permutation invariant classification head proposed by (Ye & Chao, ). This replaces the typical classification head $h_{\theta}\in\mathbb{R}^{in\times N}$ with a single-weight vector $\theta_{head}\in\mathbb{R}^{in\times 1}$ which is meta-learned at meta-training time. During meta-testing the weight vector is duplicated into each output, i.e., $h_{\theta}=\{\theta_{c}=\theta_{head}\}^{N}_{c=1}$, and adapted in an identical fashion to traditional MAML. This reduces the number of parameters in the classification head and makes NPBML permutation invariant similar to MetaOptNet (Lee et al., ), CNAPs (Requeima et al., ), and ProtoMAML (Triantafillou et al., ).

Loss Network: The meta-learned loss function $\mathcal{M}$ is composed of three separate networks $\mathcal{L}^{S}$, $\mathcal{L}^{Q}$, and $\mathcal{R}$ whose scalar outputs are summed to produce the final output as shown in Equation (11). The network architecture for all three networks is identical except for the input dimension of the first layer as discussed in Section 3.3. The network architecture is taken from (Bechtle et al., ; Psaros et al., ; Raymond et al., ) and is a small feedforward rectified linear neural network with two hidden layers and 40 hidden units in each layer. Note $\mathcal{L}^{S}$ and $\mathcal{L}^{Q}$ are applied instance-wise and reduced using a mean reduction, such that they are both invariant to the number of instances made available at meta-testing time in the support and query set.

Relation Network: The transductive loss $L^{Q}$ takes an embedding from a pre-trained relation network (Sung et al., ) as one of its inputs, similar to (Rusu et al., ; Antoniou & Storkey, ). The relation network used in our experiments employs the previously mentioned ResNet-12 encoder to generate a set of embeddings for each instance in $D^{S}\cup D^{Q}$. These embeddings are then concatenated and processed using a relation module. The relation module in our experiments consists of two ResNet blocks with $2\times 512$ and $512$ filters, respectively. The output is then flattened and averaged pooled to an embedding size of $512$, which is passed through a final linear layer of size $512\times 1$ which outputs each relation score. In regard to training, the relation network is trained using the settings described in (Sung et al., ).

### A.3 Proposed Method Settings

#### A.3.1 Pre-training Settings

In NPBML, the encoder portion of the network $z_{\theta}$ is pre-trained prior to meta-learning, following many recent methods in few-shot learning (Rusu et al., ; Qiao et al., ; Requeima et al., ; Ye et al., ; Ye & Chao, ). For both the 4-CONV and ResNet-12 models, we follow the pre-training pipeline used in (Ye et al., ; Ye & Chao, ). During pre-training, we append the feature encoder $z_{\theta}$ with a temporary classification head $f_{\theta}$ and train it to classify all classes in the $\mathcal{D}^{train}$ (e.g., over all 64 classes in the mini-ImageNet) using the cross-entropy loss. During pre-training, the model is trained over $200000$ gradient steps using SGD with Nesterov momentum and weight decay with a batch size of $128$. The learning rate, momentum coefficient, and weight decay penalty are set to $0.01$, $0.9$, and $0.0005$, respectively. Additionally, we use a multistep learning rate schedule, decaying the learning rate by a factor of $0.1$ at the following gradient steps $\{100000,150000,175000,190000\}$. Note that in our ResNet-12 experiments on mini-ImageNet, CIFAR-FS, and FC-100, we observed some meta-level overfitting, which we hypothesize is due to the expressive network architecture coupled with the relatively small datasets. We found that increasing the pre-training weight decay to $0.01$ in these experiments resolved the issue.

#### A.3.2 Meta-Learning Settings

In both the meta-training and meta-testing phases, we adhere to the standard hyperparameter values from the literature (Finn et al., ). In the outer loop, our algorithm is trained over $S^{meta}=30,000$ meta-gradient steps using Adam with a meta learning rate of $\eta=0.00001$. As with previous works, our models are trained with a meta-batch size of 2 for 5-shot classification and 4 for 1-shot classification. In the inner loop, the models are adapted using $S^{base}=5$ base-gradient steps using SGD with Nesterov momentum and weight decay. The learning rate, momentum coefficient, and weight decay penalty are set to $0.01$, $0.9$, and $0.0005$ respectively. During meta-testing, the same inner loop hyperparameter settings are employed, and the final model is evaluated over $600$ tasks sampled from $D^{testing}$. Notably, unlike prior work (Antoniou et al., ; Baik et al., ; ; ), we do not ensemble the top 3 or 5 performing models from the same run, as this significantly increases the number of parameters and expressive power of the final models.

#### A.3.3 NPBML Settings

In our experiments, the backbone encoders $z_{\theta}$ (i.e., 4-CONV and ResNet-12) are modified in two key ways during meta-learning: (1) with linear projection (warp) layers $\bm{\omega}$ for preconditioning gradients, and (2) feature-wise linear modulation layers $FiLM_{\psi}$ for task adaptation. In all our experiments, we modify only the last (i.e., fourth) module with warp and FiLM layers based on the findings from (Raghu et al., ), which showed that features near the start of the network are primarily reused and do not require adaptation. Additionally, we freeze all preceding modules in both the inner and outer loop, which significantly reduces the storage and memory footprint of the proposed method, with no noticeable effect on the performance.

Following the recommendations of Flennerhag et al. (), warp layers are inserted after the main convolutional stack and before the residual connection ends in the ResNet modules. Regarding the FiLM layers, they are inserted after each batch normalization layer in a manner similar to the backbone encoders used in (Requeima et al., ). Where for convolutional layers, we first pool in the spatial dimension to obtain the average activation scores per map, and then flatten in the depth/filter dimension before processing $FiLM_{\psi}$ to obtain $\gamma$ and $\beta$. As discussed in Section 3.4, the loss networks $\mathcal{L}^{S}$, $\mathcal{L}^{Q}$, and $\mathcal{R}$ also use FiLM layers, these are interleaved between each of the linear layers as shown in Figure 4.

## Appendix B Further Discussion

The proposed NPBML framework is a highly flexible and general approach to gradient-based meta-learning. As a result of this generality, many existing meta-learning algorithms are closely related to NPBML, with some potentially arising as special cases of NPBML. In what follows, we discuss some salient examples:

- •
MAML++: In (Antoniou et al., ), the inner optimization of MAML is modified by learning per-step batch normalization running statistics, weights and biases, and layer-wise learning rates. This is closely related to the FiLM layers used in NPBML, as they also output meta-learned scale and shift parameters, i.e., $\gamma_{\bm{\psi}}$ and $\beta_{\bm{\psi}}$, which improve robustness to varying feature distributions. Furthermore, as shown by Equation (16), NPBML implicitly learns a layer-wise learning rate through $P_{\bm{\omega}}$.
- •
SCA: In (Antoniou & Storkey, ), the authors propose replacing $\mathcal{L}^{base}$ with a meta-learned loss function $\mathcal{M}_{\phi}$ similar to NPBML. The primary difference between their parameterization and ours is that their meta-learned loss function is represented as a 1D dilated convolutional neural network with DenseNet-style residual connectivity. In our experiments, we found that a simple feedforward ReLU network is sufficiently expressive to obtain high performance as shown in the ablation study in Table 6.2.2.
- •
MeTAL: In (Baik et al., ), a set of task-adaptive loss functions, represented as small feedforward networks are meta-learned for each step in the inner optimization. Task-adaptivity is achieved by using a separate set of networks to generate FiLM scale and shift parameters which are applied directly to $\bm{\phi}$ instead of the network’s activations. Unlike MeTAL, NPBML does not maintain separate networks for each step; thus, it can be applied to more gradient steps at meta-testing time than it was initially trained to do.
- •
ALFA: In (Baik et al., ), MAML’s inner optimization is modified with meta-learned layer-wise learning rate values and weight decay coefficients. As shown in Equation (16), NPBML implicitly learns layer-wise learning rate values. Additionally, since the learned loss function $\mathcal{M}_{\bm{\phi}}$ in NPBML is conditioned on $\theta_{i,j}$, our method also implicitly learns a scalar weight decay values, since $\smash{\exists\lambda\exists\bm{\phi}:\theta_{i,j}-\nabla_{\theta_{i,j}}(
\mathcal{L}^{base}+\lambda||\theta||^{2})\approx\theta_{i,j}-\nabla_{\theta_{i
,j}}\mathcal{M}_{\bm{\phi}}}$.
- •
MetaSGD: In (Li et al., ), a parameter-wise learning rate is meta-learned in tandem with the parameter initialization. This corresponds to using an identical inner update rule to Equation (6) where $\smash{\mathcal{M}_{(\bm{\phi},\bm{\psi})}=\mathcal{L}^{base}}$ and $\smash{P_{(\bm{\omega},\bm{\psi})}=diag(\bm{\omega})}$, i.e., no meta-learned loss function, and a diagonal matrix used instead of a block diagonal matrix for gradient preconditioning.
- •
WarpGrad: In (Flennerhag et al., ), a full gradient preconditioning matrix is induced by modifying the meta-learned linear projection layers employed in T-Nets (Lee & Choi, ) with non-linear activations. Furthermore, a trajectory-agnostic meta-objective is used. As NPBML utilizes T-Net style gradient preconditioning, simply introducing non-linear activation functions and replacing the outer objective $\mathcal{L}^{meta}$ in Equation (4) would be sufficient to recover WarpGrad’s gradient preconditioning.
- •
ModGrad: In (Simon et al., ), a low-rank gradient preconditioning matrix is meta-learned. This is, in essence, an identical inner update rule to Equation (6) where $\smash{\mathcal{M}_{(\bm{\phi},\bm{\psi})}=\mathcal{L}^{base}}$ and $\smash{P_{(\bm{\omega},\bm{\psi})}=\bm{\omega}_{1}\otimes\bm{\omega}_{2}}$, where $\bm{\omega}_{1}\in\mathbb{R}^{in\times r}$, $\bm{\omega}_{2}\in\mathbb{R}^{r\times out}$, and $\otimes$ is the outer product between the two low-rank matrices with rank $r$.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: See my questions in the weaknesses section.
soundness: 3
strengths: - While low-hanging, the motivation to combine multiple meta-learning approaches is sound. I mention some caveats below, but I can see how enabling different components of the meta-learner to adapt could accelerate convergence and boost final performance.
- In all their experiments, the authors’ method performs the best. The ablations also support their claims.
summary: The authors propose to combine several meta-learning methods into a single one, which they dub “Neural Procedural Biases Meta-Learning” (NPBML). The main idea is to meta-learning the initialization, optimizer, and loss function of a neural network over a distribution of tasks. They show gains on few-shot image classification benchmarks (FC100, CIFAR-FS, mini-/tiered-ImageNet), and perform ablation studies showing each component adds to the overall performance.

While I think the paper is sound, I find it lacks novelty and significance. While combining existing methods shows promising results on (outdated) benchmarks, the results are not compelling enough to justify acceptance. Moreover, the resulting method is actually similar to the MT-nets of Lee et al., 2018; the major difference seems to be learning FiLM layers instead of binary masks. For these reasons I think the paper should be revised and resubmitted.
weaknesses: Conceptual weaknesses:

- While it’s tempting to combine existing meta-learning work, a major caveat is not discussed: the more powerful the meta-learner, the higher the risk of meta-overfitting. In other words, the meta-learner risks to overfit to the train task distribution and fail to adapt to new unseen distributions. I wished the authors mentioned this trade-off — and others that arise from designing stronger meta-learner — explicitly and potentially even addressed it directly.
- None of the components in the proposed combination are novel. So this paper is only an incremental contribution, especially since the empirical results are underwhelming (more below). I would also like to note that the final combination is very close to the 8-year old work of Lee et al., 2018 (MT-nets): MT-nets also learn an optimizer, they also learn an initialization, the loss function is implicitly learned by the optimizer in the last network’s layer, and they also learn a modulating function. The main difference seems to be that here the modulating function uses FiLM layers whereas MT-nets uses binary masks.
- The proposed method is more difficult to implement and computationally more expensive than alternatives. This is a significant weakness, which the authors should also mention. For example, what is the runtime of their methods vs MAML, ProtoNet, or SimpleShot?

Experimental weaknesses:

- The benchmarks used in this work are somewhat outdated and don’t challenge modern meta-learning methods. In fact, I believe this is why the proposed method outperforms all other baselines: none of the benchmarks challenge the meta-generalization ability of the methods — instead, they reward overfitting to the train task distribution.
- Additionally, I think some baselines are missing. For example, ProtoNet or SimpleShot mentioned above. One could even make the argument that these two methods deserve larger backbone architectures, given their inference-time adaptation is much faster than gradient-based algorithms like NPBML.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. How to prevent meta-overfitting within the NPBML framework?
2. How is the number of update steps in the inner loop determined? What would be the effect of using a large number of update steps, such as 50 or 100, in the inner loop?
3. Is this framework still effective for larger convolutional networks or transformer architectures?
4. What would the experimental results be when using this framework on more realistic tasks, such as segmentation or detection?

Please see Weaknesses for details.
soundness: 3
strengths: 1. NPBML combines the gradient-based meta-learning methods into a unified end-to-end framework, which meta-learns the key components of learning, i.e., initializations, optimizers, and loss functions, simultaneously. It enables meta-learning to acquire more optimization components and potentially enhances performance.

2. The framework is flexible and general, with many existing gradient-based meta-learning approaches emerging as special cases within NPBML.
summary: This work proposes to meta-learn task-adaptive procedural biases by simultaneously learning initializations, optimizers, and loss functions that adapt to each specific task. It demonstrates that by meta-learning these components, the framework NPBML can induce strong inductive biases towards a distribution of learning tasks, leading to robust performance across several few-shot learning benchmarks.
weaknesses: 1. There is a risk of meta-overfitting, where the model learns too well from the meta-training tasks and fails to generalize to new, unseen tasks. Although the authors mention this issue in the paper and suggest that it can be alleviated using regularization techniques, this introduces many manual choices, which contradicts the goal of automatically learning to learn from tasks. How to prevent meta-overfitting within the NPBML framework should be carefully discussed.

2. Although the authors state that the gradient-based optimizer is meta-learned, the number of steps to be updated in the inner loop is still a manually set hyperparameter. The article mentions that the early stopping mechanism can be learned implicitly, but in the experimental setup, 5 steps are used instead of a larger number to leverage this early stopping mechanism.

3. The networks used in the experiments are 4-CONV and ResNet-12. It remains questionable whether this framework is still effective on larger convolutional networks or transformer architectures.

4. The tasks used in the experiments are limited to 5-way 1-shot and 5-way 5-shot classification, which is quite different from the tasks that need to be addressed in real-world scenarios, such as segmentation, detection, super-resolution, translation, text summarization, and so on. The effectiveness of this framework in practical task scenarios has not been validated.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: - Can you elaborate how you situate your method in comparison to non bilevel optimization methods? My impression is that such methods [e.g. 1, 2] are both cheaper to run and perform better. Do you disagree with this statement? Where do you see the advantages of your method in comparison to pretraining / sequence modeling based approaches?
- The reported meta learning rate of $\eta= 0.00001$ combined with only 30'000 meta-steps seems to be relatively low and makes me wonder how much meta-learning actually happens. One number that would help to contextualize this would be the performance your model achieves at initialization without any meta-training (i.e. setting $\eta= 0.0$ but keeping everything else equal).
- Are the reported baselines reproductions or are the numbers taken from their respective papers?
soundness: 2
strengths: The paper is clearly presented, motivating the combination of existing components into a new algorithm well.
summary: The paper proposes a bilevel optimization meta-learning algorithm that combines meta-learned initializations, meta-learned preconditioners and meta-learned loss functions with task-specific feature-wise linear modulation models.
It shows that this combination achieves competitive performance on visual few-shot classification task compared to other bilevel optimization meta-learning methods.
weaknesses: While the experimental baselines contain a number of bilevel optimization-based meta-learning algorithms that fall into the same paradigm, comparisons to other popular paradigms such as extended pretraining of the backbone (the presented method also pretrains the backbone) and in-context learning / sequence modelling are missing.
Such methods [e.g. 1, 2] achieve stronger performance on the few-shot learning tasks evaluated here albeit using larger models.
In combination with the large computational complexity of bilevel optimization, the significance of the presented method is therefore unclear to me (see questions).

[1] Hu, Shell Xu, et al. "Pushing the limits of simple pipelines for few-shot learning: External data and fine-tuning make a difference." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Fifty, Christopher, et al. "Context-aware meta-learning." arXiv preprint arXiv:2310.10971 (2023).

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 4
questions: How does the computational cost (runtime, memory) of the proposed method compare to baselines?
Are the results state-of-the-art on the tasks tested?
Can the authors prove a theoretical result about the proposed method?
soundness: 4
strengths: **Originality**

The paper combines together existing meta-learning techniques in a way that I believe has not been seen in prior literature.

**Quality**

Given that there is no theory introduced in this paper, the experimental results are very important. Fortunately, the experiments seem thoroughly conducted with many baselines and the proposed method strongly outperforms baselines on all experiments. Ablations are also included, which is good practice.

**Clarity**

The paper is quite well written and the figures and tables are well-presented. The form of the ablation tables (3, 4) are a good example for other papers to follow.

**Significance**

The paper will likely be of significance to meta-learning practitioners who are trying to achieve state-of-the-art performance on meta-learning tasks.
summary: The authors combine together three existing forms of meta-learning: namely, learning intializations, optimizers and loss functions. Through experiments on miniimagenet, tiered-imagenet, and CIFAR, the authors show that the approach outperforms baselines.
weaknesses: The originality of this work appears limited to me. The authors simply seem to combine existing techniques and find that the performance improves, which is not very surprising. This also limits the significance of the paper to a wider audience. I would encourage the authors to extend the work in at least one direction; some ideas are: 1) including a theoretical result, 2) proposing new performance improvements to the existing methods, 3) demonstrating the emergence of a new phenomenon when multiple meta-learning techniques are combined.

Also, in terms of practical use, it will be very important to compare the computational cost of the proposed method against baselines.

Overall, however, this is a technically solid work that lacks sufficient novely and impact.

Meta-review: This paper combines gradient-based meta-learning methods for few-shot learning, learning initialisations, optimisers and loss functions together, with empirical improvements upon existing methods. 

Reviewers largely agree that it is interesting to combine these methods together, but also agree that the novelty of the paper is low as the methods are the same. I agree with this assessment. Overall, all reviewers agree that this paper is below the acceptance threshold. There are also points raised about the experimental setup (eg by Reviewer hiLK and W9cu: both reviewers did not feel that the author rebuttal sufficiently addressed their points). For what is primarily an empirical paper, addressing these comments in a future version should help.

Decision: Reject