Few-Shot Incremental Learning Using HyperTransformers
Authors: Max Vladymyrov
Andrey Zhmoginov
Mark Sandler
Venue: Submitted to ICLR 2023 (2023)
Keywords: few-shot learning
incremental learning
continual learning
transformers
hypernetworks

Abstract: Incremental few-shot learning methods make it possible to learn without forgetting from multiple few-shot tasks arriving sequentially. In this work we approach this problem using the recently published HyperTransformer (HT): a hypernetwork that generates task-specific CNN weights directly from the support set. We propose to re-use these generated weights as an input to the HT for the next task of the continual-learning sequence. Thus, the HT uses the weights themselves as the representation of the previously learned tasks. This approach is different from most continual learning algorithms that typically rely on using replay buffers, weight regularization or task-dependent architectural changes. Instead, we show that the HT works akin to a recurrent model, relying on the weights from the previous task and a support set from a new task. We demonstrate that a single HT equipped with a prototypical loss is capable of learning and retaining knowledge about past tasks for two continual learning scenarios: incremental-task learning and incremental-class learning.

Under review as a conference paper at ICLR 2023
FEW-SHOT INCREMENTAL LEARNING USING HYPERTRANSFORMERS
Anonymous authors Paper under double-blind review
ABSTRACT
Incremental few-shot learning methods make it possible learn without forget- ting from multiple few-shot tasks arriving sequentially. In this work we approach this problem using the recently published HyperTransformer (HT): hypernet- work that generates task-specific CNN weights directly from the support set. We propose re-use these generated weights an input the HT for the next task of the continual-lcarning as the representation of the previously learned tasks. This approach different from most continual learning algorithms that typically rely using replay buffers, weight regularization or task-dependent architectural changes. Instead, we show that the HT works akin to recurrent model, relying on the weights from the pre- vious task and support set from a new task. We demonstrate that a single HT equipped with a prototypical loss is capable of learning and retaining knowledge about past tasks for two continual learning scenarios: incremental ttask learning and incremental-class learning.
sequence. Thus, the HT uses the weights themselves
INTRODUCTION
Incremental few-shot learning combines the challenges of both few-shot learning and continual learning together: it seeks a way to learn from very limited demonstrations presented continually to the learner. This combination is desirable since it represents more genuine model of how bi- ological systems including humans acquire new knowledge: we often do not need a large amount of information to learn novel concept and after learning about it we retain that knowledge for long time. In addition, achieving this would dramatically simplify learning of important practical applications, such as robots continually adapting stream of demonstrations. Another example is ppeeecypreserving learning, where users run the model sequentially on their private data, sharing only the weights that are continually absorbing the data without ever exposing it.
novel environment layout from incoming
We focus on recently published few-shot learning method called TEPPERTRANSFORMEE. (HT: Zh- moginov et al. 2022). which trains large hypernetwork (Ha et al. 2016) by extracting knowledge from set of training few-shot learning tasks. The HT is then able to directly generate weights of much smaller Convolutional Neural Network (CNN) model focused on solving a particular task from just few examples provided in the support set. works by decoupling the task-domain knowl- edge (represented by transformer; Vaswani et al. (017)) from the learner itself (a CNN), which only needs to know about the specific task that is being solved.
In this paper, we propose INCREMENTAL HYPERTR, IYPERTTAANSERR RASFFORMRR (IHT) aimed at exploring the capability of the HT to update the CNN weights with information about new tasks, while retaining the knowledge about previously learned tasks. In other words, given the weights f--1 generated after sceing some previous tasks {7 (r)+= and task t, the IHT generates the weights O, that are suited for all the tasks {T)f=e-
In order for the IHT to be able to absorb continual stream of tasks, we modified the loss function from a cross-entropp that was used in the HT to a more flexible prototypical loss (Snell et al. 2017). As the tasks come along, we maintain and update set of prototypes in the embedding space, one for each class of any given task. The prototypes are then used to predict the class and task attributes for given input sample.

---

Under review as a conference paper at ICLR 2023
CNN weights (Suited for tasks o...()
Next lask (t
Weight slices 0
Class Embeddings
Sample Embeddings
Transformer / Weight Generator
Shared Weight Embeddings
Incoming Weight Slices
Discarded
Samples from task t
Weight slices f
Generated Weight Slices
Previous task (t- 1)
Figure I: The information flow of the IHT. In the original HT each of the input weight embeddings initialized with empty weight slice. Our proposal is to pass weight slice information from previously learned tasks as an input to the new iteration of the HT.
The IHT works in two different continual learning scenarios: task-incremental learning (predict class attribute using the task information) and class-incremental learning (predict both class and task attributes). Moreover, we show empirically that model trained with the class-incremental learning objective is also suited for the task-incremental learning with performance comparable to the models specifically trained with a fask-ineremental objective.
We demonstrate that models learned by the IHT do not suffer from catastrophic forgetting. More- over, in some smaller models we even see cases of positive backward transfer, where the perfor- mance given task actually improves for subsequently generated weights.
Since the IHT is trained to optimize all the generated weights {0- together, the model can be preempted at any point 7 S T during the inference with weights 6, suited for any task 0 Moreover, we show that the performance of the model improves for all the generated weights when the IHT is trained on more tasks T. We also designed the IHT work recurrent system and its parameters are independent from a given step. Therefore, it can be continued and applied beyond the number of tasks T' was trained for.
т.
2 RELATED WORK
Few-shot learning Many of few-shot learning methods fall into one of two categories: metric- based learning and optimization-based learning. First, metric-based learning methods (Vinyals et al., 2016 Snell al. 2017 Sung al. 2018, Oreshkin al. 2018) train fixed embedding network that works universally for any task. The prediction is then based on the distances between the labeled and query samples in that embedding space. These methods are not specifically tailored for the continual learning problem, since they treat every task independently and have no memory of the past tasks. In contrast to this method, our proposed IHT can be seen as an "'daptiiv"" metric-based learner, where the weights 0, are changing to adapt better to the task t and retain the knowledge of the past tasks.
Second, optimization-base based methods (Finn et al. 2017 Nichol & Schulman 2018 Antoniou al. 2019 Rusu et al. 2019) consisting of variations of a seminal MAML paper propose to learn an initial fixed embedding. that is later adapted to given task using few gradient- based steps. All by themselves these methods are not able to learn continually, since naively adapting for a new task will result in catastrophic forgetting of the previously learned information.
Continual learning. We believe that compared to the related work (see Biesialska et al. 2020 for an overview), our approach requires the least conceptual overhead, since it does not add additional constraints to the method beyond the weights generated from the previous task. In particular, we do not inject replay data from past tasks (Lopez-Paz & Ranzato 2017 Riemer et al.] 2018 Rol- nick et al. 2019; Wang et al., 2021a). do not explicitly regularize the weights (Kirkpatrick et al.] Zenke et al.) (017) and do not introduce complex graph structures (Tao et al.) 2020 Zhang
2017
et al.
2021), data routing or any other architectural changes to the inference model (Rusu et al.)
2016
Instead, we reuse the same principle that made HYPERTR TANERRRASSOOREE work in the first
2

---

Under review as a conference paper at ICLR 2023
Task t
Jr J--1 + Σ Lv (loo)izon for (8(T)) ),
Task t ++1
{sp)p=
T=0
{cp
( 3==
Ct
CNN
HyperTransformer
---1
Oc as (
0c-1)
Ur.
{eat)) g(m)
Qm)) JT r=0
S(O)
Figure 2: The diagram of few-shot continual learning with INCREMENTAL HYPERT RANSFFORRR. For each new task the IHT uses the support set ((t) and previously generated weights 0:-1 to generate new weights O,. The support set is then passed through those weights to compute the prototypes et and update the set of prototypes for all the tasks {cp)p=o Finally, we pass the query set of every task {Q(T))+
to we evaluate the quality of the generated weights.
place: decouple a small CNN classification model re-generated for each incoming task from a large Transformer model that generates CNN weights, in effect learning how adapt them to new in- coming task in a way that avoids forgetting prior knowledge. In this regard, the closest analogy our approach would be slow and fast weights (Munkhdalai & Yu, 2017). with the IHT weights being analogous to the slow weights that accumulate the knowledge and generate CNN weights as fast weights.
Incremental few-shot learning. Prior work on incremental few-shot learning frequently uses a similar fast/slow weight separation, but the fast weights are typically generated using some form of gradient-basee optimization (Ren et al. 2019 Lee et al.) 2021). Furthermore, many incremental few-shot learning techniques rely on frozen feature extractors (Ren et al. 2019 Perez-Rua al., 2020 Wang et al., 2021b; Zhang et al., 2021), which could potentially hinder their performance in situations where the input domain changes considerably between tasks. Some of these tech- niques (Wang et al. 202 are iieettion-base and can be viewed as special cases of our approach, where most the model layers are trained or pretrained instead of being generated.
Many methods exist that try to specifically address the catastrophic forgetting by regularizing either the weights directly by restricting the update of some of them (Mazumder et al. |021 Chen & Leel 2020) or by constrain the optimization procedure itself (Shi et al. 2011: Gupta et al. 2020).
3 HYPERTRANSFORMER ARCHITECTURE
We consider few-shot learning scenario, where we are given a set of tasks {tlt E T} split into
Tirain and Trest. Each task t is specified via a K-way N-shot support set S(1) (a (t) (t) ()) and a query set (t)) = (a!")
y Ji=
(t) 0. N where K is the number of classes in the episode, N is the number
of labeled demonstrations of each class, and N (typically N > N) is the number of query samples to be classified.
At each training iteration, new episode is sampled without replacement from Terain The support set from that episode is used by the HT (parameterized by vi) to directly generate the weights of the CNN via O, an (S()). The loss is computed by passing the query (&, 9) from the same task through the generated CNN fe, (f) and using a cross-entropy objective. After the weights & of the HT are trained, the tasks are sampled from a separate set of Trest to evaluate its performance.
At the core of the HT lies an attention mechanism that generates CNN weights for a given layer from the support set. The transformer receives the embedding of the support set and placcholders for weight embeddings initialized with zero weight slices (see Fig. [T. At the output the support set embedding ignored and the generated weight slices are converted into weights of the CNN.
3

---

Under review as a conference paper at ICLR 2023
4 INCREMENTAL HYPERTRANSFORMAE
The HT can naturally extended the HT architecture to a continual stream of tasks by passing the generated weights from already learned tasks as input weight embeddings into the weight generator for a new task (see Fig. I This way the learned weights themselves act as both the input and the output of the the IHT. performing dual function: storing information about the previous tasks as well as serving as CNN weights for evaluation on already seen tasks.
Fig. 2 demonstrates the overall learning flow of the proposed algorithm. During the training. T episodes are sampled sequentially at random without replacement. For each task t, the IHT receives the support set of that task 51()) as well as the previous weights O, 0 and generates the weights
0₂ = ae. (S(1) 00-1),
(1)
that are suited for all the tasks {T) Therefore, for each step t we want to minimize the loss on the query sets of every task up to
J( = Σ Lv (
(2)
==0
The final loss function is given by the sum of the losses for all the tasks:
arg min
Σ J(b)..
(3)
t=0
The IHT sequentially generates a series of weights {--} each of which are suited for all the tasks coming before it: do performs well only on the task To. O1 performs well on the tasks To and Ti, and so forth. This design choice allows us to have "preemptive" continuall learning setup, where the IHT trained for T tasks can be stopped at any point at < T and still have well-performing weights 0, for all the tasks seen so far. An alternative would be setup where a user knows the exact number of tasks in the sequence and only cares about the performance after the final task T. This would correspond to minimizing only the last term Jr in (3). We have experimented with this version and did not observe noticeable improvement over the
propose above.
Another desirable property of the proposed IHT architecture is its recurrence. The HT parameters do not depend on the task information and simply take the 9 and the support set its input. Therefore, it is possible not only to preempt IHT by stopping early, but also extend the trained model to continue generating weights for more tasks that it has been trained on. We will demonstrate this ability in the experimental section.
4.1 PROTOTYPICAL LOSS
The last element of the algorithm that we have left to discuss is the exact form of loss function ()- in ( The original HT used the cross-entropy loss, which is not well suited for continual learning, since the number of labels that it predicts is tied to the number of parameters in the head layer of the A. As the number of tasks increases, the architecture of CNN needs to be adjusted, which goes against our design principle of recurrent IHT architecture. Another alternative would be fix the head layer to the K K-way classification problem across all the tasks and predict only the class information, ignoring the task attributes (which is known as comain-ineremental learning Hsu al. 2018). However, in this case the classes with the same label attribute from different tasks would be minimized to the same spot in the embedding space, creating a collision. Moreover, since these label attributes do not carry any semantic meaning and are drawn at random for every episode, the collisions would happen at random and the IHT would not be able to adjust or learn from them. In the Appendix A. we show that the accuracy of this setup drops dramatically as the number of tasks increases and this method becomes impractical right away for as little as two tasks.
To make the method usable we need to decouple the class predictions of every task while keeping the overall dimensionality of the embedding space fixed. One way to solve this problem is to come up with fixed arrangement of TK points, but any kind of such arrangement is sub-optimal since it impossible to place T K points equidistant from each other in a fixed dimensional space for large enough T. A much more elegant solution would be learn the location of these class prototypes
The initial weights 0g are generated using empty weight slices is done in the original HT.
4

---

Under review as a conference paper at ICLR 2023
from the support set itself, e.g. with prototypical loss (Snell al. 2017). The prototypes computed by averaging the embeddings of support samples from a given class k and task T:
1
Σ So, (x)] (2.8)
G+ -
We use the prototypes in two different continual learning scenarios. First, for the task- incremental learning, we are assumed to have access to the task we are solving and need to predict class k given the task is
only the class information. The probability of the sample belonging then equal to the softmax of the 12 distance between the sample and the prototype normalized over the distances to the prototypes from all the classes from T:
exp( - 11, ($) - c2
p = k((,T) Σx exp(-I1fo,
(5)
c+ 13)
Second, for the more general class-incremental learning, we need to predict both task and class attributes. The probability of the sample belonging to class k of task is equal to the softmax of the L2 distance between the sample and the prototype normalized over the distances to the prototypes from all the classes and all the tasks:
exp( 1op(-115a, - 12
p k, T(Ø) EP' exp(- 116, (5) C.' 1 12)
The final loss function is given by minimizing the negative log probability of the chosen softmax over the query set. Algorithm describes the procedure to compute that loss value.
Empirically, we noticed that the IHT models trained with the class-incremental learning objective 6 are applied equally well to both maass-nncremenal and meskiinrrementl settings, while models trained with the task-incremental objective (5) work well only in the task-incremental setting and rarely better than the models trained using 66). In what follows we are going to focus only on the IHT trained with (6). but evaluate them for both task- and class-incrementl learning scenarios.
Notice that the prototypes are computed with respect to the current weights 0, in (4) for the task T, however they need to be used down the line to compare the embeddings produced by subsequent weights Og in (6). Ideally, once the new weights Og are generated, the prototypes need to be recom- puted as well. However, in the true spirit of continual learning, are supposed re-use the support samples after the task has been already processed. We found out that freezing the proto- types after they are computed provides a viable solution this problem. We noticed that compared to recomputing the prototypes every step, this gives only a marginal difference in performance.
We do want to note an important use-case where recomputing the prototypes might still be possible and even desirable. Notice that the weights 6, are not affected by this issue and are computed in continual learning manner from (T) without the use of the information from the previous task. The support set is needed only to update the prototypes through generated weights, which is relatively scenario which the weights are
cheap operation. Thus, we can envision a privacy-preserving updated and passed from client client in a continual learning way and the prototypes needed to "unlock" those weights belong to the clients that hold the actual data.
CONNECTION BETWEEN PROTOTYPICAL LOSS AND MAML
While the core idea behind the prototypical loss is very natural, this approach can also be viewed as a special case of a simple 1-step MAML-like learning algorithm. This can be demonstrated by considering simple classification model q(<;;)) s (W) fo(x) + b) with d (W, b, 0), where fo(x) is the embedding and s(-) is a softmax function. MAML algorithm identifies such initial weights 6 that any task 7 with just few gradient descent steps initialized at d brings the model towards a task-specific local optimum of L.
Notice that if any label assignment in the training tasks is equally likely, it is natural for q(:; dd) to not prefer any particular label over the others. Guided by this, let us choose w and b" that are label-inde ihbee--ndependent. Substituting & = σ0 86 into (xx: Φ), we then obtain
(<(x;)) Φ) =c(x; + (6()) (SWafeo + 8be + w ØS 00
(x; 80 )0) O
5

---

Under review as a conference paper at ICLR 2023
Algorithm Incremental-class learning using JIYPERTRANSFORMER with Prototypical Loss.
Input: 7 rrandomly sampled K-way N-shot episodes: Q(0)
Output: The loss value J for the generated set of tasks.
1: J -+-
P Initialize the loss. P Initialize the weights.
2: 8-1+-0
3: 0 do
4:
01 s (5(t), (1-1)
Generate weight for current task. > Compute prototypes for every class of the current task.
5: for + 4-010 to do
6:
Ctk - N (z.))es))) Sa, 1
7:
end for
8:
for + to do
> Update the loss with query samples using (6)
9:
for + 0) to do
10:
J J Σ (2 logp(ū = -
11: end for
12: end for
13:
end for
T = 2 tasks
T = 3 tasks
T 4 tasks
T = 5 tasks
OMNIGLOT
0.88
0.87
0.86
0.84
Accuracy
3
4
0
2
3
TIEREDIMAGENET
0.74
0.72
0.70
0.68
2
3
2
3
Task Name
0, 0
03
6, ConstPN
Figure 3: mass-ineremental learning on OMNIGLOT and TIEREDIMAGENET. Each column repre- sents different IHT trained with T 3, 4 or 5 tasks in total. The tasks with a bullet marker (+) correspond to the terms in the objective function 33) that are being minimized. The lines with the diamond marker (o) are the extrapolation of the trained IHT to larger number of tasks.
where l the label index and dó = (8W, 8b, 50). The lowest-order label-dependent correction to 3..). In other words, in the lowest-order, the model only adjusts the final logits layer to adapt the pretrained embedding foo (x) to new task.
(a;;; Φd) is given simply by (-WWe 2. (o) +
For simple softmax cross-entropy loss (between predictions (zz) and the groundtruth labels y). single step of the gradient descent results the following logits weight and bias updates:
sW,-
Σ (x.8)ES
1y=
(z) [
dbg
2
Σ (x.y))
1
[
1
where the 1/|C| term results from normalization in the softmax operation. Here 7 is the learning rate, n is the total number of support-set samples, |C] is the number of classes and S is the support set. In other words, see that the label assignment imposed by 8W and 8b from (7) effectively relies on computing dot-product of for (x) with "prototypes" Ck = : N--NN D ((,,)) foo (z)1y=k-
EXPERIMENTS
We evaluated the model two standard benchmark problems: 1-shot, 20-way OMNIGLOT and 5-shot, 5-way TiEreDMAGgEEe The generated weights for each task 6, are composed of four convolutional blocks and single dense layer. Each of the convolutional blocks consist of 3

---

Under review as a conference paper at ICLR 2023
T' tasks
T = tasks
T= 4 tasks
T 5 tasks
OMNIGLOT, 8 channels
0.90
0.85
0.80
0.75
0.70
0.65
Accuracy
0-2
0-1 TIEREDMMAGENET, 64 channels
0-2
0-1
0-2
0-1
0-2
0.75
0.70
0.65
0.60
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
Task Name
0
01
0₂
0g
04
ConstPN
MergedHT
Figure 4: classinccrementl learning OMNIGLOT and TIEREDIMAGENET Each column repre- sents different IHT trained with T = 2, 3, 4 or 5 tasks in total. The tasks with a bullet marker (+) correspond to the terms in the objective function (3) that being minimized. The lines with the diamond marker (o) are the extrapolation of the trained IHT to a larger number of tasks.
convolutional layer, batch norm layer, ReLU activation and 2 max-pooling layer. For OM- NIGLOT we used 8 filters for convolutional layers and 20-dim FC layer to demonstrate how the network works on small problems. and for TIEREDIMAGENET we used 64 filters for convolutional and 40-dim for the FC layer to show that the method works for large problems as well. The re- ported accuracy computed for 1024 random episodic evaluations from Teest with each episode run 16 times with different combinations of input samples.
For the HT architecture, we tried to replicate the same setup that the authors had for their original paper as closely as possible. We used 4-layer convolutional network as feature extractor and a 2-layer convolutional model for computing activation features. For OMNIGLOT we used 3-layer 2-head transformer and for TIEREDIMAGENET, all our experiments we have trained the network on a single GPU for 4M steps with SGD with exponential LR decay over 000 steps with a decay rate of 0.97. We noticed some stability issues with training as we increase the number of tasks and had to decrease the learning rate to adjust for it: for OMNIGLOT experiments we used a learning rate 10-4 for learning up to 4 tasks and x 10-5 for 5 tasks. For TIEREDIMAGENT any T. As outline above, train the IHT models with the class-incremental objective (6), but evaluate for both task-incremental and class-incremental scenarios.
simplified 1-layer transformer with 8 heads. In
we have used the same learning rate 5 x 10-6 for training with
We compare our performance to two baselines. The first is CONSTANT PROTONET (CONSTPN), which represents vanilla Prototypical Network, as described in Snell et al. (017). There a univer- sal fixed CNN network is trained on episodes from Train This constant network can be applied to every task separately by projecting the support set as prototypes for that task and computing the pre- diction with respect to these prototypes. Strictly speaking, this is not a continual learning method, since it treats every task independently and has no memory of the previous tasks. For best results on this baseline we had to increase the number of classes times during the training (e.g. for 20-way OMNIGLOT evaluation we have trained it with 100-way problems).
The second baseline we used specifically for the class-incremental learning is a MERGED HYPER- TRANSFORMER (MERGEDHT), where we combine all the tasks together and train single original HT instance as a single task. This method is not a continual learning, since it has the information about all the tasks from the beginning, but it produces solution for every class and task that we can still compare to the weights generated by the IHT.
In fig. and 4 we present the main findings of our paper. In order to understand the effect of continual learning with multiple tasks, each column represents a separate run of the IHT trained on T 2, 3,
Note that, in contrast with cross-entropy, we do not need to have the head layer dimension be equal to the number of predicted labels when using the Prototypical Loss.
7

---

Under review as a conference paper at ICLR 2023
0o
02
03
O₂
CONSTPN
d
Figure 5: UMAP projection of the IHT prototypes and the query set embeddings. The query set points are connected with the corresponding prototypes. The top plot shows the embeddings of different weights from incremental IHT training, colored according to the task. Right plot: UMAP projection of the CONSTPN embedding for 25 different classes from IMAREDT
or 5 tasks in total (for training higher T, please see results in the Appendix). To demonstrate the recurrence of the method, we have extended the number of tasks to 5 for the evaluation. Therefore each plot shows 5 curves that correspond to the IHT, split into two groups: bullet marker (+) for tasks that the model was trained for and diamond marker (o) for extrapolation to more tasks. Each trained model is applied to both task-incremental (fig. 3 and class-incremental (fig. setting.
For the plass-incrementll learning, the task name is given by two numbers indicating the range of tasks we used for evaluation (c.g. task name 0-3 corresponds to four tasks from 0 to 3).
The black constant dashed line is a baseline performance of the CONSTPN, which uses a fixed embedding and does not differentiate between the tasks. Starred blue markers represent separate run of the HT for particular configuration of merged tasks.
Task-incremental learning. We start by analysing the task-incremental learning results. For the OMNIGLOT dataset, we see no signs of catastrophic forgetting for the IHT, moreover, we observe a positive backward knowledge transfer. For given IHT, the performance on the past tasks improves as more weights are being generated: e.g. for most cases the performance of 0₁ (green markers) is higher than Og (orange markers), 02 is higher than both 0 and 6g and so forth. We also noticed that as we train the IHT with more tasks, the overall performance increases: the IHT trained for T tasks has better results than the one trained with T = 2.
For the WEREDIMAGENTE the results are much better than the CONSTPN baseline, but the positive backward knowledge effect is not as pronounced as with the OMNIGLOT dataset. The performance for every training task stays roughly the same for all the generated weights. This is still a useful outcome, since it means that the model does not suffer from catastrophic forgetting.
Finally, for every trained IHT, the performance is always better than the CONSTPN baseline up to the number of tasks T that it was trained for. Extrapolating the IHT to more tasks decreases the accuracy, but not significantly. Already for T = 3 tasks, generating 4 or 5 weights performs better than the CONSTPN.
Class-incremental learning. Switching to the class- inassinncremetall learning, the accuracy trends down across all the models as more and more tasks are included in the prediction. This is ex- pected, because the size of the generated CNN does not change, but the number of overall classes that needs to be predicted is increasing. For OMNIGLOT transfer taking place: the IHT models trained with a higher T perform better across the board. For a given model trained for fixed T the performance is comparable. This shows the preemptive prop- erty of the IHT: models trained for T tasks can be still run for any number of tasks smaller than T with a similar performance.
still observe the positive backwards
Comparing results to the baselines, the IHT has better results than the CONSTPN up to the number of tasks 7' it was trained for, with extrapolation results improving as T grows. Very surprising, for T = 5 the IHT was able to outperform even the MERGEDHT baseline for OMNIGI.T,, which has all the information about the tasks from the beginning. We hypothesise that having more classes to classify makes the learning problem very hard for the original HT since the image embeddings are not able to learn good embeddings. This is specifically apparent for the TIEREDIMAGENET where the numbers for the MERGEDHT are so low, they fall below the graph at 60% even for the 0-1 task.
Multi- Multi-Domaie Domain Episode Generators. In all of the experiments above, the support and query sets in different episodes were always sampled from the same general distribution of tasks. And even though the set of chosen image classes varied task-to-task, the overall image domain and typical
8

---

Under review as a conference paper at ICLR 2023
image features remained consistent across all the tasks. If different tasks could be sampled from entirely different distributions and different image domains, we would expect task-agnostic CON- STPN approach to experience accuracy degradation because it would need to identify a universal representation suited for all of the image domains at once. The IHT approach, on the other hand, could in principle adapt sample representations differently for different detected image domains.
We verify this by creating a multi-domain episode generator including tasks from a variety of image datasets: OMNIGLOT, CALTECU101, CALTECHBIRDDS2011 CARS 196, EXFORDFLOWBBE 102 and STANFORDDOGS. We measured the accuracy of CONSTPN vs. the IHT on this multi-domain generator using episodes contained two tasks with 1-shot, 5-way problems. The generated CNN model used 16 channels with the 32 channels for the final layer. Other parameters matched those used in the TIEREDIMAGENE! experiments. Measured the CONSTPN accuracies were 53% for task O, 52.8% for task 1 and 50.8% for combined tasks. For IHT accuracies were 56.2% for task O, 55.2% for task 1 and 53.8% for combined tasks. The accuracy gap of nearly 3% between these two methods (higher than that for OMNIGLOT and TIEREDIMAGENET) suggests that the IHT is better at adapting to multi-domain task distribution, but additional experiments are needed to confirm that adding new image domains hurts the CONSTPN performance more than it does the IHT.
Analysis of prototypical loss. We want to take closer look prototypes and query embeddings for generated weights. For a given IHT trained for T 5 on TIEREDIMAGENET 5 5-way, 5-shot problem, we picked a random test of tasks and ran them through the model, pro- ducing weights &o to 04 along with prototypes for each class for every task. We then com-
0.80
0.87
Accuracy
0.9
puted the logits of the query sets consisting of 20 samples per class for every task. The re-
0.8
sulting 40-dim embeddings for prototypes and query sets are concatenated and projected to 2D using UMAP (fig. 5) Notice that the embed- dings of the tasks are well separated in the logits space, which explains why the model performs well for both task- and class-incremental learn- ing. Normalizing the softmax over the classes from the same tasks or across the classes for all the tasks makes little difference when the tasks are so far away from each other. On the right of fig. we show the projection of the CONSTPN em- bedding of the same 25 classes. The CONSTPN does not make a distinction between tasks and treats every class separately. The fact that we see 3 clusters emerge has to do purely with the semantics of the chosen clusters and the way the CONSTPN decides to group them. This also sheds light on why the IHT performs better than the CONSTPN, since it separates the tasks first before separating the classes within each task.
0.7
0.6
D-2
0-3
Task name
min eq.
55)
On -...
B 6
6.
A
8.
min eq. 60
0g
0
62
0
6.
Figure 6: IHT trained using task iikkineeeeeetal ob-
jective 5 vs. class- caassinnrremenal objective (6).
Finally, in fig. we show the accuracy of two different models trained with task inceenneeeenttll and class-incred class-inermmentl objectives (6). The performance of both models on task-ineremental problems are comparable, while for dasseinncrmental the model trained for that objective is better.
7 CONCLUSIONS
The presented INCREMENTAL HYPERTRANSFORNEE model has several desirable properties. On one hand, as an efficient few-shot learner, it is able to generate CNN weights on the fly with no training required from a small set of labeled examples. On the other hand, as a continual learner, it able to update the weights with information from a new task by piping them recursively through a new iteration of the HT. We showed empirically that the learning happens without the catastrophic forgetting and, in some cases, even with a positive backward transfer. By modifying the loss from cross-entropy to prototypes of the prototypes of every class for each task. A single trained IHT model works in both task- and class-incremental regimes.
were able to define learning procedure that optimizes the location

---

Under review as a conference paper at ICLR 2023
REFERENCES
Antreas Antoniou, Harrison Edwards, and Amos J. Storkey. How to train your MAML.. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Magdalena Biesialska, Katarzyna Biesialska, and Marta R Costa-Jussa. Continual lifelong learning in natural language processing: A survey, arXiv preprint arXiv:2012.09823 2020.
Kuilin Chen and Chi-Guhn Lee. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. Doina Precup and Yee Whye Tch (eds.), Proceedings of the 34th International Conference on Machine Learning. volume 70 of Proceedings of Machine Learning Research, pp- 1135-1135. PMLR, 06-11 Aug 2017.
Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems, 33:1 102111588111 11598, 2020.
David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. arXiv preprint arXiv: (60::::69999910 09106, 2016.
Yen-Chang Hsu, Yen-Cheng Liu, Anita Ramasamy, and Zsolt Kira. Re-evaluating continual learning scenarios: categorization and case for strong baselines. arXiv preprint arXiv: /2488, 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Bard Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-35 -3526, 2017.
Eugene Lee, Cheng-Hat Han Huang, and Chen- Yi Lee. Few-shot and continual learning with attentive independent mechanisms. In Proceedings of the IEEE/CVE International Conference on Com- puter Vision, pp. 1455-9464 2021.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. arXiv preprint arXiv:2103.00991 2021.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In International Conference on Machine Learning, pp. 2554-2563. PMLR, 2017.
Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:/803.0200 2(3):4, 2018.
Boris N. Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. TADAM: task dependent adaptive metric for improved few-shot learning. Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Informa- tion Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 719-729, 2018.
Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy M Hospedales, and Tao Xiang. Incremental few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13846-18 13855, 2020.
Mengye Ren, Renjie Liao, Ethan Fetaya, and Richard Zemel. Incremental few-shot learning with attention attractor networks. Advances in Neural Information Processing Systems, 32, 2019.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arXiv:/810. 1910, 2018.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
10

---

Under review as a conference paper at ICLR 2023
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.0466 04671, 2016.
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osin- dero, and Raia Hadsell. Meta-learning with latent embedding optimization. In 7th International Conference on Learning Representations. ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting incremental few-shot learning by finding flat minima. Advances in Neural Information Processing Systems, 34:6747-6766 2021.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Ad- vances in neural information processing systems, 30, 2017.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1199-1208. IEEE Computer Society, 2018. doi: 10.1109/CVPR.2018.0000
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few- shot class iaaseinncremntall learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12183-12192 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998-6008, 2017.
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference Neural Information Processing Systems 2016, December 5-10, 2016. Barcelona, Spain, pp. 3630-3638, 2016.
Liyuan Wang, Qian Li, Yi Zhong, and Jun Zhu. Few-shot continual learning: brain-inspired approach. arXiv preprint arXiv:2104.0900 2021a.
Yu Wang, Nicholas Bryan, Mark Cartwright, Juan Pablo Bello, and Justin Salamon. Few-shot continual learning for audio classification. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 321-325. IEEE, 2021b.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 12455-12466 2021.
Andrey Zhmoginov, Mark Sandler, and Max Vladymyrov. Hypertransformer: Model generation for supervised and semi semi-supervised seo-shot learning, arXiv preprint arXh::2201.0416 04182, 2022.
11

---

Under review as a conference paper at ICLR 2023
0.8
Accuracy
0.6
0.4
0.2
0
3
Task name
6 0 02
04
ConstPN
Figure 7: The accuracy of the HT trained for 7 5 using cross-entropy loss. While the accuracy of the first weight 0n high and is better than the accuracy of CONSTPN embedding, drops dramatically when more tasks are added, because of collisions between the same classes for different tasks in the cross-entropy loss.
81
0
03
0,
Figure 8: 2-dim UMAP projection from 20-dim embedding the support set (prototypes) and query for different weights from incremental HT training. The query points connected with their corresponding prototypes with a line. The top and bottom plots are identical, but colored according to both their class and task (top plot), or just according their task (bottom plot). Notice that the prototypes from the earlier tasks remain the same for the later task. however since UMAP projection non-parae non-parametric and have prototypes is not the same. We tried our best to align the embedding using the Procrustes alignment.
re-run for every new 0r. the 2-dim projection of the
A ADDITIONAL FIGURES
A.1 LEARNING WITH CROSS-ENTROPE LOSS
Figure shows attempt to do learn incrementally from multiple tasks using HT with cross- entropy loss. Since we do not increase the size of the last layer's embedding, we can predict only class and not task, which corresponds to the class from different tasks are mapped to the same location in the embedding space, thus creating collisions when more tasks added. This is why the accuracy drops very dramatically when the number of tasks increase. On the other hand, CONSTPN is more flexible, since the prototypes for each tasks are computed from the support set of given task and do not have be fixed to the one-hot vector, as it done for the cross-entropy.
learning setup. Moreover, the same
A.2 UMAP PLOT OF THE MULTI-TASK PROTOTYPES
UMAP embedding for the OMNIGLOT embedding using ProtoNet (fig. X looks different from similar embedding projection of TIFREDIMAGENET. In particular, it looks like the embeddings
12

---

Under review as a conference paper at ICLR 2023
OMNIGLOT, channel
Number of tasks
0.8
Number of tasks 15
Number of tasks 20
Accuracy
0.7
0.5
0-1
0-2
0-3
044
0-5
0-6
D-1
0.8
0-9
0-10
0-11
0-12
0-13
0-14
0-15
0-10
0-17
0-18
0-10
Task name
OMNIGLOT, channel
0.95
Number of tasks
Number tasks 9
0.90
Number tasks 10
Accuracy
Number tasks 11
0.85
0.80
0-1
0-2
0.3
00
0
0-6
0-7
0.6
6-0
0-10
Task name
Figure 9: OMNIGLOT with 32 channels trained with different number of tasks T.
from different tasks are overlapping, in contrast the TIEREDIMAGENES embedding that are sepa- rated. We think this because classes in OMNIGLOT are much more closely connected than TIEREDIMAGNNET. Curiously, even though the classes between the tasks overlapping, the final accuracy is still the high 90% and barely degrades when more tasks are added.
A.3 LEARNING WITH MORE TASKS
Although, we mainly analysed the results of IHT for up tasks, we can run the IHT for much more than that. Fig. showws the IHT run for different number of total tasks T for the OMNIGLOT dataset with 8 and 32 channels convolutional layers. Similar to the results in the main paper, the nearly overlapping curves mean that the model trained for T tasks can generalize and run for up to that many tasks with the same accuracy as the model trained for smaller number of tasks.
A.4 INCREMENTAL HYPERTRANSFORMER VS MERGEDHT FOR TIEREDIMAGHNET
Fig. T0 shows the zoomed out view of the same figure presented in the main paper (fig. Here, one can clearly see how much worse the MERGEDHT is compared to the IHT.
A.5 ADDITIONAL FIGURES FOR OMNIGLOT TASK- AND CLASS-INCREMENTA LEARNING
Fig. 12
K3
and show additional experiments with OMNIGLOT with different number of
channels in the CNN.
13

---

Under review as a conference paper at ICLR 2023
T tasks
T= 3 tasks
4
T 5 tasks
MIEEDIMAGENET, 64 channels
0.7
Accuracy
0.6
0.5
0.4
0.3
0-1
0-2
0-3
0-1
0-2
0-3
0-4
0-3
0-4
0
0-1
0-2
044
Task Name
0
01
2
O4
ConstPN
MergedHT
Figure 10: Zoomed out view of fig. so that the results of the MERGEDHT is visible.
7 tasks
= 3 tasks
T= 4 tasks
= 5 tasks
iask-inccrmmental learning
0.675
0.675
0.675
0.675
0.650
0.650
0.650
0.650
0.625
0.625
0.625
Accuracy
3
3
0
Class-incremental learning
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0-1
0-4
0-1
0-2
0-3
0-4
0.2
0.3
04
0.2
0.3
Task Name
0
01
0₂
0:
Figure 11: Task- and inasemineeemnttlll learning on OMNIGLOT with 4-channels convolutions.
T tasks
tasks Task-incremental learning
T tasks
5 tasks
0.80
0.80
0.80
0.80
0.78
0.78
0.78
0.78
Accuracy
3
Class-incremental learning 0.8
0.8
0.8
0.8
0.6
0.6
0.4
0.4
0.4
0.4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0.3
0-4
Task Name
0,
0
O2
0:
04
Figure 12: Task- and class-incremental learning on OMNIGLOT with 6-channels convolutions.
14

---

Under review as a conference paper at ICLR 2023
T tasks
tasks
7 tasks
5 tasks
laskeincremental learning
0.95
0.95
0.95
0.94
0.94
0.94
0.94
0.93
0.93
0.93
0.93
Accuracy
Class-incremental learning
0.95
0.95
0.95
0.95
0.90
0.90
0.90
0.90
0.85
0.85
0.85
0.85
0.80
0.80
0.80
0.80
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0.2
0-3
0-4
Task Name
0o
01
O2
03
04
Figure 13: Task- and class-incremental learning on OMNIGLOT with 16-channels convolutions.
tasks
= tasks
T = 4 tasks
= 5 tasks
Task-incrementee learning
0.960
0.960
0.960
0.960
0.955
0.955
0.955
0.955
0.950
0.950
0.950
0.950
Accuracy
3
3
Class-incremente learning
0.95
0.95
0.95
0.95
0.90
0.90
0.90
0.90
0.85
0.85
0.85
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
Task Name
0,
0
O2
0
0
Figure 14: Task- and class-incremental learning on OMNIGLOT with 32-channels convolutions.
15


Reviews:
[{"review_id": "ZdHV_ol32s", "reviewer": "Reviewer_UZsQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper makes a few contributions: using a newly published weight generation model HyperTransformer to generate model weights for incremental task/class learning; modifying the classic SCE loss with an existing prototype loss (from a few-shot learning paper). ", "summary_of_the_review": "It is a good baseline paper based on existing techniques (or losses), but does not contribute a novel idea or solution to class-incremental learning or task-incremental learning explicitly. The paper is clearly written and easy to follow.", "strength_and_weaknesses": "Strength:\n\nIt is a good baseline method for using the weight generation model in class-incremental and task-incremental learning. \n\nWeakness:\n\nIts applications of the existing weight generation model and the typical prototypical loss are straightforward. Its overall idea is too similar to many related few-shot learning works (which however not being mentioned in the related work section). Please check some of them in [1,2,3]. It does not explicitly solve the forgetting problems in incremental learning, but simply relies on the performance of the adopted hypernetwork. I can take this work as a good baseline but without clear novelty over the baseline.\n\n[1] Li et al. CVPR 2022. Sylph: A Hypernetwork Framework for Incremental Few-shot Object Detection. It introduced a few-shot hyper network to generate weights and biases for each few-shot detection task.\n\n[2] Przewięźlikowski et al. arXiv 2022. HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks. It introduced HyperMAML — an approach to the Few-Shot learning problem by aggregating information from the support set and directly producing weights updates.  \n\n[3] Sendera et al. ArXiv 2022. HyperShot: Few-Shot Learning by Kernel HyperNetworks. It fused kernels and hyper network paradigm. Its hypernetwork takes the support set as input to predict the classifier weights to do few-shot task adaption.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Z9p3QhNsyb", "reviewer": "Reviewer_XnH3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In the paper, the authors use HyperTransformer (HT) in a new setting that mixes continual learning and few-shot learning. The authors propose to reuse these generated weights as input to the HT for the next task.", "summary_of_the_review": "The paper is interesting but the presentation and evaluation of the model are not enough for publication.", "strength_and_weaknesses": "1. The paper is not well written. For me, it is not trivial to extract information about the setting. What authors mean by continual learning in few-shot learning. \n\nFEW-SHOT INCREMENTAL LEARNING is well described in literature and authors should concentrate on such a problem, not few-shot or continual learning.\n\n2. The paper should contain a section that explains what authors understand by continual learning mixed with few-shot learning. How exactly looks FEW-SHOT INCREMENTAL LEARNING in the paper?\n\n3. Chapter 3 HYPERTRANSFORMER ARCHITECTURE starts with the formulation of few-shot learning. But section 4 INCREMENTAL HYPERTRANSFORMER directly starts with a model, without a specification of how the few-shot learning task was generalized. \n\n4. Experimental section is not well done. First of all, it is not clear what the training looks like and how the evaluation is computed. \n\n5. Authors do not specify numerical measures for evaluations\n\n6. Authors evaluate the method only by plots. It is difficult for future investigators to work whit such papers. The charts are nice the authors should give exact numbers in the appendix. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "LK_83mIDJI", "reviewer": "Reviewer_qvBd", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a few-shot continual learning approach that uses a hypertransformer to infer the task-specific CNN weights. These weights are then used as inputs to the hypertransformer thus serving as a state for it. The whole architecture resemble a recurrent neural network but with added feature of assimilating few-shot learning. I liked the idea of extending hypertransformers to continual learning.", "summary_of_the_review": "The paper proposes a new way to assimilate continual learning in hypertransformers architecture. I liked the approach, but have few concerns regarding the evaluations. I have pointed them out in the Weaknesses and would appreciate if the authors could address them.", "strength_and_weaknesses": "### Strengths\n- The approach might seem like simple extension of hypertransformers, but to make it work is not easy. Thus I believe it's novel and substantial contribution.\n- The paper is fairly well-written and easy to follow.\n- The results are convincing but could have been better. I expand on this in the following section.\n\n### Weaknesses\n- I would have appreciated a slightly harder task for tiered imagenet i.e. 20 way, 5 shot.\n- From task incremental learning (fig 3), it seems that adding more tasks does lead to some forgetting as performance for $\\theta_{0}$ and $\\theta_{1}$ is consistently higher than $\\theta_{3}$ and onwards (esp. for tiered imageNet). Could authors explain why is this the case?\n- I would have appreciated if the authors had considered some external few-shot continual learning baselines such as FSCL from [Wang et al. 2021b].\n- Instead of freezing the prototypes, could they be updated with momentum update as it was done in momentum contrastive learning [1]?\n- What  does the x-axis of individual plots in figure 3 refers to?\n- Why is there is no MergedHT baseline for tiered imageNet (fig4) and task incremental learning (fig3)?\n- Since you use task embeddings as a state (like in recurrent neural networks). Do these embeddings get updated in between tasks by some other mechanism? For context, I am trying to understand if there could be an issue like vanishing gradients with the proposed recurrent neural network like approach?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BUMYUg4YA2H", "reviewer": "Reviewer_hN2L", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper extended the HyperTransformer and proposed an Incremental HyperTransformer(IHT). The proposed IHT re-used the old weights as an input to generate the new weights for the incoming task in the continual sequence. This mechanism encourages the new model to utilize the knowledge in the old model. Experiments on class-incremental setting and task-incremental setting demonstrate the effectiveness of the proposed method.", "summary_of_the_review": "Based on the comments in the Main Review part, I tend to reject this paper. The main reason is the lack of explanation and experiments of why the proposed method can catastrophic forgetting. ", "strength_and_weaknesses": "Strength:\n- The architectural innovation seem effective and are easily interpretable when thinking about utilizing the learned knowledge.\n- The prototypical loss is orthogonal to the architecture design.\n- I think even general classification task can benefit from the proposed architecture design and loss design.\n\nWeaknesses:\n- I think it is better to give a clear definition of few-shot class-incremental-learning\n- In my opinion, the re-use of old weights can help the convergence of the new task because the old weights can be treated as an initialization of the new model, but how does this mechanism alleviate the catastrophic forgetting? Does author use the replay buffers? It seems that we have to store all the query sets of each seen tasks.\n- The proposed prototypical loss is similar to the Nearest-Mean-of-Exemplars Classification in iCaRL[1]. More discussion is expected.\n- The experiments lack comparison with other continual learning baselines.\n\n\n[1] Rebuffi S A, Kolesnikov A, Sperl G, et al. icarl: Incremental classifier and representation learning. CVPR 2017.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject