Towards Practical Few-shot Query Sets: Transductive Minimum Description Length Inference
Authors: Ségolène Tiffany Martin
Malik Boudiaf
Emilie Chouzenoux
Jean-Christophe Pesquet
Ismail Ben Ayed
Venue: NeurIPS 2022 Accept (2022)
Keywords: few-shot learning
transductive inference
minimum description length
primal-dual optimization
clustering

Abstract: Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of a given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as an instance of the challenging yet practical problem of extremely imbalanced $K$-way classification, $K$ being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-art methods. Motivated by these observations, we introduce a \textbf{P}rim\textbf{A}l \textbf{D}ual Minimum \textbf{D}escription \textbf{LE}ngth (\textbf{PADDLE}) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of a few-shot task. It is hyper-parameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments over the standard few-shot datasets and the more realistic and challenging \textit{i-Nat} dataset show highly competitive performances of our method, more so when the numbers of possible classes in the tasks increase. Our code is publicly available at \url{https://github.com/SegoleneMartin/PADDLE}.

Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inference
Ségolène Martin Université Paris-Saclay, Inria, CentraleSupêlec, CVN
Malik Boudiaf ÉTS Montreal
Emilie Chouzenoux" Université Paris-Saclay, Inria, CentraleSupélec, CVN
Jean- Chriihhisttophe Pesquet Université Paris-Saclay, Inria, CentraleSupélce, CVN
Ismail Ben Ayed ÉTS Montreal
Abstract
Standard few-shot benchmarks often built upon simplifying assumptions the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as instance the challenging yet practical problem of extremely imbalanced K-way classification, K being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-ar methods. Motivated by these observations, we introduce PrimAl Dual Minimum Description LEngth (PADDLE) formulation, which balances data data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of few-shot task. It is hyper- parameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments
the standard few-shot datasets and the realistic and challenging i-Nar dataset show highly competitive performances of our method, more so when the numbers of possible classes the tasks increase. Our code is publicly available at htthhtp::/// // teutthu...c. com / SSgoo weagooeeeeaattn/DDD.....
1 Introduction
The performance of deep learning models is often seriously affected when tackling new tasks with limited supervision, i.e., classes that were unobserved during training and for which we have only handful of labeled examples. Few-shot learning [I] [2] 3 focuses on this generalization challenge, which occurs in breadth of applications. In standard few-shot settings, a deep network is first trained
"E. Chouzenoux acknowledges support from the European Research Council Starting Grant MAJORIS BRC-2019-STG-8888 STG-850925.
'The work of J.-C. Pesquet is supported by the ANR Chair in AI BRIDGEABLE. !The work of I. Ben Ayed supported by the DATAIA Institute, and part of his sabbatical-leave visit the Université Paris-Saclay.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

on a senge-scale&on dataset including labeled instances sampled from an initial set of classes, commonly referred to as the base classes. Subsequently, for novel classes, unobserved during the base training. supervision is restricted to a limited number of labeled instances per class. During the test phase, few-shot methods are evaluated over individual tasks, each including a small batch of unlabeled test samples (the query set) and a few labeled instances per novel class (the support ser).
The transduction approach has becomes increasingly popular in few-shot learning, and large body of recent methods focused on this setting, including, for instance, those based on graph regularization 4 [], optimal transport 6 7 feature transformations g 9. information maximization [10 [2]
and transductive batch normalization 13 2), among other works [I4] 13 9 16, T7]. Transductive few-shot classifiers make joint predictions for the batch of query samples of each few-shot task, independently of the other tasks. Unlike inductive inference, in which prediction is made for one testing sample at a time, transduction accounts for the statisties of the query set of task typically yielding substantial improvements in performance. On standard benchmarks, the gap in classification accuracy between transductive and inductive few-shot methods may reach 10%; see [II, for example. This connects with a well-known fact in classical literature transductive inference [19] 20 21],
which prescribes transduction as an effective way to mitigate the difficulty inherent to learning from limited labels. It is worth mentioning that transductive methods inherently depend on the statistical properties of the query sets. For instance, the recent studies in [2 10 showed that variations in the class balance within the query set may affect the performances of transductive methods.
Transductive few-shot classification occurs in variety of practical scenarios, in which we naturally have access to batch of unlabeled samples at test time. In commonly used few-shot benchmarks, the query set of each task is small (less than 100 examples), and sampled from a limited number of classes (typically 5). Those choices are relevant in practice: During test time, one typically has access to small unlabeled batches of potentially correlated (non-i.i.d)) samples, e.g., smart-device photos taken at given time, video-stream sequences, in pixel prediction tasks such as semantic segmentation 121, where only handful of classes appear in the testing batch. However, the standard few-shot benchmarks are built upon further assumptions that may not always hold in practice: the few classes effectively present in the unlabeled query set are assumed both to be known beforehand and to match exactly the set of classes of the labeled support set. We relax these assumptions, and extend current benchmarks so that the query-set classes are unknown and do not match exactly the support-set classes, but just belong total number of classes represented in the support set to be higher than typical values, while keeping the number of classes that are effectively present in the query set to be much smaller.
much larger set of possible classes. Specifically, we allow the
Our challenging yet practical setting raises several difficulties for state-of-the-an transductive few- shot classifiers; (i) it is instance of the problem of highly imbalanced classification; (ii) the labeled support set includes "distraction" classes that may not actually be present in the test query samples; (ii) we consider K- K-way classification tasks with K much larger than the typical values used in the current benchmarks. We evaluated 7 of the best- pest-performing methods and, expectedly, observed drops in their performance in this challenging setting.
-he transductive few-shot
Motivated by these experimental observations, we introduce a Minimum Description Length (MDL) inference, which balances data-fitting accuracy and model complexity for a given few-shot task, subject to supervision constraints from the support set. The model-complexity term can be viewed as continuous relaxation of a discrete label cost, which penalizes the number of non-empty clusters in the solution, fitting the data a given task with as few unique labels as necessary. It encourages competition among a large set of possible classes, preserving only those that fit better the task. free, and could be applied on top of any base-class training.
Our formulation is Furthermore, derive a fast primal-dual block coordinate descent algorithm for optimizing objective, with convergence guarantee, and a linear computational complexity at each iteration thanks to closed-form updates of the variables. We report comprehensive experiments and ablation and the more realistic and challenging iNat dataset
studies on mini- Imagenet, fiered-Imagent (33 for fine-grained classification, with 908 classes and 227 ways at test-time. Our method yields competitive performances in comparison to the state-of-the-ar with gaps increasing substantially with the numbers of possible classes in the tasks.
"Noe that the transductive setting is different from semi- supervised sew-shot learning 18 that uses extra data. The only difference between inductive and tranductive inference is that predictions are made jointly for the query set of task, rather than one sample at time.
2

---

(In, Yn ) [Pead_ where yv E {0, 1] J.Disil, with constraint Yous O Mest = Ø, i.e., the test and base classes distinet. Each task includes labelled support set S { Vn nets and unlabelled query set Q Ieelo both containing examples from classes in Peett Using the feature extractor 2 Few-shot setting and task generation
[Pheal denotes the base dataset, where each In Xease is Base training Let Dosse (I M/ In=1
sample from some input space X, Yn E {0. 1) 1)Dimuul the associated ground- truth label, and Vusee the set of base classes. Base training learns feature extractor fΦ A - 2, with parameters & and 2 lower-dimensional space. For this stage, an abundant few-shot literature adopts episodic training, which views Dhaee as a series of tasks (or episodes) so as to simulate testing time. Then, a meta- meta-learner devised to produce the predictions. However, it has been widely established over recent years that basic training, followed by transfer-learning strategies, outperforms most meta-learning methods Hence, we adopt standard cross-entropy training in this work.
24 25 26
Evaluation Evaluation is carried out over few-shot tasks, each containing samples from Diet
So trained base data, the goal is predict the classes of the unlabeled samples in Q for each few-shot task, independently of the other tasks.
Task generation For a given task, let K denote the total number of possible classes that are represented in the labeled support set S, and Ken the number of classes that appear effectively in unlabeled query set Q (i.e. classes represented by at least one sample). The standard task generation protocol assumes that the set of effective classes in Q matches exactly the set of classes in S, i.e. K= Keff < [Dess which amounts to knowing exactly the few classes that appear in the test samples. Then, a fixed number of instances per class are sampled for each query set, forcing class balance. We relax these assumptions, and propose setting where the set of effective classes in Q is not known exactly and belongs to a much larger set of possible classes. First, we allow the total number of possible classes K to be higher than typical values, with K = [Dees while keeping the number of effective classes in the query set be typically much smaller: Kett << [Dissl K. For instance, in our experiments, K can go up to 227. This, as expected, results in K K-way problems that are more challenging than the standard 5-way tasks used in the few-shot literature. Secondly, once the effective classes are fixed, and given a budget of images, sample the query set under the data joint distribution (i.e. uniform draws among all available examples), that the statistics of the sampled query sets more faithfully reflect the natural distribution of classes. Figure [iguutailllstrttte our framework (base training, inference and task generation).
3 Proposed few-shot inference formulation
For given few-shot task, Q = [ Jonnnete and (mn, ; Yn Jnet be two subsets Diest such that Q S = Ø. Let N Q + [S]. Up to reordering, and ll = Q + 1, NJ.. We denote zn fo(xx) the feature vector corresponding to the data sample F,, and un = (un.
suppose that 1l {l...., [Q)
E {0, 1 the variable assigning the data to one of the possible .......,,i. i.e un if æn belongs to class & and 0 otherwise. We define the variables
classes in (1..... K). giving the class proportions á = (ūx) Ax as
12
Un
Vk E (L....,K)
where Ax is the unit simplex of R
We propose to cast transductive few-shot inference as the minimization of an objective balancing data-fitting accuracy and partition complexity, subject to supervision constraints (known labels) from the support set S. Our approach estimates jointly U (un)IsnsN, which defines partition of
3

---

Base training
Inference for few-shot task
Support set
tetepy
Prodicted
Clasetter
Query
Fosiune
f
Parametrized module
Supervision constraint
Loss finction
1/
PADDLE
Predicted clases
Classiffier
+ partition complesity
Class represented in query
Class represented query
Weights
ones during training
Figure I: Overview of the proposed framework: Base training, PADDLE inference and task genera- tion. The example depicted in the right-hand side illustrates how the support and query classes do not match exactly, unlike in standard few-shot settings. The support set includes "distraction" classes that may not actually be present the query set, e.g. classes "Alligator", "Peacock" and "Butterfly". All possible classes (K classes) are represented in the support set, but only an unknown subset among these possible classes (Keff effective classes) appear in the query set, with Kay < K.
S U Q, and the class prototypes W (wE)ISkSK E (TRC) through the following problem:
minimize U,W
Un.A 11 Zn
úR In(ūx),
(2)
k= n=
data-liting sccuracy
portition complexity
S.t
Uh E AA
vn E (1,..,,,/Q))
Uh yn
Vn [ +1,...,
(C)
Note that, in the second line of facilitate optimization.
2 we have relaxed the integer constraints on query assignments to
Effect of each term The purpose of objective (2 to classify the data of few-shot task with as few unique labels necessary. The data-fitting term has the same form as the standard K K-means objective for clustering, but is constrained with supervision from the support-set labels. This term evaluates, within each class k, the deviation of features from class prototype wg, thereby encouraging consistency of samples of the same class. The number of effective (non-empty) classes that appear in the solution (K I K ), encouraging low cardinality partitions of query set Q. This term is the Shannon entropy of class proportions within Q: it reaches its minimum when all the samples of Q belong to a single class, i.e., =j (l,..., such that ú = and all other proportions vanish. It achieves its maximum for perfectly balanced partitions of Q satisfying û, 1/K for all k. In practice, this terms promotes solutions that contain only a handful of effective classes among a larger set of K possible classes.
term implicitly penalizes the
Connection Minimum Description Length (MDL) The objective in (2) could be viewed as partially-superviin instantiation of the general MDL principle. Originated in information theory, MDL is widely used in statistical model selection 227. Assume that we want to describe some input data Z = (zn) isssjoi with statistical model M, among family of possible models. MDL prescribes that the best model corresponds to the shortest description of the data, according some coding scheme underlying the model. It balances data-fitting accuracy and model complexity by minimizing L(Z + XM) w.r.t M. L M) measures the code length of the prediction of Z made by M, while L(M) encourages simpler models (Occam's 's razor (27). e.g., models with less parameters. For instance, in unsupervised clustering, is common to minimize discrete label cost as measure of model complexity [28] 29 30). Such label cost is the number of effective (non-empty)

---

clusters in the solution, and is used in conjunction with a fog-likelihood term for data- sita-fitting:
K
L(M) =
(3)
==1
K
1Q
((Z =
Un,k In Pr(zz, k;; w)x)
(4)
k= n=1
In this expression, M ((wk) U} with wy being the parameters of some probability distribution Pr(z,, kc; 2 describing the samples in cluster kt while U contains assignments as denoted above.
The discrete model complexity in (3) is commonly used in MDL-based clustering 28
29 30 despite
the ensuing optimization difficulty. As this measure is discrete count of non-empty clusters, it does not accommodate fast optimization techniques. Typically, it is either handled via cluster merging heuristics (starting from an initial set of clusters) [30] or via combinatorial move-making algorithms [28]. both of which are computationally intensive, more so when dealing with large sets of classes.
continuous relaxation of the discrete label count in (3) (in the supplemental material, we provide a graphical illustration in the case when K = 2). While penalizing similarly the number of effective classes, it has several advantages over the label count in 3 despite the surprising fact that it is not common in the MDL-based clustering literature, to our best knowledge. First, it is a continuous function of assignment variables U. This enables us to derive a fast primal-dual block coordinate descent algorithm (Section 4j, with convergence guarantee, and linear computational complexity at each iteration thanks closed-form updates of the variables. Second, it has clear MDL interpretation: it measures the number of bits required to encode the set of classes, given class probabilities Pr(k:) ûg. Ak K}..
Our parpiiion term in 2 can be viewed
Following the Kraft-McMillas theorem 27 any probability distribution Pr(z., [k; ws) corresponds to some coding scheme for storing the features of cluster k, and In Pr(z,, wx) is the number of bits required to represent any feature using coding scheme Pr(z,. [<; w&). Clearly, our data- aata-fitting
term in (2) also fits into this MDL interpretation by assuming that each probability Pr(z,, [k; we) is Gaussian distribution with mean wt and covariance fixed to the identity matrix:
Pr(zul ke; wa) x exp (- 11 z..112) Ak (I....,
(5)
Differences with the existing objectives for transductive few-shot inference Unlike most of the existing transductive few-shot methods, our MDL formulation does not encode strong assumptions on the label statistics of the query set (e.g. class balance or/and a perfect match between the query and support classes). Information maximization methods, such as [10 [1], maximise the confidence of the sample-wise predictions while encouraging class balance. Optimal transport methods [6] 7 estimate an optimal mapping matrix, which could be viewed as a joint probability distribution over the features and the labels, while imposing a hard class-balance constraint via the Sinkhorn-Knopp algorithm. Both information maximmiztiiiiiiinonnnn and optimal transsoor methods have inherent (lass-balance bias and, as shown in the experiments, undergo drastic drop in accuracy when the number of possible classes increases (as this corresponds highly imbalanced problems). Prototype rectification [8] transforms the query features so as to minimize the difference between the overall statistics of the query and support sets. This relies on the assumption that the support and query classes match perfectly. Therefore, it is not adapted to our setting. Inspired from graphical models, the Laplacian regularization in 4 is a pairwise label correction (rather than learning) method, which encourages assigning the same label to query samples that are close in the input space; it does not learn class representation from the query set (the prototypes being fixed as those of the support set).
Identifying À through an unbiased probabilistic K-means interpretation of 2 Another inter- esting view of our few-shot inference objective in (2) can be drawn from the probabilistic K-means objective well-known in the clustering literature [31] 32 33 In fact, probabilistic K-means cor- responds to minimizing C Z M) in (4) as a generalization of K K-means, which corresponds to the particular Gaussian choice in (5) It is well-known that probabilistic K K-means has strong bias
In the general context of clustering, the name probabilistic K --mans was first coined by 31
5

---

towards balanced partitions [3 331. The objective can be decomposed to reveal hidden term promoting class balance. Using Bayes rule Pr(k]z,; wx) Pr(z,, wa (r(k)) empirical estimates of marginal class probabilities Pr(k) ug: and the choice of Pr(zn kk; ms) in (5) we have:
K
10
Unrk lw
2n
Wn,k In (r(2nlk;wss)
k=1
k=
n=l
K
K
Wry In Pr(k: (x) m)s) + IQ
dh In (
(6)
k=
R=1
where C stands for equality, up to an additive constant independent of optimization variables. Minimizing the last term in the second line of (6) has an effect opposite to the model-complexity term in our objective in 2 it reaches its minimum for perfectly balanced partitions. Therefore, our objective in (2) is way to mitigate such a bias in K-means, allowing imbalanced partitions. This suggests setting = [ to compensate for the hidden class-balance term K-means. This would
make our inference aaper-parameted free. Thus, for our study, we fixed À without optimizing the parameter À via validation.
[Q] in all benchmarks,
4 Primal-Dual Block- Coood-Co Descent Optimization
We derive a block coordinate descent algorithm for our formulation (2). At each iteration of algorithm, the minimization steps are closed-form, with linear complexity in K and N. We emphasize that, in problem (2). the minimization over U could not be carried in closed-form, even for fixed prototypes W, and the simplex constraints are difficult handle. One straightforward iterative solution to tackle (2) would be to deploy a projected gradient descent algorithm. However, as shown in the comparisons Section 5] this strategy is computationally demanding and does not yield better classification results than the algorithm we propose below.
Primal-Dusi formulation We transform problem (2) into an equivalent optimization problem by introducing dual variable. Let first define the linear operator A: (RR)** - R which maps the probabilities to the proportions of the classes, i.e. A A: + (ūx) Then, problem (2) can be re-written as follows:
K
minimize U.W
Un,k fa.k|/10, 1w
2n AH(AU),
s.t
2
k=1n=1
where H is the negative entropy function the positive orthant of KK H(x) EK (x(x)) for (Za)iSagR E R with
(n(t)
ift
+(1)
O
if O,
+00
otherwise.
We can now appeal to the concept of Fenchel-Legendre conjugate function. For a given convex function f. its conjugate function, denoted by f*, is defined as f*VV) -Vp((VV)))((U) where
U
V is dual variable 334 Def. 13.1] and ( denotes the Fuclidean scalar product R Since function H is a proper, lower semicontinuous, convex function, according to the Fenchel-Moreau theorem 34 Thm. 13.37), the equality H (H*** holds. Hence, follows that
-AH(AU) inf H (V) (V, AU)).
Plugging 8 into 7 and using the expression of !I [35 Ex. 3,21], one obtains the following minimization problem with respect the primal variable U. the weights W and the dual variable V:
N
K
minimize U.W.V
un
zn 12
+
vu- X(V, AU), s.t

---

Handling the simplex constraint To deal efficiently with the simplex constraint in (9, we add an entropic barrier on soft assignment variables un leading the following modified problem:
N
K
minimize U.W.V
Un Mn
2n za12
2N VV, AU)
s (Un,))
(10)
1=
entropic barrier
The last term in (10) acts as barrier for imposing constraints Un 0 and, each iteration, yields
_k=1 Vv.k 1 and assignments U. This simplifies the iterative constrained-minimizatoon steps over in our algorithm below through
closed-form updates of both the dual variables for constraints Σ
simple closed-form softmax operations (derivation details provided in supplemental material).
Block coordinate descent algorithm To minimize the cost function in (10), we pursue block coordinate descent approach [36] (37), which guaranteed converge; see Proposition below. At each iteration, we successively minimize the objective with respect to the variables U, V, and W respectively. To do so, detailed in Algorithm
the adjoint operator of A, which denote A Our algorithm is
Algorithm 1: PrimAl Dual Minimum Description LEngth (PADDLE)
Initialize W()) as the prototypes computed on the support, and V 0. for & = 1,2,... do
( softmax
(-2 (1wx znl1 1 + -AAV(--)) ISASK
(2) + 1In((AUU)) )k). Ak Kl....,KK.
0
w (0)
Σ
(e-1) nn Znn
(e-1), Yk f1....,K]. 2
Convergence guarantees In addition to its practical advantages in terms of implementation, algorithm benefits from the convergence guarantee described in Proposition [
Proposition 1.
The sequence (U7(), ( v(0)) LEN. generated by Algorithm is bounded. More-
over, any of its cluster points is a critical point to the minimization problem
detailed proof of Proposition 1 [is provided in the supplementary material.
5 Experiments
5.1 Experimental details
Datasets. We deployed three datasets for few-shot classification: mini-Imagennt (38]. tiered- Imagenet 18), and i-Nat [23]. A subset of the ILSVRC-12 data [38], mini--magenet is standard few-shot benchmark, with 60, 000 color images of size 84 84 pixels [3] It contains 100 classes, each represented with 600 images. We followed the standard split of 64 classes for base training, The tiered- Imageeem is another standard few-shot
16 for validation, and 20 for testing [39 25 benchmark, which is a larger subset of ILSVRC-12, with 608 classes and a total of 779, 165 color images of size x 84 pixels. We used a standard split of 351 classes for base training, 97 for validation, and 160 for testing. Finally, the more realistic and challenging dataset i-Nat has 908 classes. We follow the split from 25 with 227 ways at test-time.
Task generation We build the few-shot tasks as follows. Let s denotes the number of shots. We constitute the support set by sampling images according to the uniform distribution for each of the K possible classes of the test set (i.e. K = Dieal 20, 160, and 227 for mini-ImageNNtt, fiered-lmageNet and i-Nat, respectively). For the query set, we first randomly pick Kar < K classes among the K possible classes. We then randomly choose [Q] samples among the images that belong
7

---

to the Ker classes but do not appear in the support set. The right-hand side of Figure example of a task, with the class color coding indicating whether support class appears the query set or not. The results presented below are obtained by fixing !Q] 75. as done in the literature. In the tables, we use fixed Ka =5, but we also present the results over larger range in Fig. 2 Given the difficulty of K- K-way tasks, with K large, we consider 5-, 10- and 20-shot supervision to form the support set. Following standard evaluation, we report the accuracy, averaged across 10, 000 tasks.
F depicts an
Slyper-arameters We emphasize that PADDLE inference does not require any hyperparameter tuning since À in objective (IO) is the number of samples in the query set Q. This is typically not the case for other few-shot methods. For the tuning phase of those methods, we have followed the protocol of [TO]. which for each dataset, uses 5-way, 5-shot scenario on the corresponding validation set. For i-Nat, because no validation split is provided, we reuse the hyper-parameters obtained from liageee-
Feature extraction To ensure the fairest comparison of methods, we use the pretrained checkpoints provided by the authors of [T] The models trained Dasse via standard cross-entropy minimization with label smoothing. The label smoothing parameter is set to 0.1, for 90 epochs, using learning rate initialized to 0.1 and divided by 10 epochs 45 and 66. We use batch sizes of 256 for ResNet-18 and of 128 for WRN28- 10. The images are resized to 84 x 84 pixels, both at training and evaluation time, Color jittering, random croping, and random horizontal flipping augmentations are applied during training.
mini-ImageNNt (K =20)
tiered- fiered-ImageNet (K =160)
Method
Backbone
5-shot
10-shot
20-shot
5-shot
10-shot
20-shot
Bascline 24
54.1
60.7
65.8
29.1
35.7
39.5
LR+ICI 15
54.7
62.0
67.2
BD-CSPN
49.6
54.1
55.6
24.0
26.7
26.0
PТ-MAP 7
ResNet-18 18
25.8
27.4
29.0
4.3
5.1
5.9
LaplacianShot
60.4
65.2
68.4
34.7
36.8
39.1
TIM [I]]
66.4
69.4
70.9
26.6
26.7
25.5
a-TIM 10
63.5
67.4
71.7
38.7
44.2
48.4
PADDLE (ours)
63.2
73.3
80.0
45.8
62.5
72.0
Baseline 24
58.0
64.8
69.5
31.8
37.0
42.1
LR+ICI 15
57.2
64.3
70.9
BD-CSPN
51.2
55.5
58.4
23.7
24.9
23.8
WRN28-10
PT-MAP 7
26.3
27.9
29.4
4.4
5.0
5.7
LaplacianShot
64.9
65.3
70.9
28.1
38.0
45.2
TIM [I]
71.9
75.2
76.1
34.1
34.1
34.5
Q-TIM 10
68.3
72.5
75.8
41.9
46.1
51.8
PADDLE (ours)
62.5
72.8
79.3
46.4
60.3
71.1
Table Comparisons of offe--of-the-rt methods on mini-Imagenet and tiered- Imagenet, using the tasks generation process described in Sec. with Keff =5. The metric is accuracy (in percentage). Results are averaged across 10,000 tasks. Results marked with "-" were intractable obtain.
5.2 Results
Main results We compare the performances of our method with state-of-the-art few-shot methods. Our first experimental setting consists in fixing the number effective classes Kar In Table I we evaluate the accuracy mini and fiered-Imagen are displayed in T2 Note that the i-Nar dataset comes with a unique support set for all tasks, with a varying number of shots (labeled samples) per class. Therefore, for i-Nat, we present the results separately. In the second experiment displayed in Figure 2 and we plot the accuracy as function of Kan on mini, Imaged-ImageNe and i-Nar for 5, 10, and 20 shots. Both experiments show that PA PADDLE very competitive with the state-of-the-art methods on proposed new practical few-shot setting, More importantly, the gap between PADDLE and the second best algorithm
for 5, 10, and 20 shots, while results on i-Nar

---

increases significantly with (i) the number of shots and (ii) the number of possible classes K, when going from the dataset mini, which has a small number of test classes (K = 20), to tiered (K 160) and finally to i-Nat (K 227). From the first point, one may conclude that the proposed MDL. formulation benefits better from additional support supervision. We hypothesize to this due to its versatility, as it does not encode strong assumptions on the label statistics of the query set. Also, the second point could be explained along the same line. For instance, methods relying on a class-balance assumptions, such as TIM-GD or PT-MAP, observe drastic drop in accuracy as K increases, even falling below the inductive Baseline. This expected because large values of corresponds to highly imbalanced classification problems for the query sets. For example, Ketr =5 effective classes on tiered corresponds to a relatively more drastic form of class imbalance than on mini, as it would mean only /60 classes are represented in Q versus 5/20. In contrast, PADDLE can deal with strongly imbalanced situations. Note that BD-CSPN 8 is the only baseline whose performance increases with Ken- This method is also among the most affected when the class overlap between the query and support sets decreases, i.e., when Ken gets smaller (left side of the plots Figure and/or K gets bigger (e.g. tiered-ImageNet, with K 160). This behaviour might be due to the fact that BD-CSPN encodes strong prior, assuming the support and query classes match perfectly.
PADDLE
TIM
T-TIM
Baseline
BDCSPN
LaplacianShot
5 shots
10 shots
20 shots
80
90
80
ResNet mini-InageNa
Accuracy
70-
80
70-
60-
60
30
50
tiered- tlered-Imagee AmageNet
50-
60-
Accuracy
ReaNet
40-
30-
10-
20-
20
2
9
5
Number of effective classes Kat
Number of effective classes Kafs
Number of effective classes Kefs
Figure 2: Evolution of the accuracy as a function of Kett Each represents dataset, and each column fixed number of shots. All methods use the same ResNet-18 network. Results are averaged across 10,000 tasks.
PADDLE
Baseline
Method
i-Nat (K =227)
TIM
BDCSPN
LaplacianShot
Baseline 24
58.2
BD-CSPN S
57.6
50
PT-MAP 7
6.8
LaplacianShot
43.3
60
TIM [IT]
37.5
a-TIM [TQ
66.7
PADDLE (ours)
84.3
Numbor offective classos Kary
Table 2: Similarly Table 1 results provided with a fixed Kan=5 on i-Nat with 227-ways tasks.
Figure 3: Similarly Fig. 2 we plot the perfor- mances of the methods as functions of Kan on i-Nat with 227-ways tasks.
A Alltion on the objective We hereby ablate on the importance of the purtition-complexg term in Eq. (2). Removing this high-order term yields partially-supervii version of K K-means, which can be optimized effortlessly through iterative closed-form alteraating steps. We provide the comparison

---

between this approach (without the complexxyyy term) and PADDLE (with the partition complexity term) in 3 On can observe that PADDLE systematically outperforms its K K-Means counterpart, with absolute differences in accuracy reaching up to 30% + on challenging scenarios.
Partition complexity term
mini (K =20)
tiered (K =160)
i-Nat (K =227)
in Problem 2
5-shot
10-shot
20-shot
5-shot
10-shot
20-shot
Without With
49.6
57.2
63.6
28.3
35.8
39.5
57.5
63.2
73.3
80.0
45,8
62.5
72.0
84.3
Table 3: Importance of the partition-compleitt term. Results are computed using a ResNet-18
and Katt =5, and averaged across 10,000 tasks. Without the partition-com cerrttioncoopleiity term in Eq. (2), the algorithm has little incentive to be parsimonious in its choice of the effective classes. As matter of fact, removing this term reduces Problem (2) to a partially-supervisee K-means algorithm, notoriously known to encourage balanced solutions across the classes.
Ablation the optimization procedure As second ablation, we propose to compare our proposed Alternating Minimization Algorithm ?? to straightforward first-order approach. More precisely, comparison. we directly optimize Eq. (2) through Projected Gradient Descent, denoted as PGD, where simplex projection step 440] takes place after every iteration to ensure that the simplex constraints on U remains satisfied. For PGD,, we use Adam 41 with
0.001. We found this to be the highest learning rate that leads to convergence. We also add the convergence time of the second best competing method, a-TIM. Results are provided in Fig. + PGD much slower than PADDLE, with ratio between run times neighboring an order of magnitude. Additionally, as hinted from the last points of the PGD curve, oscillations seem to occur, indicating that a more sophisticated learning rate policy would be necessary to achieve clear convergence.
[W we
10
TIM
Time to convergence (s)
Accuracy
10-4
PADDLE
a-TIM PGD
N/A
61.0
PGD
10-3
6.1 x 10
53.0
PADDLE
2.6 x 10
77.0
10--
16"
Elapsed time (s)
Figure 4: Comparison of PADDLE and PGD in terms of convergence speed (Left) The chosen criterion as a function of the elapsed time on a randomly chosen 20-shot, Ken =5 task sampled from heree-Imageehe Both methods iterations. (Right) time to convergence and accuracy, after reaching a value 10 on the criterion.
the same machine, with markers displayed every 100
Conclusion and limitations
We presented a practical few-shot setting where the number of candidate classes can be much larger than the number of classes that appear effectively the query set. Our setting is instance of highly imbalanced classification, with large numbers ways and potentially irrelevant supervision from the support set. We observed much higher gaps across transductive few-shot methods, some of which fall below the simple inductive baseline. As a solution, we introduced PADDLE. which casts this challenge as
supervised MDL partitioning problem, interpreting the number of unique classes found as measure of model complexity. PADDLE is hyperparameter-ffee and remains competitive over various ofte-of-thhear methods, settings and datasets, without the need for any tuning. However, we do not advocate our method as the one-fits-all, ultimate solution to the extremely challenging few-shot problem. For instance, PADDLE would not be the best performing method in situations where Ken would approach K. In applications where one has knowledge about the label statistics of the query set (e.g. class balance), other methods encoding such knowledge could be more appropriate. More generally, our setting can be seen as a particular case of class-distribution shift between the support and query sets. Interesting future works could complement our current study by overlaying other forms of shifts, e.g. feature shifts 42 43 to our setting.
10

---

References
[1] J. Snell, K. Swersky, and R. Zemel, "Prototypical networks for few-shot learningg in Neural Information Processing Systems (NeurIPS). 2017.
[2] C. Finn, P. Abbeel, and S. Levine. "Model-agnostic meta-learning for fast adaptation of deep networks," in International Conference on Machine Learning (ICML), 2017.
[3] O. Vinyals, C. Blundell, T. P. Lillicrap, K. Kavukcuoglu, and D. Wierstra, "Matching networks for one shot learning," " in Neural Information Processing Systems (NeurlPS), 2016.
[4] I. M. Ziko, J. Dolz, E. Granger, and I. Ben Ayed, "aplacia regularized few-shot Iearning," in International Conference on Machine Learning (ICML), 2020.
[5] L. Yanbin, Lee, M. Park, S. Kim, E. Yang, Hwang, and Yang, "Learning to propagate labels: Transductive propagation network for few-shot learning," in International Conference on Learning Repre- sentations (ICLR), 2019.
[6] M. Lazarou, T. Stathaki, and Y. Avrithis, "Iterttive label cleaning for transductive and semi-supervised few-shot learning," in IEEF/CVF Conference on Computer Vision (ICCV), 2021.
[7] Y. Hu. V. Gripon, and S. Pateux, "Leveraging the feature distribution in transfer-based few-shot learning." in International Conference on Artificial Neural Networks (ICANN), 2021.
[8] J. Liu, L. Song, and Y. Qin. "Prototype rectification for few-shot learning," in European Conference on Computer Vision (ECCV), 2020.
[9] Cui and Guo. "Parameterless transductive feature re-representation for few-shot learning," " in International Conference on Machine Learning (ICML), 2021.
[10] O. Veilleux, M. Boudiaf, P. Piantanida, and I. Ben Ayed, "Realistic evaluation of transductive few-shot learning," " Neural Information Processing Systems (NeurIPS), 2021.
[11] M. Boudiaf, I. M. Ziko, J. Rony, J. Dolz, P. Piantamida, and T. Ben Ayed, "Transductive information maximization for few-shot learning." in Neural Information Processing Systems (NeurlPS), 2020.
[12] M. Boudiaf, H. Kervadec, M. Ziko, P. Piantanida, I. B. Ayed, and Dolz, "Few-shot segmentation without meta-learning: A good transductive inference is all you need?" in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[13] J. Bronskill, J. Gordon, J. Requeima, S. Nowazin, and R. E. Turner, "Tasknorm: Rethinking hatch normalization for meta-learning," International Conference on Machine Learning (ICMI.), 2020.
[14] G. S. Dhillon, P. Chaudhari, A. Ravichandran, and S. Soatto, "A baseline for few-shot image classification." in International Conference on Learning Representations (ICLR), 2020.
[15] Y. Wang, C. Xu, C. Liu, L. Zhang, and Y. Fu, "Instance credibility inference for few-shot learning," in JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[16] G. Qi, H. Yu, Z. La, and S. Li, "Transductive few-shot classification on the oblique manifold," in Proceedings of the IEFF/CVF International Conference on Computer Vision, 2021, pp. 8412-8422.
[17] X. Shen, Y. Xiao, S. X. Hu, O. Sbai, and M. Aubry, "Re-ranking for image retrieval and transductive few-shot classification," " in Neural Information Processing Systems (NeurIPS), 2021.
[18] M. Ren, E. Triantafillou, S. Ravi, J. Snell, K. Swersky, J. B. Tenenbaum, H. Larochelle, and R. S. Zemel, "Meta-learnine for semi-supervised few-shot classification." in International Conference on Learning Representations (ICLR), 2018.
[19] V. N. Vapnik, "An overview of statistical learning theory," IEEE Transactions on Neural Networks (TNN), vol. 10, no. 5, pp.
1999.
[20] T. Joachims, "Transductive inference for text classification using support vector machines." in International Conference on Machine Learning (ICML), 1999.
[21] Z. Dengyong, O. Bousquet, T. N. Lal, J. Weston. and B. Schölkopf, "Learning with local and global consistency." in Neural Information Processing Syxtems (NeurIPS). 2004.
[22] M. Lichtenstein, P. Sattigeri, R. Feris, R. Giryes, and L. Karlinsky, "TAFSSL: Task adaptive feature sub-space learning for few-shot classification," in European Conference on Computer Vision. Springer, 2020, pp. 522-539.
[23] D. Wertheimer and Hariharan, "Few-shot learning with localization in realistic settings," Proceddings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5558-6567.
[24] W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-.. F. Wang, and J.-B. Huang, "A closer look at few-shot classification," in International Conference on Learning Representations (ICLR), 2019.
[25] Y. Wang. W.LL. Chao, K. Q. Weinberger, and L. van der Maaten, "Simpleshot: Revisiting neiihott classification for shot learning"" arXiv preprin::9911.0e 2019.
11

---

[26] Y. Tian. Y. Wang. D. Krishnan, J. B. Tenenbaum, and P. Isola, "Rethinking few-shot image classification: good embedding is all you need?" in European Conference Computer Vision (ECCV). 2020.
[27] D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge University Press,
2003.
[28] A. Delong, A. Osokin, H. Isack, and Y. Boykov, "Fast approximate energy minimization with label costs," International Journal of Computer Vision, vol. 96, no. 1. pp. 1-27, 2012.
[29] J. Yuan and Y. Boykov, "Tv-based multi-label image segmentation with label cost prior," in British Machine Vision Conference (BMVC), 2010.
[30] S. C. Zhu and A. L. Yuille, "Region competition: unifying snakes, region growing, and bayes/mdl for multiband image segmentation." IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI). vol. 18, no. pp. 884-900, 1996.
[31] M. Kcarns, Y. Mansour, and A. Ng, "An information-theorcti analysis of hard and soft assignment methods for clustering," in Conference Uncertainty in Artificial Intelligncce (UAI), 1997.
[32] Tang, D. Marin, Ben Ayed, and Y. Boykov, "Kernel cuts: Kernel and spectral clustering meet regularization," International Journal of Computer Vision, vol. 127, no. 5, pp. 477-511, 2019.
[33] Y. Boykov, H. N. Isack, C. Olsson, and I. Ben Ayed, "Volumetric bias in segmentation and reconstruction: Secrets and solutions." in JEEE/CVF International Conference on Computer Vision (ICCV). 2005.
[34] H. H. Bauschke, P. L. Combettes et al., Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2011, vol. 408.
[35] S. Boyd, N. Parikh, and F. Chu, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Now Publishers Inc. 2011.
[36] D. Bertsekas, Nonlinear Programming, ser. Athena Scientific optimization and computation series. Athena Scientific, 1995.
[37] Y. Xu and W. Yin, "A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion," " SIAM Journal on Imaging Sciences, vol. 6, no. 3. pp. 1758 1789, 2013.
[38] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh. S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bern- stein, A. C. Berg, and I.. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 21 211-252, 2015.
[39] S. Ravi and H. Larochelle, "Optimization as model for few-shot learning," in International Conference Learning Representations (ICLR), 2017.
[40] C. Bauckhage, "NumPy/SciPy recipes for data science: Projections onto the standard simplex," University of Bonn, Tech. Rep., 2020.
[41] D. Kingma and J. Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980. 2014.
[42] Y. Guo, N. Codella, L. Karlinsky, J. V. Codella, J. R. Smith, K. Saenko, Rosing, and R. Feris, "A broader study of cross-domain few-shot learning," in European Conference Computer Vision (ECCV), 2020.
[43] E. Bemnequin, V. Bouvier, M. Tami, A. Toubhans, and C. Hudelot, "Bridging few-shot learning and adaptation: New challenges of support-query shift," in Joint European Conference Machine Learning and Knowledge Discovery in Databases, 2021.
12


Reviews:
[{"review_id": "qPFKXA1Ex_2", "reviewer": "Reviewer_7GFu", "summary": "This paper proposes an algorithm called PADDLE to perform transductive few-shot classification where the classes in the query set may not entirely overlap with the classes in the support set. PADDLE consists of a partially supervised clustering optimization that balances partition complexity and the fit of prototypes to the data. An efficient block coordinate descent algorithm to solve this optimization is proposed and analyzed. Experiments on several few-shot classification benchmark datasets in the proposed transductive setting show that PADDLE often exhibits strong gains in accuracy relative to transductive baseline methods.", "questions": "- How does the current work differ from other transductive few-shot approaches such as those proposed by Ziko et al. (2020) and Veilleux et al. (2021)?\n- Please describe the task generation procedure in more detail. For example, how were the classes in the query set sampled? Are they guaranteed to partially overlap with the support set in some way?\n- In figure 1, is there any insight as to why the accuracy of some baseline methods actually increases with $K_{eff}$?\n- Is there any empirical evidence that setting $\\lambda = |\\mathbb{Q}|$ works better than alternatives?", "limitations": "Limitations could be improved by discussing the following:\n- How can $K$ be set in a real-world scenario where the number of novel classes at test time is unknown?\n- When is the transductive setting likely to not hold? Is there any recourse when there exists only a single query example?", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The few-shot setting investigated in this work is timely and realistic due to removing the assumption of perfect class overlap between support and query sets.\n- The proposed algorithm is highly effective in the proposed transductive scenario. Performance is often quite significantly better than baselines.\n- Experiments are thorough and informative. Sensible ablations demonstrate the utility of the partition complexity term and the proposed block coordinate descent algorithm.\n- Analysis of the complexity of the algorithm and its convergence properties are provided.\n- The writing is clear and insightful, drawing connections to classical algorithms.\n\nWeaknesses\n- It is unclear how the proposed method relates to previously proposed transductive few-shot algorithms and to existing MDL-based clustering algorithms. A related work section should be added.\n- Experiments show the effectiveness of PADDLE in the proposed imbalanced transductive scenario but they do not show the limitations of PADDLE a more balanced setting.\n- The task generation procedure is not described in sufficient detail.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Obg_TOewt6m", "reviewer": "Reviewer_ig28", "summary": "This paper introduced a PrimAl Dual Minimum Description LEngth (PADDLE) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set.It is hyper-parameter free, and could be applied on top of any base-class training. Furthermore, this paper derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments show highly competitive performances of this method, more so when the numbers of possible classes in the tasks increases.", "questions": "1. Based on their paper, I find that support is not very useful for few-shot learning compared with query set. I wonder if the randomly generated support sets would bring about good performance.", "limitations": "They have  adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.The formulation proposed by this method is hyper-parameter free, and could be applied on top of any base-class training. This is very rare and significant.\n\n2.The writing of the article is concise and clear. The formula is rigorous and the logic is clear.\n\n3.Experiments are sufficient and strongly confirm the validity of the proposed method. \n\n4.The proposed few-shot setting is practical and will have a positive impact on many related work.\n\nWeaknesses:\n\n1. There seems to be a lack of a framework to give a visual overview of the proposed approach.\n\n2. On mini-ImageNet dataset and set to 5-shot, the proposed method is slightly inferior to TIM [1]. What is the reason for this? Why didn't this happen on the tiered-ImageNet dataset?\n\n3. Experimental Settings are not clearly explained, including software and hardware conditions, the settings of each hyper-parameter, etc.\n\nReference\n\n[1] M. Boudiaf, I. M. Ziko, J. Rony, J. Dolz, P. Piantanida, and I. Ben Ayed, “Transductive information maximization for few-shot learning,” in Neural Information Processing Systems (NeurIPS), 2020. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "FD6QH-zS57c", "reviewer": "Reviewer_cgRh", "summary": "The paper introduces a Primal Dual Minimum Description Length (PADDLE) formulation, which balances data fitting accuracy and model complexity for a given few-shot task. It extends the current benchmarks, so that the query-set classes of a given task are unknown and do not match exactly the support-set classes, but just belong to a much larger set of possible classes. With a drop in performance on some of the best performing SOTA transductive few-shot methods, the paper introduces a hyper-parameter free MDL-inference formulation which balances data-fitting accuracy and model complexity for a given few-shot task, subject to supervision constraints from the support set.", "questions": "I think a few important baselines are missing. Please find then below:\n\n1. Li C, Kothawade S, Chen F, Iyer R. PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information. arXiv preprint arXiv:2201.12928. 2022 Jan 30.\n\n2. Li, X., Sun, Q., Liu, Y., Zhou, Q., Zheng, S., Chua, T.-S., and Schiele, B. Learning to self-train for semi-supervised few-shot classification. Advances in Neural Information Processing Systems, 32:10276–10286, 2019.", "limitations": "See sections above.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- Block coordinate descent algorithm guarantees a convergence\n- No hyperparameters need of tuning them\n- The formulation can be used on top of any base class\n\n**Weaknesses**\n\n- The paper is hard to follow and is very dense. It would be great to add more empirical results to make it stronger.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "CE3bHArjAW", "reviewer": "Reviewer_m7wN", "summary": "This paper considers a more practice problem about the classes between the support and query sets, i.e., imbalanced K-way classification. A PADDLE method is proposed to solve the problem, which balances data-fitting accuracy and model complexity for a given few-shot task. Experiments show that the method achieves competitive performances in the standard few-shot datasets.  ", "questions": "1. Important related work is missing from this paper, for example: transductive few-shot learning and few-shot image classification.\n2. Some transductive few-shot learning methods should be compared in table 1, such as FEAT [1], SIB [2], DPGN [3].  \n3. The equations (2) and (C) do not match, u_{n} does not appear in the (2). Is the class prototypes W a learned parameter? if not, why minimize the W? \n4. The optimization objective of transductive few-shot learning methods has a cross-entropy loss and entropy-based loss, but equation (2) has data-fitting accuracy and partition complexity.  What is the relationship between them？  \n\n\n[1] Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions\n\n[2] Empirical Bayes Transductive Meta-Learning with Synthetic Gradients\n\n[3] DPGN: Distribution Propagation Graph Network for Few-shot Learning", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThis paper considers a very interesting problem, i.e., the query-set classes are unknown and do not match exactly the support-set classes, but just belong to a much larger set of possible classes.\n\nWeaknesses\n\nThe clarity for this paper should be improved and related work should be discussed.\n \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept