Meta-OLE: Meta-learned Orthogonal Low-Rank Embedding
Authors: Ze Wang
Yue Lu
Qiang Qiu
Venue: ICLR 2022 Submitted (2022)

Abstract: We introduce Meta-OLE, a new geometry-regularized method for fast adaptation to novel tasks in few-shot image classification. The proposed method learns to adapt for each few-shot classification task a feature space with simultaneous inter-class orthogonality and intra-class low-rankness. Specifically, a deep feature extractor is trained by explicitly imposing orthogonal low-rank subspace structures among features corresponding to different classes within a given task. To adapt to novel tasks with unseen categories, we further meta-learn a light-weight transformation to enhance the inter-class margins. As an additional benefit, this light-weight transformation lets us exploit the query data for label propagation from labeled to unlabeled data without any auxiliary network components. The explicitly geometry-regularized feature subspaces allow the classifiers on novel tasks to be inferred in a closed form, with an adaptive subspace truncation that selectively discards non-discriminative dimensions. We perform experiments on standard few-shot image classification tasks, and observe performance superior to state-of-the-art meta-learning methods. 

Under review as a conference paper at ICLR 2022
META-OLE: META-LEARNED ORTHOGONAL Low- RANK EMBEDDING
Anonymous authors Paper under double-blind review
ABSTRACT
We introduce Meta-OLE, tion to novel tasks in few-shot image classification. The proposed method learns adapt for each few-shot classification task feature space with simultaneous inter-class orthogonality and intra-class low-rankness. Specifically, a deep feature extractor is trained by explicitly imposing orthogonal low-rank subspace struc- tures among features corresponding to different classes within given task. To adapt to novel tasks with unseen categories, we further meta-learn light-weight transformation to enhance the inter-class margins. As an additional benefit, this light-weight transformation lets us exploit the query data for label propagation from labeled to unlabeled data without any auxiliary network components. The feature subspaces allow the classifiers on novel tasks to be inferred in a closed form, with an adaptive subspace truncation that selectively discards dinddriiiiinatiee dimensions. We perform experiments on standard few-shot image classification tasks, and observe performance superior state-of-the -atthof-the-art meta-learning methods.
geometry-regularized method for fast adapta-
explicitly geometry-regularte
INTRODUCTION
Meta learning, also referred to as learning to learn, aims at acquiring knowledge from a distribution of tasks, and learn to quickly solve novel tasks sampled from the same or similar underlying task distribution. Meta learning is extensively studied under the context of few-shot learning (FSL), and realized by models that adapt efficiently when given only a few labeled samples of novel tasks. The research is mainly driven by how to design the adaptation for acquiring task-specified models efficiently and robustly. Prototypical networks (ProtoNets) (Snell et al.) (017) adapt to new tasks by computing the prototype of each class simply as the average of feature vectors, with all the network parameters shared across tasks. MAML (Finn et al.) 2017) adapts to new tasks by a few iterations of gradient descent, and this approach has inspired many subsequent methods (Antoniou et al.) 2018 Finn et al.) 2018 8 Nichol et al.) 2018: Yoon et al.) 2018). The adaptation of the entire network makes it hard be scaled to large networks, and many recent efforts focus on adapting the last classification layer only (Gordon et al. 2019 Bertinetto et al.) 2019), while assuming a universal feature extractor that is shared across all tasks.
In this paper, we attempt to attack few-shot image classification from a new perspective of geometry regularization of the feature space. As observed in (Lezama et al. 2018), training deep networks with softmax and cross-entropy loss does not simultaneously enforce intra-class similarity and inter- class margins. On the other hand, encouraging features to be in low-rank subspace in each class as well as orthogonal across classes can significantly improve the robustness of deep classification net- works. While such explicit orthogonal low-rank geometry regularization has been proved successful in classical classification tasks Lezama et al. (2018): Qiu et al. (2018): Lezama et al. (2017). it re- mains highly non-trivial to extend this geometry regularization approach to tasks with novel classes involved at the testing stage. In few-shot image classification, further challenges arise from the de- mand of robust generalization to novel unseen classes. As we will show in this work, large class margins resulting from explicit geometric regularization can potentially allow novel knowledge to be represented by a composition existing knowledge, and can reduce interference across classes. An illustration is presented in Figure]

---

Under review as a conference paper at ICLR 2022
(a) Feature space trained with standard softmax and cross-entropy loss. While linear boundaries are learned for the seen classes, the lack of enforcing intra-class similarity and inter-class separation leaves little space for novel classes (purple and and yellow dots) to be well represented in the feature space with- out interference.
(b) Orthogonal low-rank embedding encourages feature across classes to collapse to orthogonal sub- spaces, cach of which has minimum dimensions. This intra-class similarity and inter-class separation allow novel classes (purple and and yellow dots) be represented in the feature space with signifi- cantly reduced interference.
Figure 1: Illustration of the advantages with explicit orthogonal low-rank geometry regularization.
Motivated by the maximal-margin feature space geometry, introduce meta-learned orthogonal low-rank embedding (Meta-OLE) to combine the simplicity of the prototype based methods and the adaptivity of the parameter adaptation based methods. Specifically. we encourage an orthogonal low-rank structure to the feature space across classes. Thus feature vectors of the same class reside in a subspace with imposed low-rankness, while subspaces across classes are encouraged to be as orthogonal as possible. While imposing poometry-regularizali
to the feature space over seen classes has been investigated Lezama et al. (2018) 2017); Wen et al. (2016), the induced feature extractor does not guarantee to generalize well to novel object classes that are unseen during training. To extend an orthogonal low-rank embedding to a few-shot learning scenario, we introduce meta- learning framework with a light-weight adaptive orthogonal low-rank transformation that is able to adapt efficiently to novel classes with very few examples. We then show that, given the imposed low- rank orthogonal geometry, the final classification of query samples can be performed by subspace projections, where the projection matrices are directly inferred from the few labeled examples in a closed form. And we show that, to adaptively adjust the dimension of the projections based on the compactness of the feature subspace, the robustness of the classifier to outlier examples can be further improved. The closed-form inference of class labels allows unlabeled samples be easily involved in the learning of the adaptive orthogonal low-rank transformation for label propagation, and improved performance is observed without any auxiliary parametric components to infer the pseudo labels.
Despite being simple and geometry-motivated the proposed method achieves on public FSI. datasets superior performance to state-of-the-at methods that often involve more sophisticated components.
In summary, our contributions are as follows:
We propose to impose low-rank orthogonal geometry in feature space for few-shot learning.
We introduce meta-learned adaptive orthogonal low-rank transformations for efficient adaptations to novel tasks with unseen classes.
Geometry-motivann motivated classifier based on subspace projections with adaptive dimension se- lection is introduced for fast and robust class inference.
The effectiveness of the proposed Meta-OLE is validated with extensive experiments on few-shot image classification.
2 METHOD
In this section, we start with the basic formulation of FSL, and then we introduce each of the com- ponents of the proposed Meta-OLE framework in detail.
2

---

Under review as a conference paper at ICLR 2022
Feature en extraction
Adaptive on orthogonal low trnnfformationn 2 (P
Adaplive subspace projection (roj.(k) U,U,
(3)
An illustration of the proposed meta-learned orthogonal low-rank embedding. The input images of both support (blue, yellow, and green boxes) and query set (red box) are all first mapped to feature vectors by universal feature extractor, where an orthogonal low-rank geometry is imposed. The features at each task then go through the adaptive orthogonal low-rank transformation, whose parameters are adapted by samples of each task, and achieve higher intra-clas similarity and inter-class orthogonality. Finally, an adaptive subspace projection used for each class, where the projection matrices are inferred directly in closed form.
Figure 2:
2.1 PRELIMINARY
Scalars, vectors and tensors are denoted as lower-case, bold lower-case, and bold upper-case letters, e.g., n1, x, X, respectively. For example, we denote image vector x, and use X [x1, X2,... to denote a collection of images. X. denotes the collection of images within X with label c.
Few-shot image classification with episodic training. A few-shot learning (FSL) task is usually defined as K-way N-shot learning problem, where N is usually a small number, c.g., N 5. FSL with meta-learning is usually formulated as series of episodic training. Typically, in each episode, one FSL task is generated by first sampling K categories from the training data, each of which contains N samples to form the support set S, ; XKxNN J. An adaptation to the FSL model is then performed on S., by, e.g., computing prototypes (Snell et al. 2017). or updating the network parameters (Finn et al. 2017). After adaptation, samples from the same categories of each episodic, referred as the query set Q {xi..... Xxx xxxxs }, are sampled to evaluate the updated model, and the error propagated back update the parameters.
Orthogonal low-rank embedding. The idea learning linear transformation to recover the orthogonal low-dimensional intrinsic structures in data originally proposed in (Qiu & Sapiro, 2013). In (Qiu & Sapiro 2015). a linear transformation is learned to restore a low-rank structure for data from the same subspace, and, at the same time. force a maximally separated structure for data from different subspaces. This idea is further generalized to deep learning in (Lezama et al. 2018). where orthogonal low-rank embedding (OLE) is introduced as a regularization term the training of deep classification networks for improved performance. Given a collection of N samples in R for a K- K-way classification task, the transformation (Qiu Sapiro 2015) is computed as
K
min THEM
TX. 11.
TX
(1)
where ||- ||, denotes the nuclear norm and serves as a convex lower bound of the rank function on the unit ball in the operator norm, and is linear transformation to learned. Specifically, the first term in (T) encourages the transformed representations within each of the K classes to reside in a low-rank subspace. The second term in ① promotes orthogonality of the subspaces across classes.
Theorem Qiu & Sapiro 2015)
< ||A]]- + [B||, with equality satisfied if and only if the column spaces of A and B are orthogonal.
According Theorem the cost value of (I) is always nonnegative. Moreover, it achieves the minimum at zero if and only if different classes become orthogonal after the transform. See (Qiu & Sapiro 2015) for more details of this formulation.
3

---

Under review as a conference paper at ICLR 2022
Based on (eezama al., 2018) introduces generalization to the training of deep neural net- works, where an orthogonal low-rank embedding (OLE) loss is proposed as a regularization term to facilitate the learning of typical image classification networks with the cross-entropy loss. Specifi- cally, the OLE loss is defined as
K COLE = Σ Izll |Z||- = Σ ((x)) (x)
K
(2)
where & denotes the nonlinear transformation associated with deep network. When is net- work that ends with a ReLU activation, training with the OLE loss leads to orthogonal inter-class subspaces, which is equivalent to explicitly pushing feature from different classes the maximum cosine distance (Lezama et al. 2018).
2.2 ADAPTIVE ORTHOGONAL LOW-RANK SUBSPACE PROJECTIONS
In the proposed method, samples within FSL task is first mapped by universal feature extractor, parametrized by a deep CNN in our setting, to a feature space, where we explicitly encourage low- rank subspace for each class well as orthogonality among different classes. To adaptively promote higher intra-class compactness and inter-class orthogonality in each task, we then meta-leamn a light- weight orthogonal low-rank transformation, that learns on the samples of each task and adapts the parameters for task-specific feature transformations. After learning the task-specific transforma- tions for better low-rank orthogonal embeddings, the classification of unlabeled samples in query set is accomplished by adaptive subspace projections, with the projection matrices inferred directly from feature vectors of the support set closed form. We further introduce adaptive subspace projections by selecting principal dimensions for projections, and truncating non-discrininative di- mensions for improved robustness. All components are detailed next.
Universal feature extractor. Following standard practice, we train universal feature extractor b, which is typically CNN that is shared across tasks. Specifically, given an image the feature extractor maps to a d-dimensional feature vector z E R (x)). Different from the common practice that the feature extractor & is solely learned using the gradient propagated back from the error at tasks, we explicitly encourage orthogonal low-rank feature geometry across classes in each training episode. This is achieved by supervising the parameters in + with OLE loss
COLE
(X.)l+ |Xx())
The same feature extraction applied to both support set and query set to obtain the respective features = {2:] KxN Z' = [2/1 KxM
Meta-learned adaptive transformation. Training universal feature extractor & on limited train- ing classes can hardly guarantee that the orthogonal low-rank feature geometry to be perfectly gen- eralized to novel tasks in practice. To fully exploit the support set samples, we therefore propose to meta-learn a light weight adaptive transformation L parametrized by a tiny network with param- eters 8, to adaptively transform the feature vectors of novel tasks for more compact intra-class and orthogonal inter-space subspaces. The parameters 0 in is adapted to the new task given features from the support set iteratively, and initialization is learned across the episodic training.
Specifically, given a collection of the features from the support set Z', we perform P iterations of parameter updating 0, in order to project the features to space that better presents the low- rank orthogonal geometry. At each iteration p. the transformed features are computed as Zp v (Z, 0p-1). and the parameters are then updated by
= 0-=0p11 BV (Z))) p= I
44)
where 0g = 0, which serves as the universal parameter initialization for all tasks and to be optimized by tasks in episodic training. The updated parameters at the last iteration Op are returned as the final parameter of & for a specific task, and transform both the support set and query set features,
Ż (ž. (p) Ž' = (z
v (2/3 KXM
(5)
4

---

Under review as a conference paper at ICLR 2022
Adaptive subspace projections classifiers. The imposed geometty-reguulrizztie of the thogonal low-rank subspaces naturally leads to a subspace projection- based classifier that fully utilizes the obtained feature geometry. Since now the desirable features of each class reside in low-rank subspace, the inference of class labels of query samples can then he effectively computed by projecting the feature vectors each subspace of classes, and observing the norm of the pro- jected vectors. Specifically, in Kwway N-shoot FSL, given a collection of support set feature vectors of a class Ze kg the subspace, i.c. span(Z.), can be directly inferred in closed-form. Let Ze = U. be the singular value decomposition of the Ze, the rows of U, [b....., bx] form an orthogonal bases of ce Then any feature vector in the query set can be projected onto the subspace span(Z.) by (2) U.U. z. In practice, based on the rank of the features, there can be non-discriminative dimensions contained in (in the most optimal case, only a single dimension is sufficient to represent the subspace of class). Given the singular values Σg = [s1,-... the non-discimiinative dimensions are bases in Ug that correspond to low singular values. When the singular values E. are sorted in descending order, we can easily truncate non-discriminative dimensions by discarding bases in U. whose associated singular values are lower than a threshold. In practice, we introduce non-negative hyperparameter < 1.0, and truncate bases in U. with singular values lower than T x 81. Formally, we obtain the projections
proj. (z) U.(1 : rr]U.[1 :r]]2.
where S. 2 x $1 and 81+1 T x $1. Ideally, each query sample will lie in the subspace of its class, thus the projection will mostly preserve the norm of the feature vector. For each 2j. we then define the norm of the projected vector of 2j to the subspace of class e as the unnormalized probability of sample x; belonging to class c, i.e.
Bc(z,) = P(z, E c) = exp(IIproj, (3)
Ed= exp(IIproj__ ( (2,)
and standard cross-entropy loss then used for computing and back- pack-propagating errors. The advan- tages are further validated with real-world experiments in Section 3
Leveraging query samples. While meta-learning the adaptive orthogonal low-rank transforma- tion L we can further leverage the query samples with no labels. In each iteration of updating in Vr, we can augment the data by assigning a pseudo label to each query sample. This can be effi- ciently implemented by projecting each query sample to each subspace inferred from the support set as in (6. and finding the label maximum probability as in 77) This achieves transductive learning in FSL without introducing any auxiliary components the network. And we introduce non nog-negative hyperparameter < 1.0 as the weight of the contribution from the query set with pseudo labels. In this transductive setting, the updating to in V becomes:
0p = 0p-1 3V (ToLeCL +a (Z,, Z₂J)),
(8)
for p I, ...., P. and [Z, Zy} here denotes concatenation of the transformed support and query features.
In summary, the proposed method consists of universal feature extractor & for projecting high- dimensional image inputs to feature vectors. Iterative updating to the adaptive orthogonal low-rank transformation & adapts the model the task at hand. The final classification is performed by
(=:0)) á proje ye- All the parameters are $ z
subspace projections obtained in a closed form. jointly updated by the loss
x
L= Lootma (Ú. v) + XCLLE [Z, ZT), with 9 being the inferred label from 7 and y the true label.
(9)
We summarize the overall training of our method in Appendix Algorithm
3 EXPERIMENTS
Datasets. We perform experiments on FSL benchmarks including miniImageNet, tieredImageNet,
and Caltech-UCSD Birds dataset (Welinder al. 2010) (CUB). In minilmageNeet (Vinyals et al.)
5

---

Under review as a conference paper at ICLR 2022
Table I: 5-way few-shot image classification comparisons on minilmageNet and rieredImageNet with 95% confidence intervals. We conduct experiments with both shallow (Conv-4) and deep (ResNet-12) networks and compare the performance with various rate-of-the-an methods. de- notes performance obtained with leveraging query samples.
minilmageNet
IagredImageNett
Methods
Backbone
1-shot
5-shot
I-shot
5-shot
ABML
Conv-4
37.65 0.22
56.08
MatchingNets (aaswani et al.
2017
Conv-4
43,56 1: 0.44
55,31
1 0.73
MAML (Finn el al. 2017
Conv.4
48.70 1: 1.84
63.11
1 0.92
$1.67 1 1.81
70.30 =: 1.75
Reptile Niihool et al.
2018
Conv.4
49.97 1 0.32
65.99
1 0.58
Nels (Snell
2017
Conv.4
44.53
65.77
53.31 0.90
72.69 0.74
R2-D2 (Berinetto
2015
Conv-4
48.70 1: 0.60
65,50 0,60
VERSA (ordon et al.
2
Conv-4
53.31 + 1.80
67.30 + 0.91
RelationNets Sung Fai 2018
Conv-4
50,44
65.32
54.48
65.32
Bayesian MAML Yooa et al.
2018
Conv-4
44.46 0,30
62.60
DKT (Patacchiola èt mi. 2020
Conv-4
49.73 1: 0.07
64,00 1: 0.09
OVE PG GP + Cosine (ML) (nell (Sell & Zemel 202
Conv-4
50.02 de 0.35
64.58 + 0.31
OVE PG GP Cosine (PL)
Snell& & Zemel
Conv-4
48.00
67.14
Meta-OLE
Conv-4
53.82
0.84
71.23
0.72
57.87 0.0
74.97 0.85
Meta-Nets (Munkhdalai & Yo. 2017
ResNet-12 12
57.10
70.04 0.63
SNAIL (Mistra u. 2018
ResNet-12
55.71
1: 0.99
68.88 1: 0.92
ProtoNeis (Snell et al. 2017
ResNet-12
59.25
+ 0.64
75.60
+ 0.48
61.74 0.77
80.00 0.55
Munkhdu (MukesNNeMMunnhdii
2018
ResNet-11
56.88
0.62
71,94
TADAM (Oreshkin ct al.
2018
ResNet-12
58.50
a: 0,30
76.70 0,30
Meta-OLE
12
65.28
1: 0.64
81,96
1 0.62
67.72 0.72
84.20 :: 0.36
Meta-OLE"
ResNet-12
67.04 1. 0.72
83,21 1 0.67
68.82 1 0.71
85.51 1 0.39
2016), there are 100 image classes from subset of ImageNet (Deng et al. 2009), with 600 images for each class. We follow the standard practice (Finn et al.] 2017) to split the training, validation, and testing sets with 64, 16, and 20 classes, respectively. magred (Ren et al., 2018) is large subset of ImageNet that contains 608 classes with 1,300 samples in each class. Specifically, in Iigred there are 351 classes from 20 categories for training, 97 classes from 6 cat- egories for validation, and 160 classes from 8 different categories for testing. Samples for both minilmageNee and tieredImageNett are randomly cropped and resized to 84 x 84 for training, and standard center cropping is performed to the testing images. The 200 classes in the CUB dataset divided into 100, 50, and 50 classes, for training, validation, and testing, respectively. Following standard practice, we report results with both 5-way 1-shot and 5-way 5-shot. Note that in the case of 5-way 1-shot learning, the inference of the projection of each class (-)j is reduced to using the normalized feature vector of the single support sample only without adaptations based on the intra-class similarity.
Implementation details. All experiments are conducted on a server with Nvidia RTX 3090 graphic cards, and each has 24GB memory. Every experiment report can be trained and tested on a single card. The machine is also equipped with 512GB memory and two AMD EPYC 7502 CPUs. We use PyTorrch for the implementations of all experiments. We train the networks using stochastic gradient descent with a Nesterov momentum (Sutskever et al. (2013) of 0.9, for total of 80 epochs and 1000 random sampled tasks within each epoch. The initial learning rate set to be 0.025, which decays by a factor of 0.1 at epoch 50 and epoch 60. Following common practice, we use random resized crop and random horizontal flip as the data augmentation transformations.
For the hyperparameters, we set the weight of the OLE loss in 9 to A 0.1. We use P 10 to adapt the adaptive orthogonal low-rank transformation for 10 iterations. The weight of the transduc- tive OLE loss is set to a 0.25. The truncation threshold of the adaptive subspace projection is set to 0.9. All hyperparameters selections will be discussed later in Section 3.1
Following the common practice, two network structures are included in the discussion of perfor- mance. Conv-4 is constructed by stacking 4 Bon--BN-ReLU-pooiie
block, with 64 channels in each layer, and the output feature is flattened into feature vector that is fed to the adaptive orthog- onal low-rank transformation. ResNet-10 and ResNet-11 are 10-layer and 12-layer deep residual networks (He et al. 2016), with 4 residual blocks, and each block has 64, 128, 256, and 512 chan- nels, respectively. We use a global average pooling to convert the 3D feature maps for each sample into a 512-dim feature vector. We use a small scale network with 3-layer fully connected (FC) lay-

---

Under review as a conference paper at ICLR 2022
Table 2: Results on cross-domain few-shot image classification with the Conv-4 backbone, and 5- way few-shot image classification on the CUB dataset with both shallow and deep backbones. denotes performance obtained with leveraging query samples.
NeniImageNet - CUB
CUB (Conv.4)
CUB (ResNett10)
Methods
I-shot
5-shot
1-shot
5-shot
1-shot
5-shot
Feature Transfer
32.77 + 0.35
50.34
0.27
46.19
+ 0.64
68.40
+ 0.79
63.64 + 0.91
81.27 # 0.57
ABML
29.35 + 0.26
45.74
49.57
68.94
0.36
Baseline ++ CChnn al
2019
39.19 1 0.12
57.31
0.11
61.75
0.95
78.51
1 0.59
69.55
1 0.89
85.17 0.50
MatchingNet Vaswani cl RI
201
36.98 +: 0.06
50.72
0.36
60.19 + 1.02
75.
0.35
71.29
+ 0.87
83.47 0.58
ProtoNets (
33.27 + 1,39
52.16
52.52 1,90
75.93
0.46
73.22
#
85.01
RelationNel Duung
2018
37.13 0.20
51.76
62.52 0.34
78.22
0.07
70.47
83,70 0.55
MAML (Finh et al 201
34.01 +: 1.25
0.62 48.83
56.11 +11 0
74.84 +: 0.62
70.32 +: 0.99
80,93 0.71
Bayesiun MAML Yoon el
2018
33.52 1: 0.36
51.35 0.16
55.93 0.7.
72.87 1: 0.26
DKT (Patacchinla èl A020
40.14
56.40
62.96
77.76 0.62
72.27 0.30
85.64
OVE (ML) (Sneu & Zemer 202
39.66
+ 0.18
55.71 +
63.98 1.43
77.44 + 0.38
OVE (PL) (Snell & Zemel
2021
37.49 0.11
57.23 1
60.11 0.226
79.07 1. 0.05
Mefa-O1.F
40.66
58.23
68.75 0.331
84,74 0.21
79.76
88.82
Mefa-OLE'
41.40
60.82
71.32
86. 11 0.23
81.10
90.04
ers as the adaptive orthogonal low-rank transformation Batch normalization (BN) and ReLU activation are adopted after each FC layer. All parameters in v are allowed to adapt at each task, including the parameters in BN layers.
Few-Shot Image Classification Following the common practice, first report standard 5-way 1-shot and 5-way 5-shot experiments on all three datasets. The results on minilmageNet and mirred are presented in Table [ Two standard backhones, Conv-4 and ResNet-12 included for comprehensive comparisons. The comparison results on CUB are presented in Ta- We adopt two backbones, Conv-4 and ResNet-10 for comprehensive comparisons following
ble 2 common practice. The proposed method achieves significant improvements over state-of-the-art methods on all datasets.
Generalization The proposed orthogonal low-rank adaptation allows the model fit novel tasks rapidly and effectively, even in the presence of domain shifts between tasks. To validate this, we include the Caltech-UCSD Birds dataset (Welinder et al.) 2010) (CUB) and present cross-domain generalization experiments on mimilmageNet - CUB. As dataset specialized for bird species, CUB poses significant challenge to the few-shot learners due to its weak intra-class discrepancy. We follow the standard practice (Chen et al. 2019) and perform experiments with the Conv-4 backbone and both 5-way 1-shot and 5-way 5-shot experiments. The quantitative results and comparisons are presented in Table Our method achieves high performance for cross-domain few-shot classification, which surpasses counterparts considerably.
3.1 DISCUSSIONS
In this section, we perform ablation study to verify the hyperparameter selections, and provide fur- ther visualizations to show the effectiveness of the proposed components. All experiments per- formed on the 5-way 5-shot task with the CUB dataset and Conv-4 as feature extractor.
Feature extractor. The only hyperparameter introduced in the feature extractor is the weight of the OLE loss X in (9). We present in comprroon of performance with different X. Imposing orthogonal low-rank geometry to the feature extractor can remarkably improve the generalization. Higher values of A consistently improve the accuracy on training categories, while the testing ac- curacy saturates = 0.1. This observation consistent with our intuition shown in Figure [] Enforcing orthogonal low-rank geometry promotes better generalization to novel unseen classes, as the improved intra-class compactness preserves more "open" space so that novel classes can be added to the feature space without causing significant interference with previously seen classes.
Meta-learned orthogonal low-rank transformation. We show in Figure 3 the accuracy of the model when performing task-specific adaptations to 0 in W. We perform 10 steps of inner-loop adaptation and visualize the moving accuracy at step 1, step 5, and step 10. The network perfor- mance is improving substantially w.r.t. the steps of inner-loop adaptation. We further visualize the
7

---

Under review as a conference paper at ICLR 2022
Table 3: Comparisons with different values of X.
À
0.00
0.01
0.05
0.1
0.2
0.5
80
Training
85.13
86.24
86.50
86.89
87.21
87.33
Testing
82.25
84.10
85,45
86.11
86.12
86.11
Accuracy
70
Table Comparisons with different values of We present both test accuracy and the average numbers of dimensions that are preserved after truncation.
60
50
Step 1
40
Step
0.1
0.3
0.5
0.7
0.9
Step
Accuracy 85.33
85,46
85.62
85.88
86.07
86.11
20000
40000 Iterations
60000
80000
Dimension
3.14
2.43
1.73
1.30
1.12
1.08
Table 5: Comparisons with different values of a.
Figure 3: Moving average of the accuracy at different when updating the adaptive orthogonal low-rank transformation.
0.05
0.15
0.35
0.45
0.55
Accuracy 84.81
85.25
86.11
86.02
85 42
83.25
feature space in Figure showing how the features task are progressively refined orthogonal low-rank geometry when 0 is being updated iteratively. It is clearly shown that this task-specific adaptation plays crucial role when learning novel tasks.
Adaptive subspace projections. The adaptive subspace projection allows extra flexibility by ad- justing т. the threshold that controls the truncation of non-discriminative dimensions in the projec- tions. We show in Table 4 how the values of affect the results. It is shown that high values like r = 0.9 result in truncating nearly all but the first basis after the singular value decomposition for projection. The compactness of the transformed intra-class features allow single basis to well rep- resent the subspace of a class, and achieve the best performance by removing all other dimensions that potentially contain noise.
Leveraging unlabeled samples. Our framework of meta-learned orthogonal low-rank transfor- mations allows unlabeled query samples to be easily leveraged without introducing any auxiliary network components. The only additional hyperparameter introduced is that controls the weight of the inner-loop learning with pseudo labeled query samples. We perform additional experiments shown in Table Leveraging unlabeled samples in the inner-loop adaptation is able to improve the performance. However, imposing a large value of a close to 1 can decrease final accuracy, as higher values of might cause the wrong assignments of the pseudo labels overwhelm the inner-loop adaptation. We therefore use consistently a = 0.25 across all experiments.
4 RELATED WORK
Feature geometry in deep learning. The idea of explicitly imposing intra-class similarity and intra-class separation is extensively studied in metric learning (Wang et al. 2017 Chen et al. 2017
Hadsell et al. 2006, Schroof et al. 2013) Sun et al., 2014 Wen et al. 2016 Oh Song et al. (016). As
in the most representative loss functions for metric learning, pairwise loss (Hadsell et al. 2006) and
triplet loss (Schroff et al. 2015), effective training of metric learning requires careful sampling of samples, especially negative ones for the most informative training. The basic assumption of metric leaming is that a common metric space is shared across related tasks. Such idea has also been extended few-shot learning as in Matching Networks (Vinyals et al. (2016) and PrototypeNets (Snell et al. 2017), where the networks remain shared across all tasks. Relational Networks (Sung) al. 2018 further extend to learnable metric, parametrized by a network trained across tasks.
Meta Meta-learning. Meta learning, also referred as learning to learn Thrun & Pratt 2012). trains the models to leverage shared knowledge among tasks within a distribution to solve novel task ef- ficiently and effectively (Andrychowiez al. 2016 Rusu et al. 2019: Finn et al.) 2017 Gordon ctt al., (ndryychwo et 21,] Ravi & Larochelle 2016 Rusu et al. (019). It has attracted increasing attention in recent years, and recent advantages driving the development of meta
8

---

Under review as a conference paper at ICLR 2022
4
OLE: 26.18 Acc: 78.0%
OLE: 18.17 80.4%
OLE: 13882 Acc: 81.6%
12.20 83.2%
OLE: 10.50 Acp: 85.2%
lteration 6
Iteration
Iteration
Iteration 9
Iteration 10
OLE: 8.93 Acc: 87.6%
OLE: 9.27 Acc: 90.4%
OLE: 6.48
Acc: 91.2%
OLB: 5.60 Ace: 91.6%
OLE: 4.35 Acc: 92.0%
Figure 4: Visualization of the feature space of Zp while updating 0 in v for 10 iterations. Feature vectors from three classes in a 5-way FSL task are embedded with PCA, and visualized in three colors. Viewing angles are adjusted for better visualizations. The value of the OLE loss and the accuracy at each iteration are noted in the figure.
learning with different directions. Early efforts focus training feature extractor that is compat- ible with certain metric, in massive training episodes. ProtoNets (Snell et al., 2017) learn feature
projection that is robust to feature comparisons in Euclidean space. DSN (Simon al. 2020) al-
lows high-order statistics the subspace for each class to be considered. And R2-D2 (Bertinetto) al. 2019) learns the feature extractor that adapts well closed-formed linear classifiers. Gradient-
descent based methods (Finn al. 2017, Rusu et al. 2019 Finn et al.) 20188) Yoon et (018)(01)) learn et an initialization that allows the network to adapt efficiently to new tasks given supervisions from few samples. Parameter prediction based models (Gordon et al.] 2019 Qiao et al. [018 Gidaris Komodakis 2019) generate cask-dependent network parameters, typically linear classifiers, given observations on novel tasks. Recently, leveraging unlabeled samples in query sets further boosts the performance of FSL., where the pseudo labels of the query samples are inferred either directly from comparing features (Simon al. 2020), by a labeling network (Kye al. 2020). Finally, in addition to its successtul application in few-shot learning, the idea meta-learning has also been proved be effective at diverse tasks such memory (Bartunov et al.) 2020) and reinforcement learning (Schweighofer & Doya 2003 Gupta et al. 2018 Sæmunndsoo et al. 2018).
5 CONCLUSION
In this paper, we introduced meta-learned orthogonal low-rank embedding (Meta-OLB) for effective generalization novel few-shot classification tasks by meta-learning with geometry regularization to feature space. We imposed orthogonal low-rank geometry in feature space across categories to promote maximum intra-class similarity and inter-class separation simultaneously. To further al- low effective generalization to novels tasks with unseen categories, we meta-learned an orthogonal low-rank transformation that can fully utilize both labeled support set and the unlabeled query set to update the task-specifi transformations. This explicit geometry regularization allowed us to formu- late the final classification layer as class projections, with projection matrices directly obtained from the feature vectors in closed-form. Determined by the intra-class similarity of each class, an adap- tive dimension truncation is further introduced to selectively discard non-discriminntiie dimensions in the subspace projections for improved robustness. The idea of orthogonal low-rank geometric regulation a central theme that motivates every component in the proposed Meta-OLE. We per- formed both comparisons against state-of-the-art methods and ablation study, to fully validate the effectiveness of each proposed component.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. In NeurlPS, 2016.
Antreas Antoniou, Harrison Edwards, and Amos Storkey. How train your maml. arXiv preprint arXiv:1870.09502, 2018.
Sergey Bartunov, Jack W Rac, Simon Osindero, and Timothy P Lillicrap. Meta-learning deep energy-based memory models. ICLR, 2020.
Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differ- entiable closed-form solvers. ICLR, 2019.
Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. arXiv preprint (rXiv:1904.04232 2019.
Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. Beyond triplet loss: a deep quadru- plet network for person re- ce-identification. In CVPR, pp. 403 403-412, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, pp. 248-255. IEEE, 2009.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leaming for fast adaptation of deep networks. In ICML, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. arXiv preprint arXiv:1806.02287,, 2018.
Spyros Gidaris and Nikos Komodakis. Generating classification weights with gnn denoising autoen- coders for few-shot learning. In CVPR, 2019.
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner. Meta- learning probabilistic inference for prediction. ICLR, 2019.
Abhishek Gupta, Benjamin Eysenbach, Chelsea Finn, and Sergey Levine. Unsupervised meta- learning for reinforcement learning. arXiv preprint /e..iv::00...0 04640, 2018.
Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, volume 2, pp. 1735-1742. IEBE, 2006.
K. He, X. Zhang, Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
Seong Min Kye, Hae Beom Lee, Hoirin Kim, and Sung Ju Hwang. Meta-learned confidence for few-shot learning. arXiv e-prints, pp. arXiv-2002, 2020.
José Lezama, Qiang Qiu, and Guillermo Sapiro. Not afraid of the dark: Nir-vis face recognition via cross-spectral hallucination and low-rank embedding. In CVPR, pp. 6628-6637, 2017.
José Lezama, Qiang Qiu, Pablo Musé, and Guillermo Sapiro. Ole: Orthogonal low-rank embedding- plug and play geometric loss for deep learning. In CVPR, pp. 8109-8118, 2018.
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta- learner. In ICLR, 2018.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In ICML., 2017.
Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons. In International Conference on Machine Learning, pp. 3664-3673.
PMLR, 2018.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv: (803. 1803.0299. 2018
10

---

Under review as a conference paper at ICLR 2022
Hyun Oh Song. Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In CVPR, pp. 401444012,20 2016.
Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning. In NeurIPS, 2018.
Massimiliano Patacchiola, Jack Turner, Elliot J. Crowley, and Amos Storkey. Bayesian meta-
learning for the few-shot setting via deep kernels. In NeurIPS, 2020.
Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan Yuille. Few-shot image recognition by predicting parameters from activations. In CVPR, 2018.
Qiang Qiu and Guillermo Sapiro. Learning transformations for clustering and classification, Journal of Machine Learning Research, 16(1):187-223, 2015.
Qiang Qiu, José Lezama, Alex Bronstein, and Guillermo Sapiro. Foresthash: Semantic hashing with shallow random forests and tiny convolutional networks. In ECCV, pp. 432-448, 2018.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. ICLR, 2016.
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classifica- tion. ICLR, 2018.
Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In ICLR, 2019.
Steindór Sæmundsson, Katja Hofmann, and Mare Peter Deisenroth. Meta reinforcement learning with latent variable gaussian processes. arXiv preprint arXiv: 1803.0751, 2018..
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In CVPR, pp. 815-823, 2015.
Nicolas Schweighofer and Kenji Doya. Meta-learning in reinforcement learning, Neural Networks, 16(1):5-9, 2003.
Christian Simon, Piotr Koniusz, Richard Nock, and Mchrtash Harandi. Adaptive subspaces for few-shot learning. CVPR, pp. 4136-4145, 2020.
Jake Snell and Richard Zemel. Bayesian few-shot classification with one-vs-each P Polya-Gamma augmented gaussian processes. ICLR, 2021.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurlPS, 2017.
Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation by joint identification- verification. NewrIPS, 2014.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In CVPR, pp. 1199-1208, 2018.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initializa- tion and momentum deep learning. ICML, pp. 1139-1147, PMLR, 2013.
Sebastian Thrun and Lorien Pratt. Learning to learn. Springer Science & Business Media, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NeurIPS, 2017.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukeuoglu, and Daan Wierstra. Match- ing networks for one shot learning. NeurIPS, 2016.
Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin. Deep metric learning with angular loss. In ICCV, pp. 2593-2601, 2017.
11

---

Under review as a conference paper at ICLR 2022
P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. (CNS-TR-20100-01)) 2010.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In ECCV, pp. 499-515. Springer, 2016.
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian madel-agnostic meta-learning. In NeurlPS, 2018.
12

---

Under review as a conference paper at ICLR 2022
APPENDIX
We summarize the training of the proposed meta-learned adaptive orthogonal low-rank subspace transformations in Algorithm |
Algorithm Meta-learned adaptive orthogonal low-rank subspace transformations.
1: Given: A description the tasks as K-way S-shot with M query samples in each class.
2: Given: B for truncating subspace dimensions, À for the weight of OLE loss, for transductive weight applicable.
3: Initialize & and 0 in V
4: repeat
5: Sample task with support set S { Kxx and query set Q [x1
KX in each task. 6: Extract feature vectors for both S and Q, 2, = ((xx), 2j = (xx)).
7: for Inner iterations p do
8:
Compute adapted features = V (2 0p) Compute OLE loss (oLE($).
9:
10:
Update with (4) for inductive setting, or with 8 for transductive setting with pseudo labels inferred by 66)
11:
end for
Obtain transformed features
[z; Pp) for both support and query samples.
13:
Obtain adaptive subspace projection for each class as in (6).
Compute class probability of query sample in (7)
15:
Update parameters with the loss in
16:
until Converge
17:
Return Φ y with parameter 0.
13


Reviews:
[{"review_id": "uvXAeHbGDU", "reviewer": "Reviewer_jSyC", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "pros:\n1.\tThe proposed modification of OLE in FSL is interesting. \nCons:\n1.\tThe experiment results on single-domain FSL cannot prove the effectiveness of the proposed method. The authors provide both inductive and transductive results on several datasets. However, almost all results are far beyond the state-of-the-art method, especially the transductive ones.  \n2.\tThe inner loop in Eq.(4) is similar to that in MAML, except that no classification loss is used. It would be better to explain why this term is missed in inner update.\n3.\tThe authors need to provide more detail on the subsection ‘Adaptive subspace projections as classifiers’, from which several questions arise:\na)\tThe proposed projection is only usable when K>1. It is not clear how the proposed method can help the original OLE to better adapt to 1-shot setting.\nb)\tAs I can understand Eq.(7), the classification results depend on the norm of the projected vector of original feature on each subspace. I wonder if the feature is normalized before projection.\nc)\tWill including SVD in training process lead to low training efficiency?\n4.\tMore ablation study can be provided, for example, the result of directly using nearest neighbor instead of projection after training with Meta-OLE; result of combining Meta-OLE with other regularization such as S2M2 [1], BF3S [2], etc.\n[1] Mangla P, Kumari N, Sinha A, et al. Charting the right manifold: Manifold mixup for few-shot learning[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2020: 2218-2227.\n[2] Gidaris S, Bursuc A, Komodakis N, et al. Boosting few-shot visual learning with self-supervision[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 8059-8068.\n", "summary_of_the_paper": "This paper mainly targets few-shot learning. The authors leverage the former work OLE in FSL to let novel class features more discriminative. The key contribution lies in the modification to make OLE robust enough to be extend to novel classes. To solve this, the authors propose to meta-learn a light-weight module ", "summary_of_the_review": "Despite interesting perspective, this paper lacks detail on the methodology and ablation study. Meanwhile the performance is not convincing enough. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "P2u0hvwtGMa", "reviewer": "Reviewer_jCT7", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Strengths] \n+ The paper is well organized and easy to follow.\n+ The visualization of the feature space of z_p is good.\n+ The leveraging of meta-learning is interesting. \n\n\n[Weaknesses] \n- The level of technical contribution and novelty is incremental.\n   1) The proposed universal feature extractor is directly borrowed from OLE.\n   2) The meta-learning strategy of the proposed meta-learned adaptive transformation\tis similar to the MAML algorithm.\n   3) The proposed leveraging-query-samples is similar to [A,B] by employing the unlabeled query images to transductive learn the low-rank transformation with the predicted pseudo labels.\n- It is better to analyze the effect of the proposed adaptive subspace projection. For example, which cases of images benefit from such a classifier? \n- The model performance is left behind some existing FSL methods:\n\n   [A] Peyman Bateni, Jarred Barber, Jan-Willem van de Meent, Frank Wood: Improving Few-Shot Visual Classification with Unlabelled Examples. CoRR abs/2006.12245 (2020).\n\n   [B] Malik Boudiaf, Imtiaz Masud Ziko, Jérôme Rony, José Dolz, Pablo Piantanida, Ismail Ben Ayed: Transductive Information Maximization For Few-Shot Learning. CoRR abs/2008.11297 (2020).\n\n   [C] Da Chen, Yuefeng Chen, Yuhong Li, Feng Mao, Yuan He, Hui Xue: Self-Supervised Learning for Few-Shot Image Classification. ICASSP 2021: 1745-1749.\n\n   [D] Peyman Bateni, Raghav Goyal, Vaden Masrani, Frank Wood, Leonid Sigal: Improved Few-Shot Visual Classification. CVPR 2020: 14481-14490.\n\n   [E] James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, Richard E. Turner: Fast and Flexible Multi-Task Classification using Conditional Neural Adaptive Processes. NeurIPS 2019: 7957-7968.\n\n   [F] Imtiaz Masud Ziko, Jose Dolz, Eric Granger, Ismail Ben Ayed: Laplacian Regularized Few-Shot Learning. ICML 2020: 11660-11670", "summary_of_the_paper": "The paper aims to address the few-shot image classification problem. The authors propose a geometry-regularized method via meta-learned orthogonal low-rank embedding. The technique first employs an orthogonal low-rank geometry imposed feature extractor for extracting image-level feature vectors and then followed by parameter-adapted orthogonal low-rank transformation to achieve higher intra-class similarity and inter-class orthogonality. Finally, each class employs an adaptive subspace projection to directly classy images. In sum, the performance of the proposed method is somewhat weak compared with the existing FSL methods. ", "summary_of_the_review": "The primary concerns of this paper are the weak performance and limited novelty. Though the usage of meta-learning is interesting, the proposed learning strategy is borrowed from the existing method, and the final model performance does not surpass the existing techniques.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2O4HZNBljq2", "reviewer": "Reviewer_HpAs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths. \n\n+The proposed method, which imposes class-wise orthogonalized distribution in feature space for generalization to novel classes, is elegant. \n\n+Good writing. The paper is well structured and easy to understand. \n\n+The proposed method outperforms the state-of-the-art methods in few-shot learning. \n\n+Ablation studies, which show the effects of OLE, meta-learned lightweight adaptation transformations, adaptive subspace projections, and the leverage of unlabeled samples, are satisfactory.\n\nWeakness. \n\n-The OLE loss is the same as the existing work (Lazama et al., 20018). The contribution of this paper is only the combination of OLE and meta-learning for a few-shot learning problem.  \n\n-The difference between the adaptive subspace (Simon et al., 2020) should be highlighted, and performance should be compared in Table.1.\n\nMinor problems. \n\n-Figure 4 is not referred to in the main text.\n\n-What is the number of mini-batch samples in each iteration of the Eq.(4)? \n", "summary_of_the_paper": "This paper proposes Meta-OLE, which imposes a low-rank orthogonal geometry in feature space for few-shot learning. An adaptive orthogonal low-rank transformation is introduced for efficient adaptations to novel tasks with unseen classes. Also, a geometry-motivated classifier based on subspace projections with adaptive dimension selection is presented for fast and robust class inference\n\nThe proposed method is composed as follows: \n\n1.　Samples within a FSL task is first mapped by a universal feature extractor trained with the OLE loss.  (Actually, with softmax loss as shown in Eq.(9))\n\n2.　To exploit the support set samples, the proposed method meta-learns a lightweight adaptive transformation $\\Psi$ with OLE loss as shown in Eq.(4).  \n\n3.　The proposed method projects each data to subspace, and its lengths are used for classification as shown in Eq.(7). (But it is effective only using one leading direction for each class.)\n\n4.　The proposed method uses pseudo labels for using query samples for training. \n", "summary_of_the_review": "The proposed method exploits the existing OLE loss to the few-shot learning problems. The method, which imposes class-wise orthogonal distribution in feature space, is elegant for generalization to the novel classes, and the evaluation is satisfactory. \n\n==\nPost rebuttal\n==\n\nMost of my concerns are addressed. \nHowever, I have degraded my score slightly because the novelty is incremental. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a meta-learning method with a latent feature space with a special structure of orthogonality and low-rankness. This paper is well written, and the use of the orthogonal low-rank embedding for meta-learning is interesting. The experimental results (including additional experiments in the author response) demonstrate the effectiveness of the proposed method. The author response addressed some concerns of the reviewers. However, the novelty of the proposed method is not high enough.
Decision: Reject