Learning from One and Only One Shot
Authors: Haizi Yu
Igor Mineyev
Lav R. Varshney
James Evans
Venue: ICLR 2022 Submitted (2022)

Abstract: Humans can generalize from one or a few examples, and even from very little pre-training on similar tasks. Machine learning (ML) algorithms, however, typically require large data to either learn or pre-learn to transfer. Inspired by nativism, we directly model very basic human innate priors in abstract visual tasks like character or doodle recognition. The result is a white-box model that learns transformation-based topological similarity akin to how a human would naturally and unconsciously ``distort'' an object when first seeing it. Using the simple Nearest-Neighbor classifier in this similarity space, our model approaches human-level character recognition using only one to ten examples per class and nothing else (no pre-training). This is in contrast to one-shot and few-shot settings that require significant pre-training. On standard benchmarks including MNIST, EMNIST-letters, and the harder Omniglot challenge, our model outperforms both neural-network-based and classical ML methods in the ``tiny-data'' regime, including few-shot learning models that use an extra background set to perform transfer learning. Moreover, mimicking simple clustering methods like $k$-means but in a non-Euclidean space, our model can adapt to an unsupervised setting and generate human-interpretable archetypes of a class.

Under review as a conference paper at ICLR 2022
LEARNING FROM ONE AND ONLY ONE SHOT
Anonymous authors Paper under double-blind review
ABSTRACT
Humans can generalize from only a few examples and from little pre-training on similar tasks. Yet, machine learning (ML.) typically requires large data to learn pre-learn to transfer. Inspired by nativism, we directly model basic human-innate priors in abstract visual tasks e.g., character/doodle recognition. This yields a white- box model that learns transformation-baat how humans naturally "distort" an object when first seeing it. Using the simple nearest-neighbor classifier in this similarity space, our model achieves human-lever character recognition using only 1-10 examples per class and nothing else (no pre- training). This differs from few-shot learning (FSL) using significant pre-training. On standard benchmarks MNIST, EMNIST-letters, and the Omniglot challenge, our model outperforms both neural-network-be based and classical ML in the "tiny-data" regime, including FSL pre-trained on large data. Further, mimicking k-means but in a Euclidenn space, our model enables unsupervised learning and generates human- interrrntterreetalle archetypes as cluster "centroids".
topological similarity by mimicking
INTRODUCTION
Modern machine learning (ML) has made remarkable progress, but this is accompanied by increasing model complexity, with hundreds of neural layers (e.g.. ResNet-152) and millions of parameters (e.g.. AlexNet: 62.3M, VGG16: 138M, BERT: 110M, GTP-3: 175B). This results in a huge appetite for data and increasing difficulty in model interpretability both for users to understand and for developers to tune (e.g.. byperparameters, architecture). As such, Al researchers have pushed for ML models that are prior- and data-efficient (Chollei 2019), that are human-like (Lake et al. 2015). and that exhibit human-intee imman-interpretabble behaviors (Adadi & Berrada 2018).
This poses the fundamental scientific question: How can humans learn so much from so little (May 2015), whereas ML models, e.g., those achieving near-perfection on MNIST using all 60k training images, deteriorate rapidly as (pre)training reduces? Being data-hungry can pose real challenge in data-scarce domains, e.g., in a rapidly evolving pandemic or a low-resource environment. Focusing on its scientific contribution, this paper presents a theoretically sound, white-box model that learns like humans do, with initial success on first set of benchmarks. This includes our state-of-the-art results of hitting 80% 90% MNIST accuracy using the only first / first four training images per class and achieving 6. 75% eттоr (human performance is 4.5%) in the Omniglot one-shot learning challenge without pre-traiiing--on one and only one shot.
We follow the nativist principle: given an example, humans make abstractions: we envision an equivalence class of unseen examples, equivalent the given, in several abstract senses. Many abstraction abilities are inborn in humans and occur unconsciously. When baby sees a mug, (s)he can immediately recognize it regardless of whether is translated, rotated, scaled, or deformed. Such abstraction abilities are considered innate as Core Knowledge priors (Spelke & Kinzler 2007), rather than acquired later life such as learning that mug topologically a doughnut.
We computationally realize the above intuition via our so-called distortable canvas camas- imagining every image smoothly painted on a rubber canvas that can be distorted in many ways, e.g., bent, stretched, squeezed (Figure [A). Due to rubber's viscosity, more distorted canvas transformations expend more energy. This induces a topological similarity, or distance, based on minimal energy: two images are similar if one almost transform into the other with little distortion (energy). Distance is computed by minimizing color and canvas distortions. In general, visual similarity involves not only a canvas but also color transformation. Here, we focus canvas transformations and grayscale images only.

---

Under review as a conference paper at ICLR 2022
A.
identity
bent
stretched
squeezed
conformal
7
7
7
1
-
1
B.
transformation flows
(auullired on the standard carvas)
5
5
5
5
S
6
6
6
6
tramsformations and distortions
smallerr distortion
larger distortion (twisted. torn more)
6
7
smaleer color distortion
arger color distorbion
Figure 1: Canvas transformations (A), transformation flows (B), and distortions (C).
Besides final desired transformation (and distance), we apply the minimal-energy principle to the entire transformation process, i.e., to keep canvas and color distortions small along the entire mind does not process
optimization process (Mesa et al. 2019). When perceiving translation, as sudden displacement from one location to another, but auto-completes a translation path- continuous and preferably short. Gradient descent naturally fits this goal by always choosing the steepest descent. Yet, it suffers from the curse of local minima. Our solution is to lift gradient descent multiple levels of abstraction via multiscale canvas lattices and color blurring, mimicking human abstraction ability that is extremely flexible in multiscale optimization. This yields visualizable and interpretable transformation flows (Figure BB) that either match human intuition (e.g., what humans would naturally do to transform "7" to "1") or provide human intuition to initially nonintuitive settings. The latter case suggests new, human- intnnan-interretabble transformation:: "ah, did not realize this other way of transforming '7' into '1', but now see and it makes perfect sense!"
We show initial success on abstract visual tasks such character and doodle recognition (future work generalizes to photorealistic images after preprocessing technique to turn them into abstract doodles or "emojis"). Our model can be used with the simple nearest-neighbor method to classify images and also in simple k-means style to cluster images. On benchmarks including MNIST (LeCun et al. 1998), EMNiST-letters (Cohen et al., 2017), and the more taxing Omniglot challenge (Lake et al. learned similarity space outperforms both neural-network
2015). running nearest-neighhbo based and classical ML in the "tiny-datæ" or single-datum regime. This includes beating few-shot models pre-trained extra background data (which our model does not require). On MNIST, need one and only one training image per class to hit 80% accuracy and four to hit 90%. In an unsupervised setting, our model enables k-means-like clustering, but our learned similarity space. We generate archetypes as cluster "centroids", e.g., different ways of writing "7" or doodling giraffes.
Literature on data scarcity: few-shot learning (FSL). One-/few-shot learning (Lake al. 2011 Wang et al. 2020) via transfer or meta learning (Pan & Yang 2009] Finn et al., 2017] (017] Hsu et al, 2019
has achieved impressive success in data-scarce scenarios. FSL's success relies on the assumption that source and target tasks are similar enough for pre-training to be relevant (Storkey (2009). However, knowing a priori how relevant the tasks are and understanding what has been pre-learned are often considered "black art". It remains challenging for FSL practitioners to pick proper source data/models for pre-training so as to transfer most effectively and avoid negative transfer (Pan Yang 2009
Meiseles & Rokach 2020). This is especially the case in new domains (e.g., discovering rising trends)
over classical ones (e.g., vision, language). So, rather than "big transfer" (Kolesnikov et al. 2020).
the Omniglot challenge urges "small transfer"- advooatinng reduced pre-training (Lake et al.
2019
This paper follows Omniglot's s pursuit and pushes such reduction to the limit: with absolutely zero pre-training. we still achieved near-human performance and human-ine innrannitterpettablitty.
2

---

Under review as a conference paper at ICLR 2022
Literature data scarcity: transformation and data augmentation. Our distortable canvas is conceptually akin to other transformation- based models, e.g., optimal-transpenr maps (Villani 2009).
transformation-induu induced descriptors (Lowe, 1999) and equivariances (Bronstein et al. [2017] Like all these models, we build transformations into the model rather than into the data as through data augmentation (Krizhevsky et al. 2012). This has the native advantage of hamnsssigg transformational properties directly rather than learning them from data. Models with built-in transformations may be viewed those perfectly learned from infinitely augmented data. Unlike transformations commonly considered in existing models and data augmentation techniques, we do not encode domain knowledge about preferred transformations such as translation, rotation, or scaling. Instead, our model considers all transformations while still maintaining efficiency via our abstracted gradient descent.
2 SMOOTH IMAGE ON DISTORTABLE CANVAS
We introduce a distortable canvas model: any image is thought of as smoothly painted on rubber canvas that can be bent, stretched, etc. We further introduce canvas transformations that can flexibly "distort" an image as we naturally simulate in our mind. More specifically. we define a smooth image by piecewise differentiable M R²² + R,, where IR² denotes an infinite canvas and R, denotes color (grayscale in this paper). We define a canvas transformation by o : TR22 - IR,,, which "reshapes" the underlying canvas of smooth image. Examples include translation, rotation, scaling, and more. We also define a color transformation by X : R., - R,, which "repaints" a color. In this paper, we simplify color transformation and only use it to adjust image contrast via affine x(c) == ac ac+ b. Nevertheless, we do not restrict canvas transformation, but consider all 2D transformations. Given M, a, X. the composition X o M a denotes the transformed image of M by transformations a, X-
To mimic innate human intuition about topological similarity, we introduce canvas distortion (a) for any canvas transformation and color distortion Dc(M, M') between two smooth images M'. Our idea is to search for a transformation that mimics what humans naturally do to transform one image into another. That is, low-distorted a which makes little difference in color between M and the transformed M', or more precisely, an a that minimizes both Dv (a) and Do(M, xoM'oa).
Representating digital and smoothed images. An m x digital image is a discrete M [m] x [n] +
[0, 11, where [k] := {0, 1,..., K - 1}. We call [m] x [n]] the canvas grid and any E [m] x [n] a
grid point. For any m n digital image M. we smooth it to M via a sum of kernels:
M(x) = Σ M(=) k()(z,)))
for any x E R2.
zE(m]x(n]
where a kernel k : R4 + R., is a decaying function (e.g., linear, polynomial, Gaussian decay) and p is metric on I2 (c.g.. .... l,, f, loc). In this paper,
linear decay and loo, i.e., k(p(2, x)) = 1 1 2 zo10o if ||2 1/00 < Pc (for some cutoff radius Pe > 0) and k(p(2, x)) = 0 otherwise.
Note: .M is defined everywhere on R.2. This differs from Gaussian blurring as do not discretize kernels. It is key to use the smoothed image as input, which allows computing gradients analytically. As such, we always smooth any digital image at first and then only manipulate the smoothed image.
Representing arbitrary canvas transformations. We consider all 2D transformations (including those without formula), but how do we represent them in computer? With respect the standard grid [m] x [n]. we use the transformed grid a[m]] x [n]) to represent a digitally. Thus, any canvas
transformation a is digitally represented by (=) a matrix a E [mm))x2 whose ith row is the 2D coordinate of the transformed ith grid point. We use the lexicographical order of 2D grid, e.g., with respect to [2] [3], the identify transformation id d' id = [[0,0] 0]. 00, 11, [0, 2), [1, 0], [1, 1], [1, 21]. Any MM(a ) E TR( i.e., a (vectorized)
transformed image Moa a d M(a))= == (M(ap),
digital image sampled from .M at the transformed grid a.
Representing color and canvas distortions. The color distortion De measures the color discrepancy between (id) and (a) up affine color transformation x. The canvas distortion Dy measures the distortion between the original grid id and the transformed grid a. Formally,
Dc M' a)) De( M(id), x(M'(a)) = (aM - ((id 1
(2)
| a2yl2 ll2
Dy(a)= 1 Dv(id, a)
max (F
A ( - / AQr,) := log
(3)
id: id,112
3

---

Under review as a conference paper at ICLR 2022
(0,0) (0.1) (0.2)
(1,0)
canvas distortion (at pair of neighboring edges)
convas grid
convas lottice
Figure 2: Canvas grid and its corresponding lattice. Local distortions are computed at every pair of neighboring edges. One example of neighboring edges is highlighted in red.
Here, BE comprises all pairs of neighboring edges in a canvas latrice (introduced below). Eq. is derived from the mathematical definition of distortion of a function by discretizing across the canvas lattice. This formula measures how far an arbitrary transformation is from being conformal, which is flexible for local isometries and scaling. Given a canvas grid [m]] x [12].. its corresponding canvas lattice is an undirected graph L = (V, E), with the set of vertices V = [m] x [m] and the set of edges obtained by connecting neighhoring vertices the lo sense: = {{ |/ dyllso = for v;; Uj E V).
We say two edges are neighbors if they form 45 angle (Figure 2
Computing topological distance by minimizing distortions. To minimize color and canvas distor- tions (2) and (3). we consider two dual views: minimizing De among low-distorted a 's or minimizing Dy among best-matching a's. We write the two views as the following two constrained optimization problems, together with their respective unconstrained equivalents: with - 04 and + 0+,
min. Dc(M, M a.)
s.t. Dv(a) S + min. Dv(a) + DDc(M, MM (4)
min. Dv(a) s.t. Do(M, X Mo a.x
min. Do(M, x M' oa) pDv(a) (5) a.x
We let the optima Di for (4) and Di- for (5) denote two versions of our desired topological distance that mimics innate human intuition. We call them De-distance and Dy-distance, respectively.
Transformation flow, To obtain both transformations and transformation processes that are human- like, we run (projected) gradient descent. The iterative gradient steps yield not only a transformation a" in the end but also transformation flow id = (9) + a1) - + a². The resulting sequence of transformed images M' = =M' o (0) + M' o ( - ... - M' a² = M (we omit x for simplicity) makes up an animation (Figure D.. which helps with human intuition on transforming M' to M. However, directly running (projected) gradient descent on (4) or (5) does not work, because it suffers from the curse of local minima, which we discuss and solve in the next section.
GRADIENT DESCENT AT HIGHER LEVELS OF ABSTRACTION
The canvas distortion Dy is invariant under variety of transformations (e.g., (a) 0 for any conformal a), which nicely mimics humans' flexible transformation options. But this also implies lots of local/global minima and other eritical points where the gradient is zero. How much the color distortion De fluctuates as a function of a depends on the images M, .M'. But in most cases, De also has lots of local/global minima, the majority of which represent unwanted "short transformations that make De + 0 but would break the rubber canvas or create holes in it. The curse of vanishing gradients can freeze gradient descent. To unfreeze it, we lift gradient descent to higher levels, mimicking once again humans' abstraction power, as our internal optimization system quite flexible in pursuing "gradient-deseen two abstraction techniques: chain of anchor lattices to make hierarchical abstractions of canvas transformations and a chain of color blurring to make hierarchical abstractions of image painting.
moves at multiple levels of abstraction. We design
Anchor grids and lattices. An anchor grid and its corresponding anchor lattice offer a simpler parameterization (i.c., an abstraction) of canvas transformations. Without such an abstraction, any transformed [m] x [n] grid a E R (mm)x2 consists of 2mn free parameters. So, the optimization problems (4) and 5) are 2mn dimensional, which is not only computationally inefficient for large images but also has too much room for vanishing gradient. We use a simpler a-parameterization that regularizes transformation, lowers distortion, and agrees with our intuition on rubber transformations.
Formally, anchor system (G, G) (M x,, N) uses two layers of grids: underlying grid G and an anchor grid & atop, satisfying M M, N and G C (ol))) Figure 3 shows
4

---

Under review as a conference paper at ICLR 2022
A. 1
C.
1g
1
A A
(1/2) (1/20//40 (1/99/2/90, 0//00000
W
Figure 3: An anchor system (A) and its transformation (B), (C) exemplifies a configuration of (G, pe)-slutiion path consisting of a chain of anchor grids/lattices and chain of blurring.
one example, where = (5) x [6] = (0....... ... 4) x (O...., 5) and G = 20, x 20,2,5) 5). Under an
anchor system, we can uniquely represent any grid point g E G via four anchors Ag. By, Cy. Dg E & via proportional interpolation, or more precisely, the following double convex combination
g = (1 - X1) Ag + (1 Bg + (1 - My)Cs + Dg
(6)
Here, A, D,C, can be uniquely selected as the smallest rectangle G's lattice containing g: the two weight parameters Ag. ' are computed based on relative position, e.g., as in Figure 3 The
relation between grid points and anchors can be summarized by a weight matrix W E R G|X|G| Its ith row stores weights for the ith grid point (say g in O and contains most four non-zero entries (i.e., coefficients in (6)) located at the columns corresponding to Ag: Bg. Cg. Dg. respectively.
Given an anchor system (G, any canvas transformation a - a E R|G|*2 under G and â e RIdIx2 under G. á is submatrix of a. which induces an equivalence relation on the set of all canvas transformations: a. are equivalent iff & B, and & abstracts the equivalence class {B e à}. Based on the maximum entropy principle (Jaynes 1957). a reasonable selection of a representative of this equivalence class is W because W à E { B &} and evenly distributes the transformed grid points. Figure 3 illustrates this type of even distribution, which agrees with human intuition on how rubber surface would naturally react when transforming forces are applied at anchors.
Using an anchor system in optimization problems and (5) adds very little to computing distortions and gradients: we reuse the computation with a = W & and perform only one additional chain-rule step Òa/ jā = W. By doing so, however, the number of optimization variables in (4) or 5 reduces
from |G| + 2 to |G] (e.g.. if G = [28] x [28] and & = (0, 27) x {0,27), the number reduces from 1570 to 10). It is important to note that using simpler anchor grid is not the same downsampling. it were, one would plug in a t å, but we plug in a + W &. In our case, image colors are still sampled from the underlying grid rather than downsampled from the anchor grid. So, using our anchor system is not information lossy while still benefiting from reduced optimization size. Running gradient descent (w.r.t. anchors) in abstracted optimization spaces effectively bypasses critical points.
Blurring. Another view to lifting gradient descent to high-level, abstracted optimization space, is to blur the image. Intuitively, blurring ignores low-level fluctuation, similar to how humans naturally abstract an image. Blurring helps remedy vanishing gradients and is done in our image smoothing process. The cutoff radius Pe in K in I controls the blurring extent: larger Pe means more blurred.
Guided gradient descent. Mixing the two abstraction techniques yields our guided gradient descent proceeding from higher- to lower-level abstractions. Given an anchor grid G' and cutoff radius pe- we denote the corresponding 4 and (5) by DC(G, and DV DVG Pc), respectively. For either, we solve for a (č,, pc) -oo-utin path, from coarser & and larger Pe finer & and smaller Pe- Let Gj. be k ls evenly distributed anchor grid and LA be its corresponding lattice. Figure CC shows a chain of anchor lattices {Lg' 1 i, and cutoff radii (17² Pco j==,,....... It is easy initially align two blurred blobs via small canvas adjustments, implying a small number of iterations to converge to Do R Dy R As we proceed along the solution path, the images restore more detail but the finer La helps manage that detail. subsequent solve step, which further alleviates the curse of vanishing gradients. Notably, even the starting in comprising only four corner anchors parameterizes large family of transformations containing all affine transformations. Finer anchor grids/lattices express more flexible transformations (including local, global, piecewise affine, and more), approaching human-level flexibility.
solution path, an earlier solution is used to warm start the
5

---

Under review as a conference paper at ICLR 2022
0
/
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
6
9
0.9
De No-Neessst- Neighbor
MNIST
Dy-Neerest- Neighbor
Neighbor
TexCCpps
28
SVN
SVM
DecisienTnee
Neura
RandomiFarest
Test
TectCaps
NeuralNicwork
RendomForest
Neighoor
Nearest-
163
2000
Di-Mtearest Neighber Neighber
MNIST
DedsionTree
200+
0.0
5
9 12 13 14 16 18 20
75
100
125
150
175
200
Number of training examples Iclass (N)
Minimum number training examples /olass needed reach 90%
Figure 4: MNIST (10 classes) in the tiny-data regime: first 1-20 training images per class and full test set (examples shown on top). For each model listed in the legend, we plot its test accuracy versus the training size N (bottom left) and also the smallest N needed to reach threshold of 90% accuracy (bottom right). Our model outperforms all other models for all N E {I, ......,,20 20}. requiring the fewest training examples (first four six per class) reach 90% accuracy.
A
B
C
D
E
G
H
K
L
R
P
R
S
X
b
C
A
Kiki
K
h
2
1.00
Do-Nearest-
EMNIST-EETTER
EMNIST-LETTERS
Dy-Nearest- Neighbor
0.75
Nearest Neightrr
TextCaps
Test securas
SVM
71
Rendomilorest
Decivion
Neura/Newoork
83
NeuralNetwork
Neighbor
Nearest
Dy Nearett Neighbor
De Noarest Neighber
DacisionTee
Number of training examples /class
Minimum number training examples needed 75%
Figure 5: EMNIST-letters (26 classes) in the tiny-data regime: first 1-20 training images per class and full test set (examples shown on top). Results are shown the same way as in Figure Our model outperforms all other models for all N {1...., 20), requiring the fewest training examples (first eleven or twelve per class) to reach 75% accuracy (due to increased difficulty in this dataset).
4 IMAGE CLASSIFICATION IN THE TINY DATA REGIME
We learned De- DD-ddistaece in the simplest nearest-neighhor method to classify grayscale images, named De- / -n--nnarett-neie human intuitive and interpretable. We show classification performances three standard benchmarks: the MNIST and EMNIST datasets of handwritten digits and letters restricted to the tiny-data regime, well the Omniglot challenge.
The whole process of metric learning and classification is
MNIST in the tiny-data regime. The original benchmark has 60k images for training and 10k for testing, spanning 10 classes. To evaluate how model performs in the tiny data regime, we train the model on the first N images per class from the original training set, test it on the full test set, and and
settrallnetoor--biiei record test accuracy versus N 3, ... We compare our model both of-the-art performance classical ML models. including TextCaps (Jayasundara et al. 2019) with state-of the small-data regime, SVM, nearest neighbor, etc. Classical ML is included show that nailing the tiny-data regime does not mean just using simple models. For stochastic models, we record mean and standard deviation from 5 independent runs. TexiCaps only runs when > 4 and sometimes record trimmed mean and standard deviation from 11 runs (where we trim the best two and worst four). We also copy results from other references that ran MNIST in similar tiny-data setting, including FSL that uses extra data for pre-training (whereas all our other selected models do not). These results are from the same traiiin-ttesting sizes but not the same data sets, and hence, considered indirect comparisons. We present all results in Figure
returns random guess (10%),

---

Under review as a conference paper at ICLR 2022
A.
train
M
9
B
D
ve
+)
4
4
26
3
R
Run
test
+
2
z
2
8
A
<
x
12
2
3
3
x+
P
train
i
3
&
3
C
?
a)
2
Av
t
P
+
2
Run 2
test
e
&
Y
Y2
+
%
e
G
b6
R
2
A
Y
B. Background set size
Siamese ConvNet
Simple ConvNet
Prototypical Net
BPL
Humans
Ours
RCN
VHE
(for pre-training)
6.75%
reduced
4.2%
23.2%
30.1%
original
3.3%
4.5%
7.3%
13.5%
13.7%
18.7%
augmented
8%
Figure 6: One-shot classification the Omniglot challenge (A) and its error-rate leaderboard (B). The red bounding area marks one out of 400 unit tasks, made up of test and 10 training images.
EMNIST-letters in the tiny-data regime. The original benchmark has 4.8k training images per class and 0.8k test images per class, spanning 26 classes of iuse-insensitive English letters. We keep the same experimental setting as in MNIST (except for TextCaps being more stable now: we do independent runs for each N and trim the best and the worst). Results shown in Figure EMNIST-letters is harder, not only with more classes but also more intrinsic ambiguities, c.g., an and an can look identical, so can an and an n when written carelessly. Hence, all models perform significantly worse than in MNIST. The intrinsic ambiguity, as well as more labeling errors, narrows our superiority over other models as training size increases. This is especially true for the state-of- the-art TextCaps model, catching up quickly in Figure 5 Being sensitive to ambiguities and outliers, however, is not deficiency our distortable canvas model, but a property nearest- -nighhrr, To improve, we may integrate our model with more robust classifiers, e.g., k-nearest-neighbor (k-NN) with proper voting. However, k-NN is not applicable in the tiny-data regime, not only because the training size can be small & but also there is little room hold out validation set for selecting k. An adaptive k-NN may be desired, with & remaining 1 in the tiny-data regime and becoming tunable level that affords held-out validation set. A related issue due to
when training size increases lacking validation data is about picking a proper model configuration. One may expect better results from any selected model in Figures 4 and S by attempting new configurations. Yet, is unclear what heuristics one may use. For TextCaps, we used its original implementation and configuration; for the rest, we used scikit-lean implementations with default configurations (except for small tweaks for the tiny-data regime e.g., sgural-network size and stronger regularization). By contrast, our distortable canvas model requires little to tune, other than the (G, pc)-solution path. Theoretically, the more gradual the path, the better. We picked (G, Pe) based on only image size (28 here) and runtime.
The Omniglot challenge for one-shot classification. The Omniglot dataset contains handwritten characters from 50 different alphabets, which include historical, present, and artificial scripts (e.g., Hebrew, Korean, "Futurama") and thus, are far more complex than MNIST digits and EMNIST letters. The characters are stored as both images and stroke movements. Unlike MNIST/EMNIST coming with large training data, the Omniglot challenge was specially designed for human-level concept learning from small data. Its one-shot classification task was benchmarked evaluate how humans and machines can learn from a single example. This benchmark contains 20 independent runs of way authin-alphaber classifications. The (2k: - 1)th and (2k th runs for l 1,..., 10 use the same set of 20 characters from single alphabet. Each run uses 40 images: one training and one test image per character. The unit task here is to predict for each test image, the character class to which it belongs (one of 20), based on the 20 training images. In total, there are 400 independent unit tasks across all 20 runs. Figure (6A shows a unit task (in red) and the first two runs in the benchmark, covering 1 alphabet, 20 characters, and 80 distinct images.
The Omniglot benchmark adopted the standard FSL setting, where it also provided a background set for pre-training. The original background set contains 964 character classes from 30 alphabets; a reduced background set was proposed later to make the classification task more challenging. We our neinearestt without any background
any stroke-movement information. In
7

---

Under review as a conference paper at ICLR 2022
WCSD
100
80
60
40
0
2
8
10
14
Number of clusters (k)
7
7
7
7
+
7
777
7
1
7
7
7
7
7
Clustering "7"s from MNIST
Clustering doodles of giraffes
Figure 7: Archetype generation via k-means-style clustering in our learned similarity space.
other words, in each unit task, we predict the test image based on one and only that training image character, and we read all images from their raw pixels. Shown in Figure68. 6.75% error rate) approaches human performance (4.5%) and outperforms all models in the Omniglot leaderboard (Lake al. 2019), except for BPL specially designed for the Omniglot challenge by making additional use of both the background set and the stroke-movement information.
model (with
UNSUPERVISED LEARNING: ARCHETYPE GENERATION
Beyond use with a classifier, our distortable canvas model can also perform k-means-style clustering, but within its non-Fuctidean settings (Cuturi & Doucet, 2014), a naive way explicitly computing distances and then running k-means is realistic. Learning distance in model requires solving an optimization problem, which is not as cheap as computing Euclidean distance. Further, computing a "centroid" non-Euclidean space requires solving another optimization problem (i.e., minimizing the sum of within-cluster distances) so an optimization problem of optimization problems which is not as simple as an arithmetic mean. Our idea of transformation flow between two images can be extended to multi floois--lwww among multiple images. Under this multi-flow extension, we do not explicitly compute pairwise distances, i.e., we do not solve the inner optimizations first. Instead, we solve the inner and outer optimizations at the same time, flattening the nested optimizations into a single one. Formally, given N images M MN, to cluster them into clusters, we solve
topological similarity landscape. As in other metric learning and
K
N Σ Dr(at) i=1
minimize
Σ Σ DolMe a;) subject to K=IEEC
CK
where Cg denotes the kth cluster, M2 o Tg denotes the kth centroid, and M, a, denotes the ith transformed image flowing its corresponding centroid together with all other N transformed images. One can check 7 is an extension of (4) where we omitted for simplicity. Solving 7 is similar k-means via alternating refinement: the assignment step assigns each transformed image M, to Ckk according to i* arg min.g=1..... (M o Ca. M, a.): the update step solves for one pradient-descent step given the C&S. Upon convergence, and Mj o Ti...., MR σ centroids. We treat the learned centroids as archetypes of the N images given at the beginning, which may be further deployed for education purposes.
obtain C+ Ci..... C clusters
We run the above procedure to generate archetypes of particular image class. Like common practices used in k-means, we try different k. For each k, we try multiple random starts and record the best within-cluster sum of distances (WCSD). We use the elbow method to pick good values for k. Figure]] 7 shows the WCSD-versus-k curve obtained by running our clustering method of "7's from MNIST. The curve indicates = or as a potential elbow point. The resulting two clusters of "7" agree with human intuition regarding two general ways of writing "7". depending whether there an extra stroke. The resulting three clusters further divide the cluster of "simpler 7s" based the angle of the transverse stroke. Moving away from more strict symbol systems towards
set of 16 images
8

---

Under review as a conference paper at ICLR 2022
/
2
3 C
"emoji" creation
distortable convas
classifiers beyond I-NN
A
B
real-world images (photorealistic)
"emojis" (of real-world images)
abstract images ("emojis" already)
future work
current model
future work
Figure 8: Generalization of our distortable canvas model for two major future directions (two ends).
free-form art, we try our model on doodle data where people were tasked with freely drawing abstract sketches of real-world objects. Figure allso shows four ways of doodling a giraffe, learned from the first 16 giraffes in Google's Quick Draw dataset (Jongejan et al. 2016). The four learned archetypes cleanly separate outline sketches, pose orientations, as well as focused views of the neck.
also
CONCLUSION, LIMITATION, AND FUTURE WORK
Focusing on its scientific contribution, this paper designs a numann-ntuitivv model from first principles to learn from few and only those few shots -nn particular one and only one data for pre-training. Based on nativism, our introduced distortable canvas effectively models humans' topological intuition and learns pranformationn-asee visual similarity akin to how humans naturally "distort" objects for comparison. This notion of similarity is formalized in our proposed optimization problem, which minimizes canvas and color distortions so to transform one object to another with minimal distortion. To remedy vanishing gradients and solve the optimization efficiently, we mimic human abstraction ability by chaining anchor lattices and image blurs into solution path. This yields our gradient descent method capable of optimizing at multiple levels of abstraction. Our model outputs not only transformations but also transformation flows that mimic human thought processes. We demonstrate initial empirical success in a first set of benchmarks focused on abstract visual tasks such as character and doodle recognition. By simply using 1-NN, we achieved state-of-the-a art results in the tiny-data and single-datum regime on MNIST/E.MNITT and achieved near- hharnh performance in the Omniglot challenge. Our model also enables k- E-meens-style clustering to generate human- interpretable archetypes. This paper is a first step towards a general theory of comprehensive, human-like framework for human-level performance in diverse applications. The current paper focuses on an initial scope, but opens the pathway to future generalizations as detailed below.
no extra
Consider two general types of images: 1) images of abstract patterns, or abstract images, e.g., those of symbols and doodles; 2) photorealistic images of real-world objects, or real-world images, e.g., those in CIFAR 10/100 (rizhevsky & Hinton| 2009). This paper focuses on the first type, handling abstract images only, by modeling humans' distortion- based intuition. For real-world images, it may be more efficient to first model cognitive simplification and then apply our current distortion model. It is reasonable assumption that humans have evolved to classify real-world images by first converting them into abstract icons, or "e (picture)+ mojj (character)" "s (e.g.. the emoji of a face, the outline of mountain, the shape of lake) and then comparing these simplifications. Following this, an efficient way to apply our method to real-world images is to follow this pipeline-preprocesiine them first into "emojis" and then comparing "emojis" using our distortion model. In fact, abstract images (e.g.. doodles of giraffes, hieroglyphics) are those that can be treated as "emojis" already. There are baselines to attempt first, e.g., smart edge detectors (Xie & Tu 2015), but the human visual system does more than edge detection. In the future, we will work on complete theory of icon or "emoji" creation mimicking human capacity and deal with real-world images, 3D objects, and more.
Although our model has shown dominant classification performance in the tiny-data regime of the presented benchmarks, its dominance diminishes when training size increases. This is due to 1-NN being 100% biased towards the "nearest neighbor" and hence fragile against noisy, erroneous, and ambiguous training examples. This suggests another future direction: our distortable canvas model may be designed jointly with a new, human-liie like classifier that introduces a small amount of learning into classification. The goal is to achieve state-of-the-as results on all training sizes, which is not merely about swapping in and out existing classifiers. We will not be using black-box models, but maintain model interpretability by modeling "the direct human way". where we learn (from a small training set) a particular function to be integrated into our distortion formulas. We will introduce such functions in future work and continue to improve them. Figure & summarizes the pipeline for generalizing our current distortable canvas model into the future directions sketched above.
9

---

Under review as a conference paper at ICLR 2022
REFERENCES
Amina Adadi and Mohammed Berrada. Peeking inside the black-box: A survey explainable artificial intelligence (XAI). IEEE Access, 6:52138-522 -26600, 2018.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond Euclidean data. IEEE Signal Process. Mag., 34(4): (184444:8444) 42, 2017.
François Chollet. On the measure of intelligence. arXiv:191.001540 [cs.AA 2019.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre van Schaik. EMNIST: An extension of MNIST to handwritten letters. In Proc. 2017 Int. Joint Conf. Neural Netw. (IJCNN), pp. 2921-2926. 2017.
Marco Cuturi and Arnaud Doucet. Fast computation of Wasserstein barycenters. In Proc. 31st Int. Conf. Mach. Learn. (ICML 2014), pp. 685-693, 2014.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-lcarning for fast adaptation of deep networks. Proc. 34th Int. Conf. Mach. Learn. (ICML 2017). pp. 1126-1135, 2017.
Kyle Hsu, Sergey Levine, and Chelsea Finn. Unsupervised learning via meta-lcarning In Proc. 7th Int. Conf. Learn. Represent. (ICLR 2019), 2019.
Vinoj Jayasundara, Sandaru Jayasekara, Hirunima Jayasekara, Jathushan Rajasegaran, Suranga Seneviratne, and Ranga Rodrigo. TextCaps: Handwritten character recognition with very small datasets. In Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), pp. 254-262, 2019.
Edwin Jaynes. Information theory and statistical mechanics. Phys. Rev., 106(4):620-630) 1957.
Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. The Quick, 2016.
Draw! Experiment. https: attps://puucccceawwttttsgggg.ee
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Proc. 2020 European Conf. Computer Vision (ECCV), pp. 491-507, 2020.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Advances Neural Inf. Process. Syst., 25:1097-1105, 2012.
Brenden M Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. In Proc. 33rd Annu. Conf. Cognitive Sci. Soc., volume 33, pp. 2568-2573, 2011.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, (1126266661332-133300 2015.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. The Omniglot challenge: A 3-year progress report. Current Opinion in Behavioral Sciences, 29:97-104, 2019.
Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86(1 1 10:111))22 2):2278-2324, 1998.
David G Lowe. Object recognition from local scale-invariant features. In Proc. 7th IEEE Int. Conf. Computer Vision, volume 2, pp. 1150-1157, 1999.
Kate T May. How children learn so much from little quickly: Laura Schulz at weweeechhhhhediiee learn- so- sonrnn-souchhh- from- so
TED2015.
1 litt serteee----uuclllllllare-------
2015.
Amiel Meiseles and Lior Rokach. Source model selection for deep learning in the time series domain. IEEE Access, 8:6190-6200, 2020.
10

---

Under review as a conference paper at ICLR 2022
Diego A Mesa, Justin Tantiongloc, Marcela Mendoza, Sanggyun Kim, and Todd P Coleman. A distributed framework for the construction of transport maps. Neural Computation, 31(4):613-652 -52. 2019. doi: 10.1162/neco 01172.
Sinno Jialin Pan and Qiang Yang. A survey transfer learning. IEEE Trans, Knowl, Data Eng., 22 (10):1345-139, 2009.
Elizabeth S Spelke and Katherine D Kinzler. Core knowledge. Developmental Sci., 10(1):00099))) 96. 2007.
Amos Storkey. When training and test sets are different: Characterizing learning transfer. Dataset Shift Mach. Learn., 30:3-28, 2009.
Cedric Villani. Optimal Transport: Old and New. Springer, 2009.
Yaqing Wang, Quanming Yao, James Kwok, and Lionel Ni. Generalizing from a few examples: A survey on few-shot learning. ACM Comput. Surv., 53(3):1-34, 2020.
Saining Xie and Zhuowen Tu. Holistically-nested edge detection. In Proc. 2015 IEEE Int. Conf. Computer Vision (ICCV), pp. 1395-1403, 2015.


Reviews:
[{"review_id": "qapSJV3_hAr", "reviewer": "Reviewer_CPtf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper on hand tackles the problem of data variety in a different way. As  the typical approach nowadays would be to use some kind of data augmentation, here vice-versa some kind of transformation are proposed to provide meaningful similarities. The overall idea is not new, however, the specific way how this is realized is. The overall approach make sense and is backed up by a clear mathematical definition. From this point of view the paper would be a valuable scientific contribution. On the downside, however, there are flaws in the description and interpretation of the results. Moreover, it is unclear if the proposed approach would also work for more realistic datasets: MNIST and EMNIST might be to simple. On the other hand, the Omniglot challenge seems to be a too complex tasks, that no existing approach can solve not even rudimentary. So using this dataset was probably not a good choice?\n\n\nFurther Comments:\n\n(1)  Using partially blue text is hampering the reading flow.\n\n(2) When describing the datasets in Sec.4 the correct abbreviation (given the real numbers) should be 60k and not 60K.\n\n(3) To make the experimental setup more clear, all methods that are used for training should be mentioned explicitly.\n\n(4) For readability, it might be better to split Fig.4 into two figures, one for each benchmark,\n\n(5) Even though the results presented in Fig.4 are clear, a better description of the experimental setup and some kind of discussion would be necessarily required.\n\n(6) For better understanding, it might be meaningful also to show examples for MNIST and EMNIST.\n\n(7) To allow for a fair evaluation and to give better insights, it might be meaningful to use the same learning approaches for all datasets.\n\n(8) The ocker background in Fig.6 does not look nice and removes the focus from the main content of the figure.\n\n(9) The archetype generation does not fully fit to the rest of the paper and is not described and evaluated sufficiently. In this way, it would be better to describe the other experiments more detailed and to shift this part to a separate publication.\n\n(10) In general, the paper is missing a conclusion and an interpretation of the results.\n\n(11) The discussion on related work is rather short, which can also be seen from the very short bibliography. A more thorough discussion would be necessary.\n\n(12) The bibliography needs to be seriously checked for consistency and correctness.", "summary_of_the_paper": "The paper on hand tackles the problem of learning from very few samples, which is of high relevance for many machine learning problems. To this end, an approach to model transformation-based topological similarity is introduced, allowing for covering many kinds of invariants in image data. The approach is demonstrated for well-known benchmark datasets for different classification models, demonstrating that in this way just using a small number of samples competitive classification results can be obtained.", "summary_of_the_review": "Overall, the paper comes up with an interesting approach to learn from very few samples. The approach is plausible, backed up by theory and seems to work for simple tasks. However, it seems to be questionable if this would also work for more realistic datasets. Moreover, the experimental section would need to be overworked. In this way, the paper would be an interesting contribution, but there are still a few open points hampering a publication as it is.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "aqCN9n9lbo-", "reviewer": "Reviewer_7LJy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strong points**: The authors connect ideas in a beautiful way. I liked how they connected ideas from how the human brain works to train their model. Well-structured writing. The idea seems novel and interesting. We optimize steps to generate the target image from our trainset and then pick the one that is closest. I really liked the idea. I also liked making the model work as a white box by using their method. Also finding ways to apply their method in higher levels.\n\n**Weak points**: Biggest drawback is lacking experiments on more challenging datasets, and unfortunately it is very important for me to see the results compared at least on very well-known datasets for few-shot learning.\n\n**Review**\nCurrently, I gave a score of 5 to the paper.\nThe main reason for my review is that there are other methods that leverage other data in the domain like (model-agnostic meta-learning[1]) and tested on a few samples on the classes that are never seen during the meta-learning phase. I understand the value of figuring out a way to remove this requirement. For example, unsupervised meta-learning (CACTUs[2]) achieves very good results without needing labelled data. It seems to me that this method has replaced the need for data with a good amount of domain knowledge (We know that characters could convert to each other by transformations, but what about cats and dogs? Another example is identity recognition from faces. If we have two images of two different people. In one picture the person laughs and in the other one, the person is wearing sunglasses. Which two are getting closer to each other?) For example, in figure 1, we know that we can apply these transformations from \"7\" to \"1\". However, the experiments did not convince me that this is always true. One interesting experiment that could be added is to apply this method to CelebA[2] face recognition dataset. Another baseline could be to apply this method to the Mini-Imagenet[1] dataset. These are well-known datasets in few-shot learning and studied alongside Omniglot. Results on these could convince the reader about the effectiveness of the method. \n\n**Questions**\n\nIn section 2, under \"Representative digital and smoothed images\" it is mentioned that \"This differs from\nGaussian blurring as we do not discretize kernels\". Can you specify what you mean? In practice, we must somehow use a discretize kernel on the computer. Can you describe the process in more detail?‌ Do you cut the kernel at some point?\n\n**Other comments to improve the paper**\n\n\nRelated work does not seem comprehensive to me. Only a few papers are cited and some of the statements are kind of misleading. For example, it is true that meta-learning needs data for a pre \"meta-learn\" step, but there is a nuanced difference. The meta-learning step is applied to different classes or tasks and then evaluated on a completely new task. As a result, I would try to expand this section.\n\n[1] Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning 2017\n\n[2] Hsu K, Levine S, Finn C. Unsupervised Learning via Meta-Learning. In International Conference on Learning Representations 2018\n\n", "summary_of_the_paper": "**Summary**\n\nThe authors propose a white-box model that uses a similarity score in a space that is constructed based on topological transformation. In other words, two data points are close to each other if you can transform them into each other. To achieve this, they first compute the topological distance with training data points while minimizing the distortions (this part was a little bit unclear to me. I would appreciate it if the authors provide more context or examples of this). They use an interesting idea of using gradient descent to find a sequence of transformation which is the main reason that contributes to having a white-box model. I found this part of the paper very interesting and novel. Finally, it is not possible to apply this method at the pixel level. So they propose using a chain of lattices to be able to apply their technique in higher abstractions. \n\n\n", "summary_of_the_review": "As a result, even though I liked the ideas in the paper a lot, I am going to vote for rejection unless I can see the experiments added to the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NkBaio_qxt8", "reviewer": "Reviewer_gUCx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1.\tThis approach is a white-box model and human-interpretable. \n2.\tThis paper is well written and easy to understand.\n3.\tThe major idea is novel and interesting. This is a good attempt to explore a solution for the few-shot learning without pretraining or with very limited training data. Since most existing FSL models heavily rely on a large number of training data.\n\nWeaknesses\n\n1.\tThe proposed approach is based on topological similarity. It seems that it only is suitable for images with simple topological structure, such as the character images. Maybe it is hardly used to classifier complex nature images since we need more information for natural image classification, not only use topological structure. \n2.\tThe authors did not provide the experimental comparison with enough training data, such as the whole training set in MNIST. The reviewer wonders about the upper performance of this approach with enough data.\n3.\tThe future applications of this approach may be limited, especially for the real-world tasks which require learning from a few labeled data, such as rare accident data in auto-drive and disease data for medical diagnose.\n", "summary_of_the_paper": "This paper proposes a novel white-box model for one or few-shot learning, which tries to simulate the human recognition ability for “distort” objects. The authors use transformation-based topological similarity to build the model and propose an anchor system with gradient descent to train their models. Extensive experiments on standard character recognition benchmarks demonstrate the proposed method outperforms the classical machine learning methods with very limited training data, such as less than 20.", "summary_of_the_review": "The main idea is novel and interesting, and the experimental results on limited training data demonstrate the superior performance of the proposed method. However, it may be not easy to apply this approach to other more complex tasks. In summary, the reviewer thinks this paper is marginally above the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-DXNGxtOoA", "reviewer": "Reviewer_MBhM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1. The proposed model is interesting and neat.\n2. The empirical results look promising. With very limited data and no pretraining, the proposed method archives significant results on MNIST, EMNIST and Omniglot.\n\nCons:\n1. The benchmark seems too toy-size. Even the most challenging Omniglot dataset has a clean background and somewhat simple visual appearance. In this sense, I am not convinced whether the method can still work well when the image becomes more complex. When more complicated distortion exists (e.g.), is the model still able to capture it through the transformation? I would expect at least some more complicated benchmark datasets to be evaluated, for example, CIFAR10/100.\n2. When the task is more challenging (e.g. EMNIST), the proposed method is caught up quickly by the TextCaps. On more complex datasets, modern deep learning models may catch up earlier, and with acceptable data size, even reach much better results that the proposed method cannot obtain.\n3. I am not very convinced about the setting. It has been studied that good pretraining can benefit the downstream tasks even in a few-shot setting [R1]. So, I am not sure when the proposed method can be applied. Can authors give me some examples in real world about training a classifier with very few training samples while no additional pretraining data is available?\n3. The paper lacks a conclusion.\n\nReference:\n\nR1: Kolesnikov, Alexander, et al. \"Big transfer (bit): General visual representation learning.\" Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16. Springer International Publishing, 2020.", "summary_of_the_paper": "This paper proposes a method that models the visual difference between images as a topological transformation for images. Using this model and computing the topological distance by minimizing the distortions between imagers, one can find neighbors that are conceptually similar to the input image. Evaluation results show that this simple method can achieve strong performance when only very few samples are available and no pretraining is allowed.", "summary_of_the_review": "This paper is interesting and has some promising results. But I have concern about its application in the real world and its robustness in more complex scenarios, which I hope the authors can address. Therefore, I incline to reject this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Meta Review of Learning from One and Only One Shot

The motivation of this work is to address the problem of learning from very few samples, which is of high relevance for many machine learning problems. The paper proposes an (interpretable) approach for one or few-shot learning, which tries to simulate the human recognition ability for “distort” objects. To achieve few-shot learning, they first model the topological distance with training data points while minimizing the distortions, to find neighbors that are conceptually similar to the input image. Their experimental results show that this simple method can achieve good performance when only very few samples are available and no pre-training is allowed.

All reviewers, including myself, agree that this paper is well motivated, nicely written, and appreciated how they connected ideas from neuro-psychology to their ML model, and the novelty is recognized. But there are issues raised by reviewers with the paper that prevent it from meeting the bar for me to recommend it for acceptance at ICLR 2022.

The main issues raised by all reviewers is that the proposed method is only experimentally verified on simple datasets such as MNIST, EMNIST and Omniglot (and to some extent, Quick, Draw!). The authors (to their credit) in the rebuttal noted that the narrative of the paper is to focus on abstract images, and the purpose is more from a scientific investigation perspective (rather than proposing an algorithm that is immediately useful for ML practitioners), and this is a fair point. However, I do believe the issue here is beyond the simple criticism of "it works on MNIST, how about ImageNet?" as I think some reviewers genuinely think there are fundamental aspects of the approach that might prevent it from scaling (as planned in future work, even by the authors in the last section). For instance, as gUCx noted:

1) The proposed approach is based on topological similarity. It seems that it only is suitable for images with simple topological structure, such as the character images. Maybe it is hardly used to classifier complex nature images since we need more information for natural image classification, not only use topological structure.

2) The authors did not provide the experimental comparison with enough training data, such as the whole training set in MNIST. The reviewer wonders about the upper performance of this approach with enough data.

I tend to agree with these points. Non-topological similarity can be displayed in abstract images / datasets. Even in "abstract" images, the paper should describe limitations of the approach, and whether it breaks down (like in the "abstract" Quick, Draw! dataset, there are different types of distinct "yoga" poses in the yoga class. And likewise, in the cat or pig class, there are animals with only the heads, and animals with the head and the full body). Conveniently, Quick, Draw! had not been used in any of their classification experiments [1], and only for a simple clustering example.

And for the other points, reporting the terminal MNIST performance will be useful, even if it doesn't look good, so the readers have an idea of the limitations of the approach, where it is good, where it is not, and what needs to be improved. I would love to see improvements (either in the writing or in the experiments) in future work where the paper can effectively convince the readers that the direction has the promise of being able to scale to "real" or "complex" images. (Perhaps even performing the approach on the output of a pre-trained self-supervised autoencoder on ImageNet, as a method to get "abstract" versions of real photos, like a parallel of the giraffe experiment, though this may distract from the narrative of no-pre-training). All in all, I don't want to discourage the authors as we are all excited about the direction of this work. I hope to see an updated version of this work published in a future venue, good luck!

[1] https://www.kaggle.com/c/quickdraw-doodle-recognition
Decision: Reject