Reaching Nirvana: Maximizing the Margin in Both Euclidean and Angular Spaces for Deep Neural Network Classification
Authors: Hakan Cevikalp
HASAN Saribas
Venue: NeurIPS 2022 Submitted (2022)
Keywords: open set recognition
deep neural network classifier
margin maximization

Abstract: The classification loss functions used in deep neural network classifiers can be grouped into two categories based on maximizing the margin in either Euclidean or angular spaces. Euclidean distances between sample vectors are used during classification for the methods maximizing the margin in Euclidean spaces whereas the Cosine similarity distance is used during the testing stage for the methods maximizing margin in the angular spaces. This paper introduces a novel classification loss that maximizes the margin in both the Euclidean and angular spaces at the same time. This way, the Euclidean and Cosine distances will produce similar and consistent results and complement each other, which will in turn improve the accuracies. The proposed loss function enforces the samples of classes to cluster around the centers that represent them. The centers approximating classes are chosen from the boundary of a hypersphere, and the pairwise distances between class centers are always equivalent. This restriction corresponds to choosing centers from the vertices of a regular simplex. There is not any hyperparameter that must be set by the user in the proposed loss function, therefore the use of the proposed method is extremely easy for classical classification problems. Moreover, since the class samples are compactly clustered around their corresponding means, the proposed classifier is also very suitable for open set recognition problems where test samples can come from the unknown classes that are not seen in the training phase. Experimental studies show that the proposed method achieves the state-of-the-art accuracies on open set recognition despite its simplicity.

Reaching Nirvana: Maximizing the Margin in Both Euclidean and Angular Spaces for Deep Neural Network Classification
Anonymous Author(s)
Affiliation
Address
email
Abstract
The classification loss functions used in deep neural network classifiers be grouped into two categories based on maximizing the margin in either Euclidean angular spaces. Euclidean distances between sample vectors used during classification for the methods maximizing the margin in Euclidean spaces whereas the Cosine similarity distance used during the testing stage for the methods max- imizing margin in the angular spaces. This paper introduces a novel classification loss that maximizes the margin in both the Fuclidean and angular spaces the same time. This way, the Euclidean and Cosine distances will produce similar and consistent results and complement each other, which will in turn improve the accuracies. The proposed loss function enforces the samples of classes to cluster around the centers that represent them. The centers approximating classes are chosen from the boundary of hypersphere, and the pairwise distances between class centers are always equivalent. This restriction corresponds to choosing centers from the vertices of a regular simplex. There is not any hyperparameter that must set by the user the proposed loss function, therefore the use of the proposed method is extremely easy for classical classification problems. Moreover, since the class samples are compactly clustered around their corresponding means, the pro- posed classifier is also very suitable for open set recognition problems where test samples can come from the unknown classes that are not seen in the training phase. Experimental studies show that the proposed method achieves the state-of-the-art accuracies on open set recognition despite its simplicity.
2
4
6
10
11
12
13
14
15
17
19
21
Introduction
23 Deep neural network classifiers have been dominating many fields including computer vision by achieving state-of-the-art accuracies in many tasks such visual object, activity, face and scene classification. Therefore, new deep neural network architectures and different classification losses have been constantly developing. The softmax loss function is the most common function used 25
27 for classification in deep neural network classifiers. Although the softmax loss yields satisfactory accuracies for general object classification problems, its performance for discrimination of the 29 instances coming from the same class categories (e.g., face recognition) or open set recognition (a classification scenario that allows the test. samples come from the unknown classes) is not
31 satisfactory. The performance decrease is typically attributed to two factors: there is no mechanism
32 for enforcing large-margin between classes and the softmax does not attempt to minimize the within-
33 class scatter which is crucial for the success in open set recognition problems.
To improve the classification accuracies of the deep neural network classifiers, many researchers 35 focused on maximizing the margin between classes. The recent methods can be roughly divided into
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

36
two categories based on maximizing the margin in either Euclidean or angular spaces. The methods targeting margin maximization in the Euclidean spaces attempt to minimize the Euclidean distances among the samples coming from the same classes and maximize the distances among the samples coming from different classes. Euclidean distances are used during testing stage after the network is trained. In contrast, the methods that maximize the margin in the angular spaces use the cosine distances for classification.
37
38
39
40
41
42
To maximize the margin in Euclidean space, Wen et al. T combined the softmax loss function with the center loss for face recognition. Center loss reduces the within-class variations by minimizing the distances between the individual face class samples and their corresponding class centers. The resulting method significantly improves the accuracies over the method using softmax alone in the context of face recognition. A variant of the center loss called the contrastive center loss [3] minimizes the Fuclidean distances between the samples and their corresponding class centers and maximizes the distances between samples and the centers of the rival (non-corresponding) classes. Zhang et al. combined the range loss with the softmax loss to maximize the margin in the Euclidean
43
44
45
48
47
48
49
50 spaces. Wei et al. [5] combined softmax loss and center loss functions with the minimum margin loss where the minimum margin loss enforces all class center pairs to have a distance larger than a specified threshold. Deng et al. [6] introduced method using softmax loss function with the marginal
51
53 loss to create compact and well separated classes in Euclidean space. Cevikalp et al. 77] proposed
54 deep neural network based open set recognition method that returns compact class acceptance
55 regions for each known class. In this framework, hinge loss and polyhedral conic functions are used for the between-class separation. The methods using Contrastive loss minimize the Euclidean distance of the positive sample pairs and penalize the negative pairs that have a distance smaller than
56
57
58
given margin threshold. In a similar manner, X 9 TO 11 employ triplet loss function that used a positive sample, a negative sample and an anchor. An anchor is also a positive sample, thus the within-class compactness is achieved by minimizing the Euclidean distances between the anchor and positive samples whereas the distances between anchor and negative samples are maximized for between-class separation. Although methods using both contrastive and triplet loss functions return compact decision boundaries, they have limitations in the sense that the number of sample pairs or triplets grows quadratically (cubicly) compared to the total number of samples, which results in slow convergence and instability. A careful sampling/mining of data is required to avoid this problem. Overall, the majority of the methods maximizing margin in the Euclidean spaces have shortcomings in way that they are too complex since the user has to set many weighting and margin parameters. This is due to the fact that the main classification loss functions include many terms that needs to be properly weighted. Furthermore, many of these methods are not suitable for open set recognition problems since they do not return compact acceptance regions for classes.
59
80
61
62
63
64
85
86
67
68
69
70
71
The methods that enlarge the margin in the angular spaces typically revise the classical softmax loss functions to maximize the angular margins between rival classes, and almost all methods are especially proposed for face recognition. To this end, Liu et al. [1] proposed the SphereFace method which uses the angular softmax (A-softmax) loss that enables to learn angularly discriminative
72
73
74
75 features. Zhao et al. [14] proposed the RegularFace method in which A-softmax term is combined
76 with an exclusive regularization term to maximize the between-class separation. Wang et al. [15]
77 introduced the CosFace method which imposes an additive angular margin on the learned features. To
78 this end, they normalize both the features and the learned weight vectors to remove radial variations and then introduce an additive margin term, m, to maximize the decision margin in the angular space.
80 A similar method called ArcFace introduced in [66), where an additive angular margin is added to
81 the target angle to maximize the separation in angular space. Liu et al. [17] proposed AdaptiveFace
82 method that enables to adjust the margins for different classes adaptively. [18) introduced uniform
83 loss function to learn equidistributed representations for face recognition. We would like to point
84 out that almost all methods that maximize the margin in the angular space are proposed for face
85 recognition. As indicated in 77]. these methods work well for face recognition since face class
96 samples in specific classes can be approximated by using linear/affine spaces, and the similarities can be measured well by using the angles between sample vectors in such cases. Linear subspace 88 approximation will work as long as the number of the features is much larger than the number of
87
89 class specific samples which holds for many face recognition problems. However, for many general
30 classification problems, the training set size is much larger compared to the dimensionality of the
91 learned features and therefore these methods cannot be generalized the classification applications
92 other than face recognition. In addition to this problem, these methods are also complex since they
2

---

93 have many parameters that must be set by the user as in the methods that maximize the margin in the Euclidean spaces.
94
95
The methods that maximize the margin in Euclidean or angular spaces mentioned above have the shortcomings in the ways that the objective loss functions include many terms that need to be weighted, the class acceptance regions are not compact, or they need additional hard- mining algorithms. In this study, we propose simple yet effective method that does not have these limitations. Our proposed method maximizes the margin in both the Euclidean and angular spaces. To the best of our knowledge, our proposed method is the first method that maximizes the margin in both spaces. To accomplish this goal, we train a deep neural network that enforces the samples to gather in the vicinity of the class-specific centers that lie on the boundary of hypersphere. Each class is represented with a single center and the distances between the class centers are equivalent. This corresponds to selection of class centers from the vertices of a regular simplex inscribed in hypersphere. Both the Euclidean distances and angular distances between class centers are equivalent to each other.
Contributions:
96
97
98
99
100
101
102
103
104
105
106
107
Our proposed method has many advantages over other margin maximizing deep neural network classifiers. These advantages can be summarized follows:
109
The proposed loss function does not have any hyperparameter that must be fixed for classical classification problems, therefore it is extremely easy for the users. For open set recognition, the user has to set two parameters if the background class samples are used for learning.
110
111
112
The proposed method returns compact and interpretable acceptance regions for each class, thus it is very suitable for open set recognition problems.
113
114
The distances between the samples and their corresponding centers are minimized indepen- dently of each other, thus the proposed method also works well for unbalanced datasets.
115
116
In contrast, there is only one limitation of the proposed method: The dimension of the CNN features must be larger than or equal to the total number of classes minus 1. To overcome this limitation, we introduced Dimension Augmentation Module (DAM) as explained below.
117
118
2
Method
119
120
2.1
Motivation
In this study, we propose a simple yet effective deep neural network classifier that maximizes the margin in both Euclidean and angular spaces. To this end, we introduce a novel classification loss function that enforces the samples to compactly cluster around the class-specific centers that are selected from the outer boundaries of a hypersphere. The Euclidean distances and angles between the centers are equivalent. This is illustrated in Fig. [ In this figure, the centers representing the classes are denoted by the star symbols whereas the class samples are represented with circles having different colors based on the class memberships. As seen in the figure, all pair-wise distances between
122
123
124
125
128
127
128
the class centers are equivalent, and class centers are located on the boundary of a hypersphere. Moreover, if the hypersphere center is set to the origin, then the angles between the class centers are also same, and the lengths of the centers are equivalent, i.e, || u, (u is the length of the center vectors). After learning stage, if the class samples are compactly clustered around the centers representing them, we can classify the data samples based on the Euclidean or angular distances from the class centers. Both distances yield the same results if the hypersphere center is set to the origin.
129
130
131
132
133
134 At this point, the question of whether enforcing data samples to lie around the simplex vertices is appropriate or not comes to mind. In fact, high-dimensional spaces are quite different than the low dimensional spaces, and there are many studies showing that the data samples lie on the boundary of hypersphere when the feature dimensionality, d, is high and the number of samples, n,, is small. For example, Jimenez and Landgrebe [99 theoretically show that the high-dimensional spaces are mostly empty and data concentrate on the outside of a shell (on the outer boundary of hypersphere). They also show that as the number of dimensions increases, the shell increases its distance from the origin. More precisely, the data samples lie near the outer surface of a growing hypersphere in high-dimensional spaces. In a more recent study, Hall et al. explicitly [20] show that the data samples lie at the vertices of regular simplex high-dimensional spaces. These two studies are not contradictory and they support each other since we can always inscribe regular simplex in
135
138
137
138
139
140
141
142
143
3

---

Center of Class Samples
Class Samples
ID
2D
3D
In the proposed method, class samples are enforced to lie closer to the class-specific centers representing them, and the class centers are located on the boundary of a hypersphere. All the distances between the class centers are equivalent, thus there is no need to tune any margin term. The class centers form the vertices of regular simplex inscribed in a hypersphere. Therefore, to separate C' different classes, the dimensionality of the feature space must be at least C - 1. The figure on the left shows separation of classes in I-D space, the middle figure depicts the separation of 3 classes in 2-D space, and the figure on the right illustrates the separation of 4 classes in 3-D space. For all cases, the centers are chosen from a regular C simplex.
Figure I:
145
hypersphere seen in Fig. In addition to these studies, 21 22 show that the eigenvectors of the Laplacian matrices (the matrices computed by operating on similarity matrices in spectral clustering analysis) form simplex structure, and they use the vertices of resulting simplex for clustering of data samples. In other words, they prove that when the data samples are mapped to Laplacian eigenspace, they concentrate on the vertices of a simplex structure. These studies are also complementary to the studies showing that the high- nigh-dimensional data samples lie on the boundary of because, as proved 23 , NCuts (Normalized Cuts) 24 clustering
146
147
148
149
150
151
growing hypersphere. algorithm, which is presented as spectral relaxation of graph cut problem, maps the data samples onto an infinite-dimensional feature space. Therefore, these data samples naturally concentrate on the vertices of regular simplex due to the high-dimensionality of the feature space.
152
153
154
155
2.2
Maximizing Margin in Euclidean and Angular Spaces
In the proposed method, map the class samples to compactly cluster around the class centers chosen from the vertices of a regular simplex. All the pair-wise distances between the selected class centers equivalent. Assume that there C classes in our data set. In this case, first need to create a C-simplex (some researchers call it C - simplex considering the feature dimension, but
157
158
159
160
we will prefer C-simplex definition). The vertices of a regular simplex inscribed in a hypersphere with radius can be defined as follows:
(C - 1)-1/11, j=1.
Vy
Al + 101-1:
25isc.
162 where,
K 1+ VC
(C- 1C--)872--- C
(2)
163
Here. is an appropriate sized vector whose elements are all e, is the natural basis vector in which the j-th entry is I and all other entries are O. Such a C-simplex is in fact a C-dimensional polyhedron where the distances between the vertices are equivalent. It must be noted that the distances between the vertices do not change even if the simplex is rotated or translated. But, the dimension
184
165
186
187
of the feature space must be at least C' in order to define such a regular C simplex. Next, we must define the radius, u, of the hypersphere, This term is similar to the scaling parameter used in methods such as ArcFace [66,, CosFace [15] etc. that maximize the margin in angular spaces. As the dimension increases, it must also increase since the studies [19] show that the hypersphere whose outer shells include the data also grows as the dimension is increased. We set 64 as in ArcFace method. Then, we set the class centers that will represent the classes as,
188
169
170
171
8, uV j: j = 1,.., C.
173 The order of selection of centers does not matter since the distances among all centers are equivalent. Now, let consider that the deep neural network features of training samples given in the form

---

Dimension Augmentation Module (DAM)
Activation Function
Figure 2: The plug and play module that will be used for increasing feature dimension. It maps di-menssional feature vectors onto a much higher (C' 1)- dimensional space.
(fe, B1), I,.... ,...,, IR", yr {i} where L..,, C. Here, C' is the total number of known 176 classes, and we assume that the feature dimension is larger than or equal to C' 1, i.e., d 2 C In this case, the loss function of the proposed method can be written as,
L
f. Sy.
(4)
178
The loss function includes single term that aims to minimize the within-class variations by mini- mizing the distances between the samples and their corresponding class centers which are set the vertices of regular simplex. There is no need another loss term for the between-class separation since the selected centers have the maximum possible Euclidean and angular distances among them. As result, there is no hyperparameter that must be fixed, and the proposed method is extremely easy the users. Moreover, the data samples compactly cluster around their class centers, therefore proposed method returns compact acceptance regions for classes, which is crucial for the success of the open set recognition. We call the resulting methods as Deep Simplex Classifier (DSC).
179
181
185
185
186
2.3
Including Background Class for Open Set Recognition
In open set recognition problems, novel classes (ones not seen during training) may occur test time, and the goal is to classify the known class samples correctly while rejecting the unknown class samples (25). Earlier open recognition methods only used the known class samples during training. However, more recent studies [26] 27 28 revealed that using the background dataset that includes the samples that come from the classes that different from the known classes greatly improves the accuracies. Let us represent the deep neural network features of the background samples by f E IRd, k = I,... K. In order to incorporate the background samples, we add an additional loss term that pushes the background samples away from the known class centers follows:
188
189
190
192
193
194
K
C --
- Sy, 11 +
max (0. m + [f. - Sy, 112 [ff Sy. 12).
(5)
1=1
185
where m is the selected threshold, and is the weighting term. The second loss term enforces the distances between the known class samples and their corresponding class centers be smaller than the distances between the background class samples and the known class centers by at least selected margin, m. In contrast our first proposed loss function, this loss function includes two terms that must be set by the users. But, this is necessary only if we use the background class samples.
196
197
198
2.4
Dimension Augmentation Module (DAM)
201
The major limitation of the proposed method the restriction that the dimension of the feature space must be larger than equal to C' 1, i.e., d 2 C 1. The typical feature dimension size returned
203 by the classical deep neural network classifiers 2048 or 4096. this case, the number of classes in training set cannot exceed 2049 4097. However, the number of classes can be larger than these
5

---

(a)
(b)
(c)
Figure 3: Learned feature representations of image samples: (a) the embeddings returned by the
proposed method trained with the default loss function given in (4), (b) the embeddings returned by the proposed method trained with the hinge loss, (c) the embeddings returned by the proposed method trained with the softmax loss function.
values for some classification tasks, and we cannot use the proposed method in such cases. There are basically two procedures to solve this problem. As a first solution, we can use a method similar to
206
29) that returns more centers where the distances between centers are approximately equivalent. In this case, the number of centers is increased to 2d + 4 for d dimmmnnsional feature spaces. As a second and a more complete solution, we introduce module called Dimension Augmentation Module (DAM) that increases the feature dimension size to any desired value. The module is visualized in Fig. and it includes two fully connected layers supported with activation functions. The first fully connected layer maps the d dimmeensional feature space onto a higher C dimensional space. Then,
208
210
212
apply ReLU (Rectified Linear Unit) activation functions followed by the second fully connected layer. This similar to kernel mapping idea used in kernel methods [30 [1] in the spirit with the
214
215
exception that we explicitly map the data higher dimensional feature space as in 32 33
Experiments
216
3.1
Illustrations and Ablation Studies
218
Here, we first conducted some experiments to visualize the embedding spaces returned by the various loss functions using the vertices of the regular simplex. For this illustration experiment,
219
designed
220
a deep neural network where the output of the last hidden layer is set to 2 for visualizing the learned features. As training data, selected 3 classes from the Cifar-10 dataset. We would like to point
222
that we can use different loss functions in addition to our default loss function given in (4) once we determine the vertices of the simplex that will represent the classes. To this end, we used other
223
224
loss functions: The first one is the hinge loss that minimizes the distances between the samples and their corresponding class center if the distance is larger than selected threshold,
225
Lhinge 1 - max (o, |If. Sy: 1y m
(6)
This loss function does not minimize the distances between the samples and their corresponding centers the distances are already smaller than the selected threshold, m. This way class-specific samples are collected in hypersphere with radius, For the second loss function, used the variant of the softmax loss function where the weights are fixed to the simplex vertices as in,
227
229
2 fitbbb
Leo fmaa
Σ log
2 Eitb,
For the softmax loss, we fix the classifier weights to the pre-defined class centers and we only update features of the samples by using back- sacckproppggtion. We set the hypersphere radius to, u = 5, since this is simple dataset.
231
235
The embeddings returned by the deep neural networks using different loss functions are plotted in Fig.
The first figure on the left is obtained by our default loss function that does not need any parameter selection. All data samples are compactly clustered around their class means as expected. The second loss function using the hinge loss returns spherical distributions based the selected
235

---

237 margin, m, and the classes are still separable by margin. In contrast, when the softmax is used with
238 the simplex vertices, the data samples are very close and they overlap since there is no margin among the classes. Therefore, our default loss function seems to be the best choice among all tested variants
240 since it does not need fixing any parameter and returns compact class regions.
sitplasc ashondlle
dog
frog
sapp
Bod anteeeblle
62
18
13
Figure 4: The distance matrix computed by using the centers of the testing classes. The four classes that are not used in training are closer to their semantically related classes in the learned embedding space.
241 We also conducted experiments to see if the proposed method returns meaningful feature embeddings where the semantically and visually similar classes lie close to each other in open set recognition settings. It should be noted that the semantic relationships are not preserved for the training classes since the Euclidean and angular distances between the class centers proposed method returns good CNN features, we expect the samples belonging to classes not used in training to lie closer to their semantically related training classes. To verify this, trained our proposed method by using 6 classes from the Cifar- 10 dataset: airplane, automobile, bird, cat, deer, and frog. Then, extracted the CNN features of all testing data coming from 10 classes by using the trained network. Then, we computed the average CNN feature vector of each class, and computed the distances between them. Fig. 4 illlusttates the computed distances between the centers. The distances between the classes used for training are similar and they change between 5.8 and 6.7. The four classes, the dog, horse, ship, and truck classes, that are not used for training are represented with red color in the figure. As seen in the figure, the dog class is closest to its semantically similar cat class, the truck class is closer its semantically similar automobile class, the horse class closest to the deer class, and the ship class is closer to the visually similar airplane class (since the backgrounds blue sky and sea- are mostly similar for these two classes). This clearly shows that the proposed method returns semantically meaningful embeddings.
243
equivalent. However, if the
245
247
249
250
251
252
253
254
255
256
257
258
3.2
Open Set Recognition Experiments
For open set recognition, we need to split the datasets into known and unknown classes. To this end, we used the common standard settings that are also applied for testing other recent open set recognition methods. The details of each dataset and its open set recognition setting are given below.
260
282
By following the standard protocol, random splitting of each dataset into known and unknown classes repeated 5 times, and the final accuracies are averages of the results obtained in each trial.
284
We compared our proposed method, Deep Simplex Classifier (DSC), to other offte-of-the-ard open set recognition methods including Softmax, OpenMax (25 C2AE 34 CAC 277, CPN [35]
265
266
OSRCI (36), CROSR [37, RPL [38]. Objecttosphere [9) and Generative-Disent Discriminative Feature Representations (GDFRs) 40 methods. We used the same network architecture used in [36] as our
267
268
backbone network for all datasets with the exception of Tiny ImageNt dataset, where we preferred deeper Resnet-50 architecture for this dataset. We started the training from completely random weights (without any fine-tuning). Therefore, our proposed method is directly comparable to the published results in (36] for majority of the tested datasets.
270
7

---

Table 1: AUC Scores (%) of open set recognition methods on tested datasets (n.r. stands for not reported).
Methods
Mnist
Cifarl0
SVHN
Cifar+10
Cifar+50
TinylmageNet
DSC (Ours)
99.6 + 0.1
93.8 0.3
95.3
99.1 0.2
98.4 = 0.3
82.5 1 1.8
Soltmax
97.8 1 0.2
0.6
n..
80.5 1: in.r.
57.7 1 n...
OpenMax
98.1 0.2
69.5 = 3.2
0.88
81.7 + n.J.
79.6 -: n..
57.6 +: n.r.
G- Max
98.4 0.1
67.5 = 3.5
89.6 0.6
82.7 + n.r.
81.9 I n.r.
58.0 + n...
C2AE
98.9 0.2
89.5 = 0.9
92.2 0.9 0.9
95.5 + 0.6
93.7 + 0.4
74.8 + 0.5
CAC
+ 0.5
0.7
87.7 +
87.0
76.0 1.5
CPN
99.0 0.2
82.8
92.6 0.6
88.1 n.r.
87.9 1.9tn.r. n.r.
63.9 n.r.
OSRCI
0.1
69.9 = 2.9
91.0 0.6
83.8 + n.J.
82.7 +
58.6 n.r.
CROSR
99.1 n.r.
88.3 =
89.9 n.r.
F.T.
91.2 n.J.
90.5 n.r. n.r.
58.9 4 n.r.
RPL
0.1
82.7 1.4
0.54
84.211.0
0.7
68.8 1.4
GDFRs
n.r.
83.1= 3.9
95.5
92.8
92.6
64.7 1.2
Objecttosphere
n.r.
94.21 n.r.
n.r.
1......
94.4 n.r.
75.5 n.r.
272
3.2.1
Datasets
By using the standard setting, Mnist, Cifar10, and SVHN datasets are split randomly into 6 known and unknown classes. used 80 Million Tiny Images dataset 41 as the background class.
Mnist, Cifar10, SVHN:
274
276
Cifar+10, Cifar+50: For Cifar+N experiments, we use 4 randomly selected classes from Cifar10 dataset for training, and N non-overlapping classes chosen from Cifar 100 dataset used unknown classes as in 33 27 [7] B8]. We used 80 Million Tiny Images dataset [41] as the background class. randomly selected 20 classes known classes and 180 classes as unknown classes by following the standard setting. We used 80 Million Tiny Images dataset [1 the background class.
278
Tiny ImageNet:: For TinylmageNet [42 experiments,
280
282
3.2.2
Results
283
For open set recognition, Area Under the ROC curve (AUC) scores are used for measuring the detection of performance of the unknown samples. In addition, also report the closed-set accuracy
285
for measuring the classification performance on known data by ignoring the unknown samples as in (these results given in Appendix). AUC scores are given in Table As seen in the table, our proposed method achieves the best accuracies on all datasets with the exception of Cifar 10 and SVHN datasets. The performance difference is very significant especially on Cifar+ 10, 50 and Imageee datasets.
35
36
287
290
3.3
Closed Set Recognition Experiments
3.3.1
Experiments on Moderate Sized Datasets
292
Here, conducted closed recognition experiments on moderate sized datasets. Our proposed method did not need DAM since the feature dimension is much larger than the number of classes in the training for these experiments. We compared our results the methods that maximize the
293
294
295
margin in Euclidean or angular spaces. We implemented the compared methods by using provided backbone for all tested
296
source codes by their authors, and we used the ResNet-18 18 architecture methods. Therefore, our results are directly comparable.
297
Table 2: Classification accuracies (%) on moderate sized datasets.
Methods
Mnist
Cifar-10
Cifar-100
DSC (Ours)
99.7
95.9
79.5
Softmax
99.4
94.4
75.3
Center Loss
99.7
94.2
76. 1
ArcFace
99.7
94.8
75.7
CosFace
99.7
95.0
75.8
Spherefacce
99.7
94.7
75.1

---

298
Classification accuracies are given in Table 2 For Mnist datasets, majority of the tested methods yield the same aceuracy, but our proposed DSC method outperforms all tested methods on the Cifar- 10 and Cifar- 100 datasets. The performance difference is significant especially on the Cifar- 100 dataset. These results verify the superiority of the margin maximization in both Fuclidean and angular spaces. Achieving the best accuracies is encouraging, because our proposed method is very simple and does not need any parameter tuning, yet it outperforms more complex methods.
299
301
302
303
304
3.3.2
Experiments on Large-Scale Datasets
For all face verification tests, used the same network trained on large-scale face dataset by follow- ing the standard setting. To this end, we trained the proposed classifier on MS MSIMV2 dataset [16), which is cleaned version of MS-Celeb- 1M dataset 44]. This dataset includes approximately 85.7K individuals. We removed the classes including less than 100 samples, which left us approximately 18. 6K individuals for training. The number of classes is much larger than the feature dimension, = 2048, thus we used DAM to increase the CNN feature dimension. The ResNet- 101 architecture used backbone. Once the network is trained, we used the resulting architecture extract deep CNN features of the face images coming from the test datasets.
306
308
310
312
As test datasets, we used Labeled Faces in the Wild (LFW) [45) Cross-Age LFW (CALFW) [46]. Cross-Pose LFW (CPLFW) [47], Celebrities in Frontal-Profile data set (CFP-FP) 48 and AgeDB
314
48). We evaluated the proposed methods by following the standard protocol of unrestricted with labeled outside data [45] and report the results by using 6,000 pair testing images on LFW, CALFW,
316
CPLFW, and AgeDB. However, 7,000 pairs of testing images are used for CFP-FP by following the standard setting. The results are given in Table 3 As seen in the results, the proposed method using
318
DAM outperforms the classifiers using softmax and Center loss, but accuracies are lower than the
320
recent state-of-the-an methods. These results indicate that the DAM solves the dimension problem partially, but it must be revised for obtaining better accuracies.
Table 3: Verification rates (%) different datasets.
Method
LFW
CALFW
CPLFW
CFP
AgeDB
DSC
99.6
91.3
90.3
94.3
96.0
VGGFace2
99.4
90.6
84.0
Center Loss
99.3
85.5
77.5
ArcFace (ResNet-101)
99.8
95.5
92.1 1
95.6
CosFace
99.7
93.3
92.1
97.7
SphereFace
99.4
93.3
92.1
94.4
97.7
Summary and Conclusion
322
323
In this paper, we proposed a simple and effective deep neural network classifier that maximizes the margin in both the Euclidean and angular spaces. The proposed method returns embeddings where the class-specific samples lie in the vicinity of the class centers chosen from the vertices of a regular simplex. The proposed method is very simple in the sense that there is no parameter that must be fixed for classical closed set recognition settings. Despite its simplicity, the proposed method achieves the state-of-the-ar accuracies on open set recognition problems since the samples of unknown classes
324
325
326
327
328
329
are easily rejected by using the distances from the class- spessssc centers. Moreover, our proposed method also outperformed other state-of-the-an classification methods on closed set recognition setting when moderate sized datasets are used. The proposed method has limitation regarding learning in large-scale datasets. We introduced DAM in order to solve this problem. Although DAM
330
331
332
partially solved the existing problem, we could not get state-of-the-an accuracies on large-scale
334
face recognition problems. As a future work, we are planning to improve DAM by changing its architecture and activation functions.

---

References
336
337 [1] Y. Wen, K. Zhang. Z. Li, and Y. Qiao. A comprehensive study on center loss for deep face recognition. International Journal of Computer Vision, 127:668- 683, 2019.
339
[2] Y. Wen, K. Zhang, Z. Li, and Y. Qiao. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision, 2016.
340
[3] C. Qi and F. Su. Contrastive-center loss for deep neural networks. In IEEE International Conference on Image Processing (ICIP), 2017.
342
343
[4] X. Zhang. Z. Fang. Y. Wen, Z. Li, and Y. Qiao. Range loss for deep face recognition with long-tailed training data. In International Conference on Computer Vision, 2017.
[5] X. Wei, H. Wang, B. Scotney, and H. Wan. Minimum margin loss for deep face recognition. Puttern Recognition, 97:1-9, 2020.
346
347
[6] J. Deng, Y. Zhou, and S. Zafeiriou. Marginal loss for deep face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017.
348
349
[7] H. Cevikalp, B. Uzun, O. Kopuklu, and G. Ozturk. Deep compact polyhedral conic classifier for open and closed set recognition. Pattern Recognition, 1991100000:1-1222200 2021.
350
[8] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
352
[9] E. Hoffer and N. Ailon. Deep metric learning using triplet network. In International Conference on Learning and Recognition (ICLR) Workshops, 2015.
355
356
[10] K. Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Neural Information Processing Systems (NIPS), 2016.
358
[1]] S. K. Roy, M. Harandi, R. Nock, and R. Hartley. Siamese networks: The tale of two manifolds. In International Conference on Computer Vision, 2019.
359
360
[12] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and Song. Sphereface: Deep hypersphere embedding for face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
361
362
[13] W. Liu, Y. Wen, Z. Yu, and M. Yang. Large-margin softmax loss for convolutional neural networks. In International Conference on Machine Learning (ICML), 2016.
364
365
[14] K. Zhao, J. Xu, and M.-M. Cheng. Regularface: Deep face recognition via exclusive regular-
366
ization. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
367
388
[15] H. Wang, Y. Wang Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu. Cosface: Large margin cosine loss for deep face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2018
389
370
371
[16] J. Deng, J. Guo, N. Xue, and S. Zafeirio.. Areface: Additive angular margin loss for deep face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
372
373
[17] Hao Liu, Xiangyu Zhu, Zhen Lei, and Stan Z. Li. Adaptiveface: Adaptive margin and sampling for face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
375
376
377
[18] Yueqi Duan, Jiwen Lu, and Jie Zhou. Uniformface: Learning deep equidistributed represen- tations for face recognition. In IEEE Society Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
378
379
[19] L. O. Jimenez and D. A. Landgrebe. Supervised classification in high dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEFE Transactions on Systems, Man, and Cybernetics C: Applications and Reviews, 28(1):39-54, 1998.
381
10

---

383
[20] P. Hall, J. S. Marron, and A. Neeman. Geometric representation of high dimension, low sample size data. Journal of the Royal Statistical Society Series B, 67:427-444, 2005.
384
385
[21] Kumar, L. Niveditha, and B. Ravindran. Spectral clustering as mapping to a simplex. In ICML Workshops, 2013.
386
387
[22]
M. Weber. Clustering by using a simplex structure. Technical report, Sonrad-Zuse-Zemna fur informationsteccniik Berlin, 2003.
388
389
[3
Ali Rahimi and Benjamin Recht. Clustering with normalized cuts is clustering with a hyperplane. In Statistical Learning in Computer Vision, 2004.
390
381
[24] J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:888 905, 2000.
392
393
[25] W. J. Scheirer, A. Rocha, A. Sapkota, and T. E. Boult. Towards open set recognition. IEEE
394
Transactions on Pattern Analysis and Machine Intelligence, 35:1757-1772, 2013.
385
[26] A. R. Dhamija, M. Gunther, and T. E. Boult, Reducing network agnostophobia. In Neural
396
Information Processing Systems (NeurIPS), 2018.
387
[27] D. Miller, N. Sunderhauf, M. Milford, and Dayoub. Class anchor clustering: A loss for distance-based open set recognition. In WACV, 2021.
398
389
[28] Chuanxing Geng. Sheng-Jun Huang, and Songcan Chen. Recent advances in open set recogni-
400
tion: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10))3614- 3631, 2021.
401
402
[29] M. Balko, A. Por, M. Scheucher, K. Swanepoel, and P. Valtr. Almost-equidistant sets. Graphs and Combinatorics, 36:729-754, 2020.
404
[30] C. Cortes and V. Vapnik. Support vector networks. Machine Learning, 20:273-297, 1995.
[31] S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K.R. Mullers. Fisher discriminant analysis with kernels. Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pages 41 41-48, 1999.
406
[32] A. Vedaldi and A. Zisserman. Efficient additive kernels via explicit feature maps. IEEE Transactonns on Pattern Analysis and Machine Intelligence, 11:480-492. 2012.
409
410
[33] A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NIPS, 2007.
411
[34]
Poojan Oza and Vishal M. Patel. C2ae: Class conditioned auto-encoder for open-set recognition. In CVPR, 2019.
[35] Hong-Ming Yang, Xu- Yao Zhang, Fei Yin, Qing Yang, and Cheng-Lin Liu. Convolutional prototype network for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-1, 2020.
414
[36] Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. Open set learning with counterfactual images. In ECCV, 2018.
417
418
[37] R. Yoshihashi, W. Shao, R. Kawakami, S. You, M. lida, and T. Naemura. Classification- reconstruction learning for open-set recognition. In CVPR, 2019,
[38] G. Chen, L. Qiao. Y. Shi, P. Peng, J. Li, T. Huang, S. Pu, and Y. Tian. Learning open set network with discriminative reciprocal points. In ECCV, 2020.
421
422
[39]
Abhijit Bendale and Terrance Boult. Towards open set deep networks. In CVPR, 2016.
423
[40]
Perera, V. I. Morariu, R. Jain, V. Manjunatha, C. Wigington, Ordonez, and V. M. Patel. Generative-discriinntti feature representations for open-set recognition. In CVPR, 2020.
[41] Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(11):1958-119700 2008.
426

---

428
[42] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy,
429
A. Khosla, and M. Bernstein. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 11:201-252, 2015.
431
[43] K. He, X. Zhang. S. Ren, and Sun. Deep residual learning for image recognition. In CVPR. 2016.
433
[44] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In European conference computer vision, pages 87-102. Springer, 2016.
435
[45] Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Milles Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Workshop on faces in RRal-Life' Images: detection, alignment, and recognition, 2008
437
439
[46] Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments. CoRR, htss/1708.01999 2017.
441
[47] Tianyue Zheng and Weihong Deng. Cross-pose Ifw: database for studying cross-pose face recognition in unconstrained environments. Technical report, Beijing University of Posts and Telccommunications, 2018.
443
444
[48] Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng. Irene Kotsia, and Stefanos Zafciriou. Agedb: The first manually collected, in-the-wild age database. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1997-2005, 2017.
445
447
Checklist
448
449
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes]. [No], or [N/A]. You are strongly encouraged to include justification your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
450
451
452
453
Did you include the license to the code and datasets? [Yes] See Section ??.
454
Did you include the license to the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
457
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
458
459
460
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's s contributions and scope? [Yes] We added a Contributions subsection the Introduction describing our contributions and scope.
462
463
(b) Did you deseribe the limitations of your work? [Yes] Limitations of the proposed method are discussed in Section 2. titled "Dimensioo Augmentation Module (DAM)".
465
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
467
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] We ensured that our paper conforms to ethics.
468
469
2. If you are including theoretical results...
470
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
12

---

473
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as URL)? [Yes] We did not include source codes as supplementary material, but both our codes and trained models will be shared in our GitHub page.
474
475
476
477
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We followed the common settings in the literature for data splits and briefly described them. Appendix, we explained hyperparameter selection process for the used architectures. We do not need any parameter fixing for classical classification problems, but we need two parameters for open set recognition. We reported the used parameter values.
478
479
481
483
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No] Some experiments are conducted several times and we reported the means and standard deviations for these. But for the remaining datasets, the test sets are fixed, thus experiments are run only once.
484
486
487
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No]
489
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We used some well-known CNN architectures and cited the corresponding papers.
491
492
(b) Did you mention the license of the assets? [N/A]
493
(c) Did you include any new assets either in the supplemental material or as a URL? [No]
494
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
496
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
497
5. If you used crowdsoureing conducted research with human subjects...
499
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
502
503
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
A
Appendix
506
Here, we first explain the implementation details of the proposed deep neural network classifier, and give the parameters used for the utilized deep neural network classifier architecture. Then, we reported the closed-set set accuracies of tested methods on open set recognition datasets.
507
508
A.1
Implementation Details
510
For open set recognition, we used the same network architecture used in [36] as our backbone network for all datasets with the exception of Imagene dataset, where we preferred deeper Resnet-50 architecture for this dataset. The learning rate is set to 0. For open set recognition experiments, we 2xbatch 7 and m 2/2, where is the hypersphere radius.
512
set x
514
We do not need these parameters for closed set recognition. For closed-set recognition experiments, used the ResNet-18 18 architecture backbone for moderate sized datasets, and the 101 architecture is used for large-scale face recognition dataset. For updating network weights, we used Adam optimization strategy for large-scale face recognition whereas SGD (stochastic gradient descent) is used for moderate size datasets. The learning rate is set to 10-3 for face recognition and
515
518
517
518
519
to 0.5 for moderate sized datasets.
A.2
Closed-Set Accuracies Open Set Recognition Datasets
521
Closed-set accuracies of the open-set recognition methods are given in Table Our proposed method also obtains the best closed-set accuracies among the tested methods with the exception of SVHN
13

---

523 dataset. This clearly shows that the proposed method is very successful both at the rejection of the unknown samples and classification of the known samples correctly.
Table 4: Closed-Set accuracies (%) of open set recognition methods on tested datasets.
Methods
Mnist
Cifar10
SVHN
Cifar+10
Cifar+50
Tiny TinyImagNNet
DSC (Ours)
99.8 + 0.1
96.1 = 1.4
96.5 + 0.3
97.6 0.5
97.9
0.5
83.3 + 2.2
Softmax
99.5 0.2
80.1 = 3.2
94.7 - 0.6
OpenMax
99.5 0.2
80.1 3.2
94.7 +
n.J.
n.r.
n.r.
G G-OpenMax
99.6 = 0.1
81.6 3.5
94.8 0.8
7...
Y0.7.
R.7.
CPN
99.7 0.1
92.9 + 1.2
96.7 + 0.4
T1.7.
OSRCI
99.6 0.1
82.1 - 2.9
95.1 1 0.6
T..
Y....
T....
CROSR
99.2 = 0.1
93.0
94.5 0.5
14


Reviews:
[{"review_id": "rja5nFr-Ol7", "reviewer": "Reviewer_DKz4", "summary": "This paper proposed a method to combine the margin maximization in both Euclidean and angular spaces. More specifically, the method maps the labels to the vertices of a regular simplex as new labels.\n\nSection 2: \n1) explains the motivation of the method, i.e. the two distances are the same once the data lie on the boundary of a hyper-sphere. \n2) more details about the method, including the vertices of a regular simplex, the case that uses the background class samples and the Dimension Augmentation Module(which is used for increasing the dimension, since the method requires the dimension of data is nearly the data size).\n\nSection 3 and Appendix provide more details about the experiments:\n1) feature representations learned by different loss function.\n2) results of Open Set Recognition and Closed Set Recognition.\n", "questions": "1. Is there more explains about the hyperparameters such as, how there were chosen? Especially the u, which is the size of the hypersphere(it seems the u doesn't affect the classification?) . In contrast, is the m very sensitive? And what's the initialization?(what do the paper means by 'completely random'). \n2. Does DAM work better than the method in [1], or just the method in [1] is not suitable?\n\n[1]Almost-equidistant sets.", "limitations": "No. The authors partly mentioned the limitations in the experiments (Section 3, especially 3.3.2) and the summary (Section 4). The limitation focus on the large-scale problem and the DAM.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*The proposed method is interesting(also the DAM is interesting) and simple, and the simplicity brings several advantages (few hyper-parameters, proper acceptance regions, suit for unbalanced datasets), thus can be easily applied for classification problems. The experiments show that the method achieves good accuracy in several cases (especially the Open Set Recognition).\n*The paper is well organized(especially the Section 2, which explains the main idea of the paper).\n\n*Although the paper explains the benefits of the setting, it may need more explains or experiments or theorems to support the method. It's hard to convince me that the method works well now.\n*The DAM seems need large size when the C is large.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_PIdOxESwc2", "reviewer": "Reviewer_a4BR", "summary": "The paper studies an interesting problem of fixing a set of equi-distance classifiers and then learning features to the corresponding class center (i.e., a vertice in a simplex). The specific form of the loss function is simple and aims to minimize the Euclidean distance between learned feature and the class center (on a hypersphere). This will automatically guarantee maximum separation of class centers and also lead to discriminative feature representation. The paper uses this method in both closed-set and open-set recognition, and shows some improvements.", "questions": "- I am wondering whether the authors only uses the proposed Euclidean loss in closed-set recognition (essentially center loss on a simplex)? It is a bit difficult to believe that it can actually outperform Softmax-based margin losses (say ArcFace). Using only Euclidean losses is actually ineffective in classification problems in my experience (it may work for clustering problems like face verification).\n\n-  In the CIFAR-10 experiment, what is feature dimension in the proposed method? How the simplex is constructed if you are using the same feature dimension as the others? Are the authors using 9-dimension features?", "limitations": "n/a", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I general, I think this is an interesting idea by taking advantages of the simplex and using it as a set of classifiers to guide the feature learning.\n\nStrengths:\n\n- The core idea is interesting. Connecting the simplex and maximum separated classifiers is interesting. Using it as an inductive bias to guide the feature learning is intuitive and seems to make lot of senses.\n\n- The paper also considers the weakness of this method due to the limitation in dimension. The propose dimension augmentaion module aims to address this limitation (however, it inevitably introduces additional trainable parameters and it might not be a fair comparison to the other baselines any more).\n\n- The paper considers both closed-set and open-set recognition and show some improvements over existing baselines. \n\nWeaknesses:\n\n- As the authors have mentioned, the biggest weakness of this method is its restriction to feature dimension. Although the dimension augmentation module can partially address this, it does not solve the problem. For example, if you have a million-level classification (e.g., many face recognition datasets have this size), you have to map the feature to a similar scale of dimension, which is computationally intractable and introduces many additional overhead. This limitation largely restricts the application of the proposed method.\n\n- Some important references are missing from the paper. What the simplex is actually doing is to encourage hyperspherical uniformity. This have a large body of works on this, e.g. [1], [2], [3]. I think they are closely related, since the simplex classifier is essentially a special case of hyperspherical uniformity (I understand that the construction procedure could vary).\n\n- The empirical gain seems to be somewhat marginal, especially on closed-set recognition.\n\n[1] Learning towards Minimum Hyperspherical Energy, NeurIPS 2018\n\n[2] Learning with Hyperspherical Uniformity, AISTATS 2021\n\n[3] Regularizing Neural Networks via Minimizing Hyperspherical Energy, CVPR 2020", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Lm2GrdZbsZl", "reviewer": "Reviewer_uWUv", "summary": "Sorry for misunderstanding the paper. Revised review is here.\n\nThis paper proposed novel loss function for training deep neural network classifier, which can be generalizably applied to many domains such as evaluation for representation learning such as MoCo, BYOL or Masked Auto-Encoder. The loss function does not need any hyper-parameter tuning since the class centers are given as vertices of simplex and each data samples are trained to map to its corresponding class center. This way, traditional way of maximizing margin between samples from different classes and minimizing margin between samples from same class can be achieved in simple and effective way. While the limitation is the dimension of output feature of deep neural net should be larger or equal to C-1, authors mitigated the issue using DAM which can be further improved. In the experiment section, the authors gave intuition on how DSC clusters the data samples compared to other method, and how the learned representation can be semantically meaningful on unknown classes. The method achieved SOTA on most classification dataset which seems promising.", "questions": "1. Does the length of the center vector u affect the performance of the DSC (or need tuning)? How does this value affect the performance?\n2. In equation (4), does the dimension of f_i, and s_{y_i} match? Since it seems that f_i is d-dimensional vector while s_{y_i} is C-dimensional vector. Please clarify this part.", "limitations": "None.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "strengths: \n1. The paper is well-written and easy to follow.\n2. The proposed method DSC generalizes to domains where the classification is required as a downstream task.\n2. Proposed method achieves SOTA on various dataset\n\nweaknesses:\n1. More experiments on validating the proposed method seems required. For example, by getting learned representations from self-supervised pre-trained model (e.g., MoCo, BYOL, etc), the evaluation on downstream classification task with various loss functions will provide more strength on the proposed method. \n2. As authors pointed out in the contribution, experimental evidence on how this method is robust to unbalanced dataset needs to be addressed.\n3. The solution of limitation seems not practical, since simple multi-layer perceptron is used between d-dimensional and C-1 dimensional vector which requires a lot of parameters of d and C-1 is large.\n4. It would be better to show distance matrix of other baselines. Although baselines' distance between class centers from known classes exhibit difference, semantic property would be preserved. Reviewer is not sure if the learned representation with DSC is actually semantically meaningful since this is not preserved for known classes. Giving more explanations on this would be nice.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "6dmycsk2Of2", "reviewer": "Reviewer_mqvu", "summary": "This paper introduces a Deep Simplex Classiﬁer (DSC) that maximizes the inter-class margins in both Euclidean and angular spaces for the open set recognition problem. Specifically, this method regards the vertices of a regular C-simplex in the (C-1)-dimensional feature space as the class centers of known C classes in the dataset and then encourages the class samples to be as close to their corresponding centers as possible. In cases that the feature dimension is smaller than C-1, a Dimension Augmentation Module (DAM) is further proposed to expand the feature dimension to C-1 to make it possible to construct the C-simplex. The open set and closed set recognition experiments are conducted on Mnist, Cifar10, SVHN, Cifar100, and Tiny ImageNet datasets while the face recognition experiments are conducted on MS1Mv2 dataset to show the effectiveness of the proposed method.", "questions": "1. The novelty of the proposed method needs further clarification. More discussions and empirical comparisons between the proposed method and [1], [2], [3] are needed since they are quite related in methodology or goal. As the relations between each pair of classes are different, is it necessary to make the pairwise distances between any two centers identical as in DSC or what is its key advantage?\n2. The motivation should be expressed more clearly. The authors mostly emphasize the complexity of hyperparameters of existing methods in introduction while do not explain how would two kinds of distances complement each other and what is the key advantage of considering both spaces in open set recognition? Besides, the presentation of introduction could be further improved.\n3. How to better alleviate the feature dimension restriction for the proposed method without introducing too many parameters, especially when there are a large number of classes as in face recognition?\n4. The ablation studies for hyperparameters u, \\lambda and m in Equation 5 are needed. \n5. For the open set recognition task, the proposed DSC uses a large amount of data from 80 Million Tiny Images dataset as the background class in Section 3.2.1, but the existing methods do not adopt this setting, for which I think the comparison in Table 1 is unfair. In line 269, a deeper network is employed for the Tiny ImageNet dataset, is it the same with the settings in existing works? \n6. For the closed set recognition task in Table 2, do authors tune the hyperparameters in SphereFace, CosFace, and ArcFace for current classification datasets or directly follow the settings in original papers? \n", "limitations": "No. The authors do not adequately address the feature dimension restriction of the proposed DSC. The proposed DAM introduces a large number of parameters when the number of classes is large.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The practice of regarding the vertices of a regular C-simplex in (C-1)-dimensional space as the C classes’ centers in the training dataset is new in open set recognition task, which has a clear geometric interpretation. \n2. The proposed method is clearly explained and easy to follow.\n3. The authors conduct the illustration experiment to ease the understanding of the effect of DSC and evaluate the proposed method extensively on open set recognition, closed set recognition, and large-scale face recognition tasks.\n\nWeaknesses:\n1. The idea of maximizing the inter-class margins by explicitly manipulating the class centers is not new. For fixed centers, in Section 3.2 of [1], Do et al. propose to regard the basis vectors of the C-dimensional space as the C classes’ centers which are also the vertices of a C-simplex. The only difference is that the feature dimension in [1] is one larger than that in this paper. For learnable centers, Hayat et al. [2] also encourage the pairwise distances between any two class centers to be similar in a learnable way to bring uniformly separated centers in Euclidean space. Besides, Liu et al. [3] propose a minimum hyperspherical energy (MHE) regularizer to uniformly distribute the class centers on a hypersphere for maximizing the inter-class separability. These two methods share similar goals with Deep Simplex Classiﬁer (DSC) but do not require that the feature dimension is larger than C-2. As the relations between each pair of classes are different, is it necessary to make the pairwise distances between any two centers identical as in DSC or what is its advantage? I think more discussions and empirical comparisons are needed considering these strongly related existing works. \n2. The motivation of this paper is not clearly expressed yet. The authors claim that the Euclidean and Cosine distances would complement each other in abstract and finally choose to maximize the margin in both the Euclidean and angular spaces in introduction. However, the authors mostly emphasize the complexity of hyperparameters of existing methods in introduction while do not explain how would two kinds of distances complement each other and what is the key advantage of considering both spaces in open set recognition? Besides, the authors simply list and explain a series of existing works one by one in introduction with few summaries, which is kind of redundant, lacks logical coherence, and could be further improved. \n3. The proposed DSC is strictly restricted by the requirement that the feature dimension is not less than C-1. The Dimension Augmentation Module (DAM) would only alleviate this issue when C is small while with a large C, DAM will introduce a large number of parameters. In the face recognition experiment, with 18.6K identities, the number of parameters of the last linear layer in DAM is about 18.6K*18.6K \\approx 346M, which is seven times more than the number of parameters of the ResNet-101 backbone (45M) and hinders the practical usage. I think a better solution is needed and essential for the general usage of DSC.\n4. The authors claim that DSC does not have any hyperparameters but actually there is a hyperparameter u denoting the radius of the hypersphere. The authors simply set it to 64 following the practice in face recognition, which lacks theoretical and empirical guarantee as this paper does not focus on the task of face recognition and uses datasets beyond face recognition datasets. For this reason, an ablation study is needed to investigate the effect of the value of u. Moreover, the ablation study for the hyperparameters \\lambda and m in Equation 5 in open set recognition is also needed. Besides, I also have some concerns about the experiments. First, for the open set recognition task, the proposed DSC uses a large amount of data from 80 Million Tiny Images dataset as the background class in Section 3.2.1, but the existing methods do not adopt this setting, for which I think the comparison in Table 1 is unfair. In line 269, a deeper network is employed for the Tiny ImageNet dataset, is it the same with the settings in existing works? Second, for the closed set recognition task in Table 2, do authors tune the hyperparameters in SphereFace, CosFace, and ArcFace for current classification datasets or directly follow the settings in original papers? Besides, the methods mentioned in the first comment should also be compared in the experiments since they are quite related with the proposed method.\n\n[1] Do, Thanh-Toan, et al. \"A theoretically sound upper bound on the triplet loss for improving the efficiency of deep distance metric learning.\" In CVPR. 2019.\n[2] Hayat, Munawar, et al. \"Gaussian affinity for max-margin class imbalanced learning.\" In ICCV. 2019.\n[3] Liu, Weiyang, et al. \"Learning towards minimum hyperspherical energy.\" In NeurIPS. 2018.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject