Discovering a Zero  (Zero-Vector Class of Machine Learning)
Authors: Harikrishna Metta
Venkatesh Babu Radhakrishnan
Venue: ICML 2025 spotlightposter (2025)
Keywords: Metta
Metta-Class
Metta Class
Machine Learning
ICML
Class Vector
Class Tensor Equation
Class Integration
Repository of Classes
Continual Learning
Class Addition
Class Subtraction
Class Invert
Zero Vector Class
Set operations on Classes
Boolean operation on Classes
Unary classification
Manifold learning
Lifelong Learning
Classifier as Generator
Primary area: general_machine_learning
TL;DR: The classes are defined as vectors in a Vector Space, where Addition corresponds to the union of classes, and scalar multiplication resembles set complement of classes. Zero-Vector in that vector space has many useful applications.

Abstract: In Machine learning, separating data into classes is a very fundamental problem. A mathematical framework around the classes is presented in this work to deepen the understanding of classes. The classes are defined as vectors in a Vector Space, where addition corresponds to the union of classes, and scalar multiplication resembles set complement of classes. The Zero-Vector in the vector space corresponds to a class referred to as the Metta-Class. This discovery enables numerous applications. One such application, termed 'clear learning' in this work, focuses on learning the true nature (manifold) of the data instead of merely learning a boundary sufficient for classification. Another application, called 'unary class learning', involves learning a single class in isolation rather than learning by comparing two or more classes. Additionally, 'set operations on classes' is another application highlighted in this work. Furthermore, Continual Learning of classes is facilitated by smaller networks. The Metta-Class enables neural networks to learn only the data manifold; therefore, it can also be used for generation of new data. Results for the key applications are shown using the MNIST dataset. To further strengthen the claims, some results are also produced using the CIFAR-10 and ImageNet-1k embeddings. The code supporting these applications is publicly available at: github.com/hm-4/Metta-Class.

Discovering a Zero
(Zero-Vector Class of Machine Learning)
Harikrishna Metta
R. Venkatesh Babu'
Abstract
1. Introduction
There are many techniques in Machine learning that allow data classification, one such contraption is the Neural Net- work. A general technique used in the data classification is the following. Find a separating hyper-surface between classes such that functional value of data points belong to same class when substituted in the hyper-surface equations resulting in 'similar' sets of values. In Neural Networks those set of values are called logits. The general technique, with respect to Neural Networks, can be understood by thinking the logit equations of a Neural Network as hyper- surface equations, those logit values are passed though the Sigmoid or Softmax layer. The similarity of the logits of data points belonging to the same class is enforced by the cross entropy loss.
In Machine learning, separating data into classes is a very fundamental problem. A mathemati- cal framework around the classes is presented in this work to deepen the understanding of classes. The classes are defined as vectors in a Vector Space, where addition corresponds to the union of classes, and scalar multiplication resembles set complement of classes. The Zero- Vero-Vector in class referred
the vector space corresponds to as the Metta-Class. This discovery enables numerous applications. One such application, termed 'clear learning' in this work, focuses on learning the true nature (manifold) of the data in- stead of merely learning a boundary sufficient for classification. Another application, called "unary class learning', involves learning single class in isolation rather than learning by com- paring two or more classes. Additionally, 'set operations on classes' is another application high- lighted in this work. Furthermore. Continual Learning of classes is facilitated by smaller net- works. The Metta-Class enables neural networks to learn only the data manifold; therefore, can also be used for generation of new data. Re- sults for the key applications are shown using the MNIST dataset. To further strengthen the claims, some results are also produced using the CIFAR- 10 and ImageNet-le Ik embeddings. The code sup- porting these applications publicly available at: github. com/ hm-4 /ete/MttaCCaaaaa
An example of decision boundaries of a neural network is shown in the far-left subfigure of Figure 1. The feature space in that subfigure divided into two decision regions: the lower half is classified as Class indicated by the green region, while the upper half is classified as Class Class-1, 1, represented by the red region. However, this dichotomiza- tion of the entire feature space into these two classes is not accurate, as only small portion of the space genuinely belongs to Class-1 or Class-2 2. Therefore, the decision regions depicted in the far-left subfigure of Figure do not accurately represent the true nature of the classes. A more realistic representation of the decision regions can be seen in the mid-subfigure of Figure 1.
If neural network exhibits the decision regions as shown in the mid-subfigure of Figure 1, these regions have par- ticularly notable set properties. One such property is the union of the regions: the union of the decision regions ac- curately represents the true nature of the combined class's s decision region. Another property is the complement of the decision region: illustrated in the far-right subfigure of Figure 1, the complement of a decision region accurately represents the true complement of the class. These prop- erties are essential, as they reflect the true characteristics of the classes. Consequently, neural networks with such well-defined decision regions for each class are desirable.
VVision and AI Lab (val.dds.isc.accim Department of Computational and Data Sciences. Indian Institute of Science, Bangalore, INDIA. Correspondence Harikr- ishna Metta <hm. wtm.iisc.iitbb. gomil.com>, R. Venkatesh Babu <venky iis...i.in.........
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In this work, classes are characterized in terms of the logit

---

Discovering Zero
fix)
0.
21166)
QAUG)) 4
20//ie) (J))
Threshold Line
07-
0.1
03
+ Class
* Class
Figure 1. The far-left subfigure represents the decision regions of a neural network, learned by separating the feature space. The middle subfigure shows the actual regions corresponding the classes, while the far-right subfigure illustrates the complement region of Class-l.
001
Figure 2. A set of functions that classify Class-f f (z:) with the exact same decision regions for a threshold of ay = 0.1.
unnormalized distribution f(r). Notice that functions of the form hxx) + k, where h(x) E [f()) and k E R, with threshold a + k can perform the same classification task as h()) and ay- Therefore, to accommodate equations of the form h(x) + and thresholds of the form ay + k. as well as any other valid pair of functions and thresholds, a separate set S is introduced in equation
equations, and logit equations are viewed as vectors. Ad- dition and scalar multiplication on such vectors defined such that a vector space created. As a consequence, it allows to use all vector operations on logits hence enables set operations classes. The relevant proofs and meanings of the operations are provided, without going into very rig- orous math. Sample PyTorch implementations and results are attached supporting the theory when needed.
= [ (8(()) ay + +as BERT TR JuS (1)
2. Characterization of Logits
Here, f(x) represents the unnormalized PDF of the class, af R is the predefined threshold for fxx), Ay represents the normalization constant of f(x), and S denotes the set of unknown (does not need to be an unnormalized or a normalized PDF) Valid Logit Functions associated with that class.
If the probability density function (PDF) of the data, Jn (2r), is known, a given data point can be assessed for membership in the distribution by comparing its probability to prede- fined threshold, ox The threshold a, should be chosen as a non-negatie non-negative real number, ensuring that data points with high relative probability are recognized as belonging to the class. As illustrated in Figure 2, data points with probabilities (z)) greater than or equal to a',, 0.1 are classified as belonging Class-f, f. while those with lower probabilities are classified as not belonging to the class. The PDF function fn (x) itself is used for this classification along with the threshold f....
The set [f()) in equation is an equivalence class with respect the classification of the data, it satisfies the properties of an equivalence relation. Specifically, any equa- tion within this set classifies the data in exactly the same way as others in the set for a predefined threshold. Consequently, the set [f))] is referred to as the 'Valid Equivalence Set' of f(x). This terminology will be consistently used throughout this paper.
In cases where only the unnormalized PDF f(x is known, is still possible to classify data points by comparing fx) to the scaled threshold az A where A, represents the nor- malization constant of f()). A diagram illustrating this
The following subsection explores the resulting Valid Equiv- alence Set when two classes are combined.
mechanism can be seen in Figure 2, where A, = 1.
2.1. Merging two Classes
The value f(0) for a given data point 2g is referred to as the 'Logit.' The function fxx is defined as the 'Logit Func- tion' or 'Logit Equation.' Any Logit Equation that correctly classifies data with respect a predefined threshold a is termed 'Valid Logit Equation (VLE)' "Valid Logit Func- tion.' These definitions will be used consistently throughout this paper.
Now consider another class, Class whose unnormalized PDF is 9(x). Figure illustrates some Valid Logit Functions for Class g. similar to those shown for Class in Figure 2. Any function k(x)) belonging to the set [g(x)], as defined in Equation (2), is Valid Logit Function capable of classifying data sampled from the unnormalized distribution g(z), in the same way as was previously done for Class
All Valid Logit Functions can be grouped into a set, as shown in equation 1. Any function h(z) that belongs to the set [x)) in equation can classify data sampled from the
[g(z)) = = {((g(z) agAg) +
BERR R
} JUT (2)
2

---

Discovering Zero
Ste)
Fist
04-
Riglal
04
GRgpaa (00) cas
0.01 fie) 0.3)) aub
2Xplx)
201-/ta d.t) R.1,
Threshold Line
fiar (013)
Trreshold Line
0..
o.y10.15
0.1
0.05
-0.1-
-01
Figure 3. A set of functions that classify Class-g (g(x)) with the exact same decision regions for a threshold 0tg 0.05.
Figure 5. A set of functions that classify complement of Class- with the exact same decision regions for a threshold of a 0.15.
04-
0.5tnte) - 0.13) 1.12
Here, u(x:) represents the PDF of the combined distribution. au R is predefined threshold for PDF u(x). A.. repre- sents the normalization constant of u(x), and Z denotes the set of unknown (does not need to be an unnormalized or normalized PDF) Valid Logit Functions associated with the combined distribution u(z).
Wale) 0.18--10
Tireshold Line
02
0.12
-01
An important question arises: given the logit equations h(x) E [f(z)] and k(x:) [g(x)). is possible to construct new logit equation v(x) E [u(x)] for the combined distri- bution? If achievable, v(x) could be utilized for classifica- tion of the combined distribution with PDF u(x). However, this is not feasible, as numerous counterexamples can be identified. A related question then follows: can sets [f)] and [g(=)] themselves be combined to derive new set [u(x)], as shown in Equation (5).
Figure 4. A set of functions that classify combined Class-u (Class-u- Class-f U Class-g1 -g with the exact same decision regions for threshold of an, 0.12.
Here, g(x) represents the unnormalized PDF of Class ag E R is the predefined threshold for ((x), Ag represents the normalization constant of g(x), and 7 denotes the set of unknown (does not need to be an unnormalized or a normalized PDF) Valid Logit Functions associated with Class-g.
[f(z)] U [g(x)] = [u(x)]
(5)
Suppose there need to merge the data of Class-g with the data of Class-f. The resulting PDF of the combined class, denoted u(x), is illustrated in Figure 4. Assuming an equal number of data points in Class- f and Class-9 the equation for u(x) in terms of PDFs fn (u) and (2), the normalized distributions of f(z) and g(x) respectively, is given in Equation (3). The proof of this equation is provided Section of the Appendix.
+ + [g(x)] = fn(x) + (x())
(6)
2
[[(z)]+ + [s(=)) = [s/ (x) + (x)(
Notice that the sets [x) + gx, and fa(x)) are exactly the same. The '+" operator in the left-hand side of Equation (6) is used However, the '+' operator on the right-hand side of the same equation denotes the standard addition operator. This choice intentional, as the equation is later adopted as definition addition in the following section.
u( u(x) fa(x) (x)
symbol to represent union.
Similar to what was observed for Class-f and Class D. there can be multiple Valid Logit Functions corresponding to the combined PDF u(x), as illustrated in Figure The set [u(x)), as defined in Equation (4), represents Valid Logit Functions associated with the PDF u(z).
2.2. Complement of a Class
There is an observation on the Valid Equivalence Set [-f(=)]: represents the complement Class-f. as lustrated in Figure for the same predefined threshold ay-
[u(x)] {s(u(z) AuAs) + au
B + Juz (4)
3

---

Discovering Zero
Mathematically, this is expressed in Equation (8).
3.2.2. SCALAR MULTIPLICATION
For all À e R and for all [f(x)] E V the product ([f(x)] defined
[f(z)] =(=(( -[[(=)]][-[(1)
(8)
(9)
A [f(x)] = [a f(x)]
(12)
The '-' operator the left-hand side of Equation (9) is used as symbol to represent the complement operation. However, the '-' operator on the right-hand side of the same equation denote standard subtraction operation. This choice is intentional, as the equation is later adopted as a definition in the following section.
3.3. Verification of the Properties of vector space set V, along with addition and scalar multiplication
3.3.1. COMMUTATIVITY
For all [f(x)], [ø(x)]] e V using the property of addition
[f(z)] [o(x)) [f(=) + o(x)] [9(x)] + [f(x))
3. The Vector Space
Hence the commutativity property of vector space holds on set V.
As seen in the previous chapter, the set of Valid Logit Equa- tions of class, whose PDF is f(r), is called the Equiv- alence set and represented by the [f(=)]] Therefore the function space is divided into set of these equivalence sets each representing class.
3.3.2. ASSOCIATIVITY
For all [f()),, [o(x)], [M(x)] E V, using the property of addition
In this chapter, all the properties of vector space (Axler, 2015) are verified for that set of equivalence sets, using the definitions of addition 3.2.1 and scalar multiplication 3.2.2. During this process a zero is discovered for this vector space, this zero represents an equivalence set. As previously seen each such equivalence set represents a data class. The data class represented by the zero element of the vector space is referred to as the Metta-Class (also called the Zero-Vector Class).
(I((=) + (o()))) + (M(x)] = (I(())+ + g(x))) + [*(x))
=[f(x)] + (l9(z) + +(=)))
let a, E IR, then using the property of scalar multiplication
(ab)[f(=)) a (blf(=)))
= =a(((())))
3.1. Definition of set V
Hence the associativity property of vector space holds on set V.
Let V be the set of equivalence sets. Each equivalence set also known as Valid Logit Function set represents data class. If the PDF of data-class is (ar) then the correspond- ing equivalence set is denoted by [f()). Therefore the set V can be written as
3.3.3. ADDITIVE IDENTITY (THE ZERO)
For all [f(x)] E V there exists such that
= { (1x)), [o(x)), (M(=),, ....}
(10)
[f(x)] + = [f(x) +
[f())]
Throughout this report, whenever the vector space V is mentioned, it refers to this vector space.
Hence there exists an additive identity on set
3.2. Definition of Addition and Scalar multiplication
3.3.4. ADDITIVE INVERSE
As understood in the previous chapter, the equations for union (7) and complement (9) are taken as definitions here with slight modifications so that they are suitable for defin- ing a vector space. Specifically,
For every [f(x)] E V there exists [-f(x)) such that
[f(x)] + +--((x) [f(x) + fxx)]
[0]
3.2.1. 1. ADDITION
Hence there exists additive inverse on set V.
For all [f()),, [s(x)] E V. the sum [f(x)] + [g(x)] is defined as
3.3.5. MULTIPLICATIVE IDENTITY
For every [f(x)] e V. 1ff(=) = [f(z)). hence the multi- plicative identity exists.
= [f())+((x) gz)]
(11)

---

Discovering Zero
3.3.6. DISTRIBUTIVE PROPERTIES
i.e., [x) lim k: then equation 14 becomes
For all a, be IR and for all [f()),, [a(x)]
Ao =
v v=1-000,0 Jy
lim
lim dr R-0
(16)
+[(())+[[(()))))) = alf(x)) + c[g((]]
lim / k dxc
lim
J
(17)
[af(x)]+ + [ag(t)
v- k-0
(a (a+ [[[((]] = ([][(=)]++[((()))
lim lim Vk
dr
(18)
V-]-00,00]1 k-0
Therefore PDF corresponding to [0] can be written as: PDF((0)) =
Hence the distributive properties of vector space holds on set V.
lim lim k
(19)
V-[( 0000000 k-+ Vk
Therefore, the defined set V forms a vector space under the specified operations of addition and scalar multiplication. The equivalence sets are represented vectors in this space with each vector cor-
lim
(20)
[-0,-0000) V
Substituting Equation (18) into set (13) results in set of constant values. From Equations (15) and (20), the PDF of the Metta-Class appears to follow a uniform distribution. This observation serves as an indication rather than a rigor- ous proof. If we consider the PDF of the Vector as a uniform distribution, and since also satisfies the additive identity property, then in peculiar sense, having data uni- formly distributed across the entire space equivalent to having no data at all. For limited volume V, PDF corre- sponding to [0] is uniform distribution with magnitude 1 V. Intuitively, if a threshold a is used for the PDF fxx), then for f(x) + +. using a threshold of a - + results in the exact same decision regions; therefore, the uniform distribu- tion can be seen as the additive identity of the vector space. For the remainder of the report, the PDF of [O] is considered as uniform distribution. For an alternative explanation, Section F in the Appendix.
and are referred as responding to distinct data class. In particular, the zero vector ((O) in this space might also represents data class, referred to as the Zero- Vector Class.
To avoid ambiguity between the concept of the Zero-Vector Class and the label "zero" used for other classes, this class hereafter referred to the Metta-Class.
4. Metta-Class and its Applications
As established in the previous chapter, the set V defined in Section 3.1, together with the addition and scalar multipli- cation operations described in Subsections 3.2.1 and 3.2.2, forms vector space. Each vector in the vector space V rep- resents a data class. The additive identity vector [0] of this space (see Subsection 3.3.3) must therefore also represent data class. In the previous chapter, the class corresponding to this additive identity vector was named the Metta-Class.
4.1. Generating Metta-Class data in finite volume of the feature space.
Using the equation 1, the set [0] can be written as
The data of any natural class generally spans small volume in the feature space, as shown in the top-left plot of Figure 6. Within this small volume, data for the Metta-Class can be generated from a uniform distribution spanning the same volume. The generated data for the Metta- Catas-Classiie is illustrated in the top-right plot of Figure
[0] {000-004)) Ao) + ao 3 B R
(13)
Here, og is a threshold for PDF the class [0]. Ag repre- sents the normalization constant of the PDF corresponding to set [0].
if we define normalizing constant A, of unnormalized PDF
4.2. Applications of Metta-Class
f(x) as
1.2.1. CLEARR LEARNING
lim A₂ V- 0=0-000000 / (xx) dr
(14)
A neural network called Zero-Inclusive-NNeworkk trained on the data shown in top-right subfigure of the figure 6, which includes the Metta-Class data. For comparison, another neural network called trained on the data excluding the Metta-Class data i.e., the data in the top-left plot of the Figure 6. All front layers in both networks are identical except for the last layer. In the last layer of Zero-Inclusive- Anclusive-Network, an additional node is used for the Metta-Class.
Simple substitution fx) = 0 in Equation (14) gives Ao = 0 hence
is
0
{
(15)
where ag E IR. Therefore, the logit equation of the Metta-
Class is a constant valued function.
If f (ar) = is treated as constant going to zero the limit
5

---

Discovering Zero
Class Class 2
Cless
Closs
Zero-Vector Class
Zero-Vecta Vector Cless
Figure 7. The left plot shows the data of Class-O and the Metta- Class, while the right plot depicts the decision region of Neural Network trained on these two classes using BCE loss. This illus- trates how a Neural Network can learn a single class.
Therefore learning is possible even with only one class data, which is desirable since it is preferable use thousands of smaller neural networks for training rather than training one giant neural network for a thousand classes. Results for MNIST, CIFARIO and ImageNet data are added the Results Section 1.1. of Appendix.
Figure 6. The top-left plot shows the data of three different classes, while the bottom-left plot depicts the decision regions learned by Neural Network for these classes. The top-right plot includes the three classes along with the Metta-Class, and the bottom-right plot illustrates the decision regions learned by Neural Network for these four classes.
4.2.3. SET OPERATIONS ON THE CLASSES
Upon closer examination, bottom-right plot of the Figure 6 exhibits resemblance to Venn Diagrams. This resemblance is coincidence, as the addition 3.2.1 and scalar multipli- cation (with scalar value 1-1') 3.2.2 defined on the Vector Space have the characteristics of set union and set comple- ment respectively. As the set union (boolean addition) and set complement (boolean not) are the fundamental opera- tions on sets (boolean logic), any composite set operation (composite boolean operation) can be written in terms of set union (boolean addition) and set complement (boolean not). The table lists a few of the set operations on classes.
The differences in the learning outcomes of both networks are illustrated in the bottom plots of Figure 6. The bottom- left plot corresponds to the Eee---xllliieeeeeee the bottom-right plot represents the Zero-Ine trrro-nclusive-Netteee It can be observed that the boundaries learned by the Zero- are superior and clearer. By 'clear,' it is meant that decision regions of the ienoo-nclusive-eewwo overlaps exactly with the data. For instance, in the bottom- left plot of Figure 6, which illustrates the learning process of the Zero- Eeee--Exxuu-eeeeteeeee empty space is misclassified Class-O, Class- or Class-2.
while
Consider two classes, Class-1 and Class-2, trained on two different networks (uni-class classifiers) as described in Sub- section 4.2.2. Let Te, and Tc, denote the indicators (network outputs) for Class-l and Class-2, respectively. The indicator values are 1 if the data belongs to the class they represent and 0 otherwise. Table 1 illustrates the logical operations on these classes. Furthermore, any boolean expression in- volving the classes can be evaluated using these logical operations. These logical operations can also be performed on multiple outputs of a single network. For instance, logi- cal operations can be applied the classes of the classifier corresponding the bottom-right plot of Figure 6.
a significant portion of the
4.2.2. UNARY/UNI-CLANS CLASSIFICATION
Imagine scenario where all the collected data belongs to single class. Is it still possible to train neural network in such a case? Moreover, can the trained network determine whether a new data point belongs to this class or not? In neural networks, generally, the learning is done by compar- ing one class against another class(es). If the data belongs only one class, say Class-0, the learning does not make much sense. This is where the Metta- Ceeta-Claas is helpful. The data for the Metta-Class is a set of samples from uniform distribution. Now the original class-O can be classified against the Metta-Class data with BCE loss using a neural network. The learning of the Neural Network trained on the combined data can be seen in the right subfigure of Figure?
4.2.4. CONTINUAL LEARNING
In general, when the Metta-Class is not used to train neural network, every time a new class is added to the network, the network must be retrained or adopt some complicated techniques. For example say network-1 classifies all fe-
6

---

Discovering Zero
AL
Table Boolean operations between Class-c and Class-c2, where tor and Lez are indicators derived from classifiers trained including the Metta-Claa Class. fej 1 implies data point belongs Class- C1, while le2 = implies it does not. For example, the fifth column represents the intersection operation, indicating whether data point belongs to both Class-ci and Class-c2, demonstrating that a separate classifier is not needed to identify data belonging to both classes.
Uurry (lue of Cleen
Usary (haitt)))))) XX)) of Clus-
Predictione
Sigwoi
Tel:
Aargoee
of All Close
Acceracy
Pantiki
Unary (1uadii(NN)) of Cooedl
Tes
Log
JeLnce
La
Cleasic MNNST
0
0
1
0
0
0
0
Signcié
Predickons
0
0
1
Caadtive FrahlocMMNSs
orgrood
1
0
1
1
0
Tal Data
Accaracy
Caeilee of KINISSI
Praction
0
1
1
0
Lanic
Figure 8. In the top figure cach individual class of MNIST/CIFARRO learned on separate network, and all the individual networks are kept together act single network. It can be seen that for sighee-dimensional data this method doesn't scale well. In the bottom figure, different of MNIST classes are learned on different networks, and all the networks are kept together to act as single network.
lines (cats, lions, tigers, cheetahs, jaguars etc) and if there need to classify pets (cats, dogs, parrots etc) a new neural network must be trained. The information about cats learned on the network-1 1 is not useful for pets classifica- tion. This kind of split brains among neural network does not integrate the learning. But if the Metta-Class is used in training, the knowledge can be integrated. Which means the information learned on old networks can be used for newer classifications. Over the time a set of networks will act a repository, providing information for any kind of classification.
Caist
Ches
Clez
Clsi
Ome
The reason is that the empty space (regions with available data) is not classified as belonging to any class, as seen in the bottom-right plot of Figure 6. Consequently, the class boundaries learned do not create confusion with any new class, whose data may spread into the empty space of the old classifier's classes in the future. See Section in the Appendix for an example.
Figure 9, Each image is generated from classifier trained single class (c.g., the bottom-far-le image is generated from a classifier trained solely digit-5 MNIST data against the Metta- Class).
Results on MNIST and CIFAR are presented in the Figure 8 and in Subsection E.3 of the Appendix. The classifiers in the Figure 8 are learned separately without knowledge of the different classes, and the figure particularly shows accuracies when such networks are combined as a repository working as single classifier.
initialized with random noise image (zn)-
4.2.5. USING CLASSIFIER FOR GENERATION OF NEW SYNTHETTCC DATA
aT.
Neural network when trained properly with the Metta-Class, learns only the data manifold as shown in the Figure 7. It means that the logit values of the data from the class attain positive values only inside the manifold and negative value anywhere outside. If one hypothesizes that logit values at- tain peak inside the manifold, where the mode of the data lies, and gradually decreases when going away from the mode, then it is possible to reach the mode following the gradient of the logit hence generating a new synthetic data- point in the process. Images in the Figure 9 are generated using simple gradient descent as shown the equation 21,
T4 = Fk-1
(21)
Ör
1=
where L is the logit of particular class, shown in the Figure 8
4.2.6. EQUATION DISCOVERY
Coefficients of Taylor series equation of a class can be discovered by treating Taylor series Equation (22) (23) as Logit equation of that class, with Cross Entropy loss on
7

---

Discovering Zero
Tral
The applications discussed in this section are empirically evaluated in the Appendix. Detailed results and analysis can be found in Section E.
Coefficients:
E0-
X Xy [[x]
155
= my +
xx in +
max
No 2266516495 10.8534 ]
5. Time Complexity
My: 3.09122266706629
mi: 0..19688722 4092536
25
Zero Vector Class
my: [[0060970885 1.450288
To analyze the complexity clearly, we divide the computa- tional requirements for any e-class classifier into two distinct parts: (1) Pre-softmax logit computation, and (2) Softmax computation.
Clasn
[-1. 11.995912 -0 43427515]]
-30.0 -25
-50
-25
1.
Figure 10. Left plot illustrates Class-1 1 data, while the right subfig- shows the coefficients of function representing this Class learned via backpropagation.
Let a Excc-uxcu Network be classifier that does not use the Metta-Class during training. Suppose there are training points, and let the computation needed to compute pre-softmax logits be ((f instance, [n) might represent millions of computations. Computing the softmax function then takes O(n softmax(c)) computations, where softmax(c) represents the complexity of softmax over classes.
whether the Logit value is positive or not.
h(x) b
Σ
Zjz
j1
11m1
j2
Now consider modifying the Zero-Exclusive Network by adding an extra node at the output to classify the Metta- Class; we call this the Ierou-nce Network. In this sce- nario, we incorporate the Metta-Clas, in training. The addi- tion of the new node introduces extra calculations dependent the number of nodes in the preceding layer. Letting L denote the number of nodes in the layer preceding this new node, the additional computation required is (nI?))
Σ
(22)
jam1
The equivalent Tensor equation for the Equation (22) can be written as below.
h(*)= = ma + mıs 24 + m2sy T,æy + masjk T2 +
(23)
The computational complexities during training and infer- ence are summarized in the Table 2.
where mo, mi. m2jj, Maigk ... are unknown scalars.
These unknown scalars are learned through backpropaga- tion, as is done in neural networks. An empirical result illustrating this is shown in Figure 10. This can be used for discovering equations of lower dimensional physical phenomenons.
Typically, (n)))) dominates O(n..)) since I. corresponds only to the final internal layer. There is trade-off between computational overhead and purity improvement. This trade- off depends significantly on the intended applications lever- aging the full potential of the Kero-Vector framework.
Metrics
6. Related Work
To Identify which classifier is superior with respect to leam- ing true boundary, we used the following two metrics:
Theorem 1 of Gutmann and Hyvärinen's 's pioneering work Doise-Contrastive Estimation (2010) establishes that, under its conditions, the logit of a neural network trained to distinguish data sampled from a density pa(-) and a noise distribution (.(.)) is given by equation 26
Occupancy Factor Pooj + :u + 1oo0.c +...
No)
(24)
where Nip is total data points of the Metta- Class used in training, and ci is the data points of the Metta-Class recognized as class class-Ck- Ck-
G(u; 0) = In pa(u) - In pm. (1.)
(26)
In our work, the noise distribution P..(.) is modeled as the Metta-Class, which follows a uniform distribution, i.e.,
Purity Factor Class Cj (predd=C, A Cabet=Ca)
(25)
P(pred=Cc)
(.(.) = k where is a known constant. Substituting this into equation 26, we obtain:
where "'(pre=CC, A label=Ca) is the total training examples that are predicted as class-C; and also labeled as class-Cr, and (((rred=C.) is the total training examples predicted as class-Cr Ck. Further details on these metrics are provided in Section D of the Appendix.
G(u; 0) In pa
(27)
If the expression In pa(xa) is used as a representation in place of Valid Equivalence Sets (e.g., In fa) h for [f()).
8

---

Discovering Zero
Table Computational complexity comparison between classifier that incorporates the Metta-Class data during training (referred Network). Let the computation required to calculate
a Zero- Inclusive Network) and one that does not (referred to as a pre-softmax logits during training for n data points be ([)(n)) in the Zero-Er -ece-Excluuive Network, where f() may represent millions operations. During inference, the corresponding pre-softmax logit computation for m data points is denoted by O(g(m)).
Classifier
Training Cost
Inference Cost
Bero-Exclusive
O(f(n)) + O(n softmax(c)) = (n))
((g(m)
Network
Zero-Inclusive
Of((++k) + (n+))12) + O((+k) softmax(c+1)) O(f(++)))
+lg) O(g(m))
Network
In g(x) - for [g(x)). In (()) k for [u(z)], etc.) of the be
Estimation and Energy- Based Models, highlighting its util- ity in enriching existing paradigms.
Vector Space in section (3), the same conclusion reached-that is, the Metta-Class has constant logit value. Consequently, the data in the Metta-Class can be interpreted as uniformly distributed.
However, this approach currently demonstrates limitations in scalability, as observed on CIFARI0 datasets. Addressing these challenges offers opportunities to extend this frame- work to handle the way for broader applicability and impact.
Based on equation 27, the density can be expressed as:
data effectively, paving
pa(u)= = G(u0))+
(28)
Acknowledgements
and the score as:
Harikrishna Metta would like to express his sincere grati- tude to his advisor, Prof. R. Venkatesh Babu, for his contin- uous guidance and support throughout this project. He is also thankful to the examiners, Prof. Aditya Gopalan and Prof. Rajiv Soundararajan, for their valuable time, construc- tive feedback, and encouragement during the evaluation.
V Inpa(uu) = VG(u;;)
(29)
The score function derived from equation 29 has demon- strated its utility in generating new data using Langevin dynamics (Song& & Ermon, 2019). This approach provides an explanation for the successful generation of MNIST data, as outlined Subsection 4.2.5.
He deeply appreciates the support and collaboration of his lab-mates at VAL, IISe Priyam Dey, Rishubh Parihar, Badrinath Singhal, Ankit Dhiman, and Abhipsa Basu- - for their valuable technical discussions and suggestions.
Furthermore, notable related works include advancements in Energy-Baased Models, such as (LeCun al., 2006) and (Grathwohl et al., 2019), which employ strategies to increase the energy of noise samples while decreasing the energy of true data samples.
He would also like to give special thanks to Mohd Shadab Ansari and Shatakshi Gupta for their consistent encourage- ment, helpful comments, corrections, and moral support throughout the project.
Additional significant contributions in this domain include (Hinton, 2002), (van den Oord et al., 2018), and (Chen al., 2020), which explore various methodologies for self- supervised learning and contrastive representation learning.
We would also like to thank the anonymous reviewers for their constructive comments and insightful suggestions that helped improve the quality of this work.
7. Conclusion
Impact Statement
This work presents a mathematical framework to represent classes as vectors in a Vector Space, where Addition cor- responds to the union of classes, and scalar multiplication closely resembles the set complement of classes. Notably, it was discovered that the Zero-Vett Vector of the vector space is uniformly distributed in the feature space.
This work aims to advance Machine Learning by introduc- ing a more effective training method for existing models. The proposed approach enables applications that extend be- yond the capabilities of traditional neural network training. While we do not anticipate specific ethical concerns, the broader societal impact includes improved accessibility to advanced ML techniques. We encourage responsible and lawful use of this work, with careful consideration of the ethical implications in downstream applications.
This framework opens possibilities for developing several novel applications, many of which are not achievable with standard neural network training techniques. Additionally, this training methodology complements Noise Contrastive

---

Discovering Zero
References
(eds.), Proceedings of the Thirteenth International Con-
ference Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pp. 297- 304, Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR. URL https: http///ooceeeeeee ngs .mlr.
Axler, S. Linear algebra done right (eBook). Springer, Cham, 3rd ed. edition, 2015. URL https://doi. 005/10.1007/9900 3- -------000-----------------11-1-
Ceylan, C. and Gutmann, M. U. Conditional noise- contrastive estimation of unnormalised models, 2018.
press/ p wttgg//taannnnneee/ html.
Hinton, G. The forward-forwatt algorithm: Some pre- liminary investigations, 2022. URL https: httthttss//aake //arxiv.
URL httph: /taptpps//axxi....// org/ wbs./8/...../.............boaaea 03664.
org/ aba/bb//// 221.22221..111.....
Chandola, V., Banerjee, A., and Kumar, V. Anomaly detection: A survey. ACM Comput. Surv., 41(3), July 2009. ISSN 0360-0300. doi: 10.1145/1541880.
Hinton, G. E. Training products of experts by mini- mizing contrastive divergence. Neural Comput., 14 (8):1711-180, August 2002. ISSN 0899-7667. doi: 10. 10.1162089976662201228888 URL https: /doi. 223/1...1662//896662227228888880
1541882. URL https://dois http///ii..r/11111///./.
1541880.154188
Chen, T. Kornblith, S., Norouzi, M., and Hinton, G. E. A simple framework for contrastive learning of visual representations. CoRR, abs/2002.05709, 2020. URL htttp://arxiv..rr org/ http://.b//2000..00000 www.xxii.orrggooooo
Johnson, J. M. and Khoshgoftuar, T. M. Survey on deep learning with class imbalance. Journal of Big Data, 6(1):27, 2019. ISSN 2196-1115, doi: 10.1186/ $40537-019-0192-55. URL https: /ttp//iooog////////////////////e 1186/44053--012
Cortes, C. and Vapnik, V. Support-vector networks. Ma- chine Learning, 20(3):273-277, 1995. ISSN 1573-0565. doi: 10. 10.1007/BF00999401... URL Utthttps: /ttp:://dooooooooo/ 10. 1007 10100000000000 /3F009940009.10000
LeCun, Y., Chopra, S., and Hadsell, R. A tutorial on energy- based learning. Tutorial at the International Conference on Machine Learning (ICML), 2006.
Du, Y. and Mordatch, I. Implicit generation and modeling with energy based models. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F. Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedding neurips. cc hc/papprrfiiles/peeeeeeeeessseep /Ltpee/eee/eeeppppr/////////// 378 3755033brrabbccc44e1e444444eee pdf.
Makke, N. and Chawla, S. Interpretable scientific discovery with symbolic regression: A review. Artificial Intelligence Review, 57(1):2, 2024. ISSN 1573-7462. doi: 10.1007/ s $10462-023-10622 12-22200 URL ht /ttp://doo.o//// org/10. 1007/s10462-023-12222
Mikolov, T., Chen, K., Corrado, G., and Dean, J. Efficient estimation of word representations in vector space, 2013. 3781.
URL htw.shtpp//soo...tt / wwwaa. abs
Elsken, T., Metzen, J. H., and Hutter, F. Neural architecture search: A survey, 2019. URL htt.t:tt/////eett abs/1808 05377.
Ren, P., Xiao, Y., Chang, X., Huang, P.-y., Li, 7.,, Chen, X., and Wang, X. A comprehensive survey of neural architecture search: Challenges and solutions. ACM Comput. Surv,, 54(4), May 2021. ISSN 0360-0300. URL
Grathwohl, W., Wang, K., Jacobsen, J., Duvenaud, D., Norouzi, M., and Swersky, K. Your classifier is se- cretly an energy based model and you should treat it like one. CoRR, abs/1912.03263, 2019. URL http: //arxiv. orgo abs /tabs/199 1911. 03263.
doi: 10.1 10.1145/3447582.
1145/ 14553447582.
Song, Y. and Ermon, S. Generative modeling by es- timating gradients of the data distribution. CoRR, abs/1907.05600, 2019. URL abs 0tts/1977.06600
Grinfeld, P. Introduction to Tensor Analysis and the Cal- culus of Moving Surfaces. Springer New York. NY, edition, 2013. ISBN 778-1-4611-7866-9
doi: 10.
Udrescu, S.-M. and Tegmark, M. Ai feynman: physics- inspired method for symbolic regression, 2020. URL absg///bb/1990... 1144.....11...11.1.1....
1007/978-114611-78888 URL https: http::/doo.ooooo org/ 10 10.1007/ 01111/9988--1111-- 786 Published: 24
September 2013, 33 b/w illustrations, 4 illustrations in colour.
den Oord, A., Li, Y., and Vinyals, O. Representa- tion learning with contrastive predictive coding. CoRR, 807.003777777 2018. URL http: tittp://arxiin /arxiv.org/ abs/1807.0377
Gutmann, M. and Hyvärinen, A. Noise-contrastive esti- mation: new estimation principle for unnormalized statistical models. In Teh, Y. W. and Titterington, M.
10

---

Discovering Zero
A. PDF of resultant class when two different classes are merged.
Suppose there a need to merge the data of Class-93 with the data of Class-f. The resulting PDF the combined class, denoted as u(z), can be expressed in terms of the PDFs fn (a) and gn (z), which are the normalized distributions of f()) and (xx) respectively. The combined distribution can be determined by finding the number of data points that belong Class-ue in arbitrary interval Az, shown in Figure 11.
0.35
0.30
g(x)
0.25
0.20
0.15
0, =0.1
0.10
0.05
0.00
-0.05
0.10
Figure 11. Figure shows of two distributions combined to form a bigger class.
Let N2 and Ng be the total number of data points belonging to class f and class g respectively, Then the number of points in the interval Ar that belong to class are
N, (n(o)) N₂ + In (xo Az
Az Ng
(30)
the density of number of data points at zo of class say (z), can be written as below
u( Ar (Ny Ng) = No fa(c)) N, In (xo) Ar Ng
(31)
Therefore the probability density u(x) of the class-u
u(x) fn(x) N + (x) N₂ N₂ + Ng
(32)
If N Ng. then
u(x) = fn(x) + (x)
(33)
2
B. Continual Learning
In general, when the Metta-Class is not used to train neural network, every time new class is added to the network, the network must be retrained. And say network-1 classifies all felines (cats, lions, tigers, cheetahs, jaguars etc) and if there is a need to classify pets (cats, dogs, parrots etc) new neural network must be trained. The information about cats learned on the network- is not useful for pets classification. This kind of split brains among neural network does not integrate the learning. But the Metta-Class used training, the knowledge can be integrated. That means the information learned on old networks can be used for newer classifications. Over the time set of networks will act repository, providing information for any kind of classification. An example is shown in Figures 12 and 13.

---

Discovering Zero
MNNIST
*
MNNST
Netwock for All Classes
Network 1
Network-2
Figure 12. This figure represents how classifiers typically learn when the Metta-Clas is not included during training. The top-left subtigure shows the data (a representation of MNIST data), while the top-right subfigure illustrates the boundaries learned by neural network trained on all the classes. The bottom-left and bottom-right subfigures depict the boundaries learned by two different neural networks trained MNIST and FNSS data, respectively. The knowledge learned by Network- is compatible or transferable to the knowledge Icarned by Network-2.
MNIST
Pachion
Network for All Classes
Network--1
Network-2
Figure 13. This figure represents how classifiers learn when the Metta Metta-Class data is included during training. The top-left subfigure shows the data (a representation MNIST data), while the top-right subfigure illustrates the boundaries learned by neural network trained on all the classes along with the Metta-Class The bottom-left and bottom-right subfigures depict the boundaries learned by two different neural networks trained on MNIST and FashionMNIST data, respectively, along with the Metta-Class. Unlike the boundaries learned by Network-1 and Network-2 in Figure 12, the boundaries learned here are distinet and can be merged to produce boundaries similar to those a network trained on the classes and the Metta-Class, as shown in top-right subfigure.
12

---

Discovering Zero
C. Taylor-Series Based Network
Any vector of a vector space can be written as linear a combination a list of linearly independent vectors spanning the vector space, these spanning and independent vectors are called the basis vectors. The following one such list of linearly independent vectors for the Vector Space V.
C.1. A Basis
Basis of the vector space is list of class-vectors in V that are linearly independent and spans the vector space V.
CLAIM
The set of class-vectors { [x]] [x=],, [x=),, [o=], ... } is independent. Since the only way ccro-class-vectee can be written linear combination of the set is by making all the coefficients \1; A2, λ3, zero, it can be seen in the following equation.
[0]
Au [x*]]
(34)
i goes to infinity it may span the vector space V as long as the feature space is one dimensional, but is not going be to verified here. If the feature space is not one dimensional then the data point æ is a vector. The following Basis is for more generalized setting.
If the features space is d-dimensional, then æ has a vector representation. And let a; represent the 7th element of the vector Then the zero-class- vector be expressed as shown in the Equation (35) if and only if all the coefficients Ag Anizs Ajiziis... are zero, it is not proved here but it is easy to understand looking at the equation.
[0]
Σ A 1 j==1
Σ
(35)
1=lj==1
1=11s==1
Therefore the set IV- [1ne]]
d) is independent. Hence any class-vector [f(x)) from the vector space V can be written as a linear combination of these vectors, assuming they span the Vector Space V, as the Equation (36).
E {I.....d)
[f(x)] = Σ
Σ
A
25 252l +
Σ
Anjsi 252 +
(36)
51=1
1= 1==1
==
for some Ajr Ajija Apiatas E R.
Using addition and scalar multiplication definitions the equation can be written in the following way.
d +
d
d
d
[f()] =
TH JI=1 12=1
A
+
Ta + +....
(37)
11=1
Let hl(x)) E [f(x)] logit equation. Therefore
d
d
Mxx) b+
Σ j1=1
An
+
(38)
12=1
j1=1 1==1
The Equation (38) is a multidimensional Taylor series equation. All this means that the logit equation of any class is a multidimensional Taylor series equation. Note that the verification of whether the set ly spans the Vector Space V has not been conducted. Now, this verification is unnecessary since the Equation (38) is a Taylor series. The logit equations are written as laylor series, and focus here is learn at least Valid Logit equation for every class. Furthermore, the set Tp 1. The Fjj Far Iú Ts ja {1,.... d} is independent and spans polynomial space, proof is not required.
13

---

Discovering Zero
The equivalent Tensor equation for the Equation (38) can be written as below.
h(x) = me + mii + m2ij Bi + Maijk T52yak +
(39)
where my, Mii mai Maijk are unknown scalars.
These unknown scalars are learned using the back propagation as in neural networks. This learning of a tensor equation using the back propagation is named Taylor-Series Based Network.
D. Metrics
As seen in Figure 6, two-dimensional data can be displayed on paper to visualize how the decision boundaries are formed when the Metta- Class data is used to train a network. This displaying of the results helps in comparing how the learning is different when the Metta-Class data incorporated into the training process versus when it not.
However, for higher dimensions, visualizing the data to the decision boundaries becomes challenging. There dimensionality reduction techniques, such Principal Component Analysis (PCA) t-Distributed Stochastic Neighbor Embedding (I-SNE), which can reduce the dimensionality of data for visualization purposes. These techniques effectively display the data in two or three dimensions, making it easier to understand the distribution and clustering of the data points.
Despite their usefulness visualizing the data, these dimensionality reduction techniques fall short when it comes to displaying the decision boundaries classification models. An example is given the Figure 14. As it can be seen in the third and fourth subfigure of Figure 14 the data points can be clustered and displayed in the lower dimensions, but the boundaries cannot be accurately displayed in the reduced dimensions.
0-0 1-0000--0 00
Figure 14. Diagram illustrating the limitations of PCA and 1-SNE for visualizing decision boundaries. The top-left subfigure shows example data, while the top-right suhfigure displays the decision regions learned by neural network. The bottom-left subfigure demonstrates PCA applied to reduce dimensionality, highlighting that is ineffective for visualizing decision boundaries. The bottom- right subfigure shows t-SNE applied for clustering lower dimensions, which, while effective grouping data, does not provide insight into decision regions.
Another possible way to visualize higher dimensions is by slicing higher dimensional space into two dimensional or three dimensional spaces and visualizing boundaries in these lower dimensional spaces. A good analogy is that of a CT scan. However, this method not efficient because the data naturally is very sparse in higher dimensional space.
Therefore clear that new metrics are needed measures the effect of the Metta-Clan data on the decision boundaries.
D.1. Metric-1
In Figure 15, figure-a is the learning of two different classifiers. The decision boundaries of Classifier At are better since the classifier does not classify any given data as one of the three classes(RED, GREEN, BLUE). So there must be a metric distinguish between these two types of learning. This distinction is possible by counting volume of the predicted classes on both Classifiers. Zero-Vector Data Points(Data sampled from the Metta-Class) Number of Zero-Vector points that are classified as non-Metta- Mon-Metta-Class by both classifiers and comparing them, is possible to distinguish right classifier from the left. Therefore, the following metric called Occupancy Factor defined below to reflect this property.
be used for this purpose. Counting
D. DEFINITION: OCCUPANCY FACTOR
the ratio of the Zero- Vector points that are classified as non-Metta-Clan to the total vector points.
14

---

Discovering Zero
Classifier-1 L
Classifier R
figure
Nou-Zero Duta Points (Label: RED, GREEN, BLUE) Volumes of Predicted Classes (Predictions: RED, GREEN BLUE)
Zero Data Points (Label: Zero- Ventor- Class)
figure b
figure-
Occupancy Factor
Total Samples of Zero Class
Figure Metric-1 I (Occupancy Factor): Figure-a illustrates learning process two different classifiers. The decision boundaries Classifier-R R are superior compared to those of Classifier-L, as it avoids classifying any data point as belonging one of the three explicit classes (RED, GREEN, BLUE). The Occupancy Factor serves a metric quantify this difference. Specifically, it measures volume of decision regions corresponding to the defined classes relative to the volume of the Metta-Class decision region.
Let N be the total Zero- Vector points, be number of Zero-Vector points classified as RED class, be the number of Zero-Vert Vector points that are classified as GREEN and nig as BLUE, for the Figure 15, then
Occupancy Factor ni + n2 + 73
(40)
N
15

---

Discovering Zero
Occupation Factor is 1 for any classifier that is trained without the Metta-Class data incorporated into the train data since N n1 + n2 ng.. Similarly, Occupation Factor of Classifier L is 1, and Occupation Factor for Classifier R is less than
1.
Classifier-1 L
Classifier-R
Volumes of Predicted Classes (Predictions: RED, GREEN, BLUE))
figure d
Non-Zero Data Points (Label: RED, GREEN, BLUE)
Zero Data Points (Label: Zero- Vector- Class)
Empty-Space
figure-e
Purity Factors :=
Figure 16. Metric-2 (Purity Factors): Figure-a illustrates the learning process of two different classifiers. Boundaries of the Classifier-R are superior compared with boundaries the Classifier as they represent the true nature classes. For example, 'Empty-Space' in the RED class of Classifier L also classified RED class. The Purity Factor serves metric to quantify this difference. Specifically, measures volume of true class with respect to volume of decision region the class.
D.2. Metric-2
In the next Figure 16, let figure-a is the learning of two different classifiers. Boundaries of the Classifier R are superior compared with boundaries of the Classifier-L L as they represent the true nature of the classes. For example, the 'Empty- Space' the RED class Classifier also classified as RED class. The Occupancy Factor can used to distinguish these two classifiers. However, it is possible to have data one class in the boundary of a different class; for instance, there can be blue data in the Red Volume. Therefore, another metric defined below address this issue.
16

---

Discovering Zero
1.2.1. DEFINITION: PURITY FACTORS
Purity Factor is defined for one class. It the accuracy of the class, For class defined the ratio of the total predictions of class-c that are labeled as class-c to the total predictions of class
Purity Factor of c Pi(Pod==e Label=e)
(41)
((Prod=cc
where
(e A Label=e) is number of data points that are predicted as class-c and also labeled as class
[((red=e) is the number of data points predicted as class C.
Using this definition the Purity Factors for the classifier-L are low compared with the classifiers R. Therefore the Purity Factors tell if the learned boundaries are true boundaries representing true nature of the class or not. Where as the Occupancy Factor tells total volume of predicted Classes space.
D.2.2. CAUTION
To compare any classifiers using the Occupancy Factor and the Purity Factors, keep the density of the Zero-Vectore Vector-points the same for both classifiers during testing. Therefore, the numbers for Occupancy Factor and Purity Factors are valid only if the number of veeoor data points along with the range of the space in which they are generated, is presented.
D.3. Understanding the Metrics through an Example
The three class data in the Figure 17 is used to to understand the behavior of the Purity Factors and Occupancy Factor.
The data is used for training two networks, the Metta-Class data is used to train only one of the networks. The results are presented in the following Figure 17. In subfigure 17a the boundaries learned are true boundaries representing true natures of the data. The Occupation factor is reaching zero, indicating that the volume occupied by the Predicted Class is small. Similarly the Purity Factors are high, indicating the empty space the each class is low.
In contrast, in subfigure 17b, results of classifier trained without incorporating the Metta- Netta-Class data, the boundaries do represent the true nature of the classes. They merely are separating surfaces. Hence, the Occupation Factor is always one. and purity factors are low due to the unwanted empty space present in the each class.
17

---

Discovering Zero
Www
M
(a) The Metta-Class is incorporated during training. The top-left subfigure illustrates the learned true boundaries. The top-right subfigure depicts the accuracy plot over 100 epochs. The bottom- left subfigure presents the purity factors, all of which approach one, indicating that the data within the decision boundaries becomes increasingly pure the training progresses. The bottom-right subfigure shows the occupancy factor converging towards zero, implying that the decision regions are progressively enclosing the classes more tightly.
Bataas
(b) The Metta-Class is not incorporated during training. The top-left subfigure shows the decision regions after 100 epochs. which fail to represent the true regions. These decision regions merely partition the space arbitrarily. Consequently, the occupancy factor remains throughout the training, the purity factors are approximately 0.5, as the number Zero-Vector data points and class data points in cach region are nearly equal.
Figure 17. Example to understand the Metrics.
18

---

Discovering Zero
E. Results
E.1. MNIST
Standard MNIST data set is used for the following results.
E.I.I. ONE CLASSIFIER FOR ALL MNIST CLASSES
The MNIST data is used for training two separate classifiers: one with incorporating the Metta-Class data (Zero-Inclusive- Network) and one without incorporating the Metta-Class Eew.luuuiiuueetwrrrkee Results shown in Figure 18 (Purity factors, Occupancy factors and Accuracies) of both classifiers side by side.
E.1.2. ONE CLASSIFIER FOR EACH MNIST CLASS
Unary classifier is trained on each MNIST class data including a Metta-Class data for that class. Results are sown in figures 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, That is each MNIST class data is separated and Metta-Class data is appended. the resultant data divided into train and test data, this train data is used train a classifier, that classifier is called unary classifier of that particular class. At the end these individual classes are used as repository as experiment. The repository accuracy is calculated at the end as shown in the Figure 41.
E.2. CIFAR10
Standard CIFAR10 data set is used for the following results.
1.2.1. ONE CLASSIFIER FOR ALL CIFAR10 10 CLASSES
The CIFARIO data is used for training two separate classifiers: one with incorporating the Metta-Class data (Zero-Inclusive- Network) and one without incorporating the Metta-Class IZrroExclusive-Neewool The results are shown in Figure 29 (Purity factors, Occupancy factors and Accuracies) of both classifiers side by side.
E.2.2. ONE CLASSIFIER FOR EACH CIFARIO CLASS
Unary classifier is trained on each CIFARIO class data including suitable Metta-Class data for that class. Results are sown in figures 30, 31, 32, 33, 34, 35, 36, 37, 38, 39. That is each CIFARIO class data is separated and Metta-Class data is appended, the resultant data is divided into train and test data, this train data is used train classifier, that classifier is called unary classifier of that particular class. At the end these individual classifiers are used as repository as experiment. The repository accuracy is calculated at the end as shown in the Figure 42.
E.3. Rover on an MNIST Planet
The Section 4.2.4 mentions example of a rover visiting an alien planet. Here it is implemented for the three classes as shown in the following Figure 43.
The following graphs are the results of the training of individual networks. The experiment conducted is as follows, has been assumed that the rover got to know MNIST data alone on that planet, so a network is trained for MNIST data alone and added to the repository of the MNIST. Later the Rover comes across the FashionMNIST data, new network trained on for the FashionMNIST data alone without disturbing the previously add Network in the repository, and added the newly trained network to the repository. At this point the accuracy of the repository is calculated. At last the rover reaches the KMNIST data, a new network is trained on that KMNIST data and added the new network to the repository. Again the repository accuracy calculated here. The following are the corresponding diagrams.
19

---

Discovering Zero
Purity Factors (Zero-inclusive-Net)
Purity Factors (Zero-Exclusive-Net)
0.8
0.8
class-0
class-1
0.6
0.6
clapo-3
class-4
class-5
0.4
class-1
0.4
class-6
cluss..
class-2
class-3
classe
0.2
class-4
0.2
class-3
class-6
class-7
class-8
class-9
Epochs
Epochh
Occupancy Factor (Zero-Inclusive-Net)
Occupancy Factor (Zero-Exclusive-Net)
0.4
0.4
0.2
0.2
0.0
0.0
100
100
Epoche
Epochs
Accuracies (Zero-Inclusive-Net)
Accuracies (Zero-Exclusive-Net)
100
150
80
00-
accurady accuray
accursoy
20
10
60
30 1.0
25
60
00
100
opoch
opuch
Figure The MNIST is used training two separate classifiers: one with incorporating Network) and one without incorporating the Metta-Class Pero-EExllsiveeNNetwrrr) When trained with the Metta-Class the purity is high as shown in the top two plots, occupancy is low as shown middle plots, and test accuracy same for both networks as shown bottom figures. (The Metta- Metta-Class data used calculate the train accuracy but not used in calculation of test accuracy)
detta-Class
-nclu-nne
20

---

Discovering Zero
Purty Becii (caeed)
Ocr-pancy feeir ytare
Locens (iaee
Figure 19. Training results of Unary Classifier of MNIST Class 0
102
Figure 20. Training results of Unary Classifier of MNIST Class 1
Aarity Beruus (caee
Occupancy Bartrr yybee
Longes Siloon
Figure 21, Training results of Unary Classifier of MNIST Class
Occupan Near blea
Useato Idaa d1
Accusscia (dland))
Figure 22, Training results of Unary Classifier of MNIST Class 3
Figure 23. Training results of Unary Classifier of MNIST Class 1
21

---

Discovering Zero
Purty Beii (caee
Ocr-pancy faeir (tare
Locens (1aee Sp
Figure 24, Training results of Unary Classifier of MNIST Class 5
352
Figure 25, Training results of Unary Classifier of MNIST Class 6
Aarity Beruus (caee
Occupance Bartrr yibee
Longes Silee
Figure 26. Training results of Unary Classifier of MNIST Class
Ny Neiin B0
Occupann Neva blea
Useato Idaa d1
Accusscia (dland)
Figure 27. Training results of Unary Classifier of MNIST Class 8
Figure 28. Training results of Unary Classifier of MNIST Class 9
22

---

Discovering Zero
Purity Factors (Zero-Inen leero-inllusive-Net
Purity Factors CerooExcluuive-Nees
0.6
0.6
class-0
dass-0
0.4
class-1
0.4
dass-1
dlass22
class-3
dass-
0.2
class-4
0.2
dass
dlass-5
class-6
class-8
class-7
dass
0.0
class-8
0.0
dass-8
class-9
class-9
20
30
40
50
20
30
40
50
Epochs
Epochs
Occupancy Factor (Zero-Inclusive-Net)
Occupancy Factor (Zero-Exclusive-Nett
0.4
0.4
0.2
0.2
0.0
0.0
20
30
40
10
20
30
40
Epochs
Epochs
Accuracies (Zero--nclusive--eee
Accuracies Ext--vxxlusive-Neet
100
100
80
50-
20
30
40
30
30
23
35
45
opoch
opoch
Figure 29, The CIFARIO data is used for training separate classifiers: with incorporating the Metta-Class data Inclusiee Network) and one without incorporating the Metta-Class shown in the top two plots, occupancy is low shown in the middle plots, and test accuracy is comparable with the network trained without the Metta-Class. (The Metta-Class data used to calculate the train accuracy but not used in calculation of test accuracy)
Seano-xcllsivv-Neeworr When trained with the Metta- Class the purity is high
23

---

Discovering Zero
Purty Tecsi (cseed)
Ocr-pancy faeir ytare
(daas-4)
Figure 30. Training results of Unary Classifier of CIFARIO Class
Maack
Figure 31. Training results of Unary Classifier of CIFARIO Class
Aarity Berurr (caee
Occupance Barter yibee
Figure 32. Training results of Unary Classifier of CIFARIO Class
Ny Necan (0
Occupann Near blea
Lseato Idaa d1
Accusaiia
Figure 33. Training results of Unary Classifier of CIFARIO Class 3
Figure 34. Training results of Unary Classifier of CIFAR10 Class 1
24

---

Discovering Zero
Purty Beesi (csee
Ocr-pancy faeir ytare
Locens (1aee Sp
Figure 35. Training results of Unary Classifier of CIFARIO Class
Louae Sdlned
Figure 36. Training results of Unary Classifier of CIFARIO Class 6
Aarity Berurr (caee
Occupance Bartrr yybee-
420
320
Figure 37. Training results of Unary Classifier of CIFARIO Class
Nyr Necan 10
Occupann Near blea
Lnsses S106s
Accusaiia
Figure 38. Training results of Unary Classifier of CIFARIO Class
3300
Figure 39. Training results of Unary Classifier of CIFAR10 Class 9
25

---

Discovering Zero
Purity Factors (Zera Exclusive Net)
Purity Factors (Zero inclusive Net)
08-
3609
0006-1
0.-
Epocts
Epeahs
Occusancy (Zero-ExdusiveeNNet
Factor (Zere-inclusive-Nett
1.0
08-
02-
02-
20
103
100
Spocts
Spachs
Accurades iZero- (Zero-Fxclusivee Exclusive-Nei Not)
Accuracies (Zero-nnduusive.aat)
42-
Renameay
24
104
100
(Zero-inclusive-Nett
2.38
3.75
Telt
3.50
1.5
3.25
1.20
Mw M A
0.58
0.8
0.25
0.2
8:35
Figure The ImageNet-Ik embeddings (extracted using Masked Autoencoder MAE) are used training two separate classifiers: one with incorporating the Metta-Class data Leero--nncusiie-eetweee Network). When trained with the Metta-Class the purity high as shown the top two plots, occupancy is low as shown the second row plots, and test accuracy same for both networks as shown third row plots. (The Metta-Class data used to calculate the train accuracy but not used in calculation of test accuracy).
and one without incorporating the Metta-Class Zero-Exelusive-
26

---

Discovering Zero
Logit Class-
Unary Classifier(NN) of Class-U
Unary Classifier(N)) of Class-1
Sigmoid (argmax)
Tost Data
Predictions
Accuracy
All Classes
Function
81.65
Labels
Unary Classifier(NN) (NN) of Class-9
Figure 41. Accuracy of the Repository of MNIST Unary Classifiers: Figure shown Method used for calculating combined accuracy of all individual classifiers, where each classifier trained to recognize specific MNIST class.
Unary CNN) Class-O
Unary (NN) of Class-1
Sigmoid (argmax)
Tost Data
of All Classes
Predictions
Aocuvary
Function:
25.39x
Labels
Unary Clussifer (NN) of Class
Figure 42. Accuracy of the Repository of CIFARIO Unary Classifiers: Figure shown Method used for calculating combined accuracy of all individual classifiers, where each classifier trained recognize specific CIFAR10 class.
27

---

Discovering Zero
Planet
Rover
KMNIST
MNIST
Fashion MNIST
Figure 43. The figure illustrates example of how continual learning can be implemented. Imagine rover landing on new planet, encountering different types data as explores various locations. A separate neural network can trained for each unique type data. By incorporating the Metta-Class during training, knowledge from different classifiers can be shared without needing to retrain the one big classifier for classes each time it encounters new data of different class.
Classifier of MNIST
Classifier of MNIST
Classifier of MNIST
Phalizzrnn
Phaltarre
Prafizzoo
MIST
Classifier of MNIIT
Classifier of FashionMNIST
MNNT date
MNSS
unpsmna
%67
%S8
KANNST
Classifier of KMNIST
DA Laba
JNNST PAR
Figure 44. Figure shows continuous lcarning on MINST datascts. The ler- figure shows a classifier trained only on MNIST, incorporat- ing the Metta-Class data during its training. In the middle, a new, separate classifier trained on FashionMNIST and the Metta-Class data added to the existing classifier. The far-right figure shows another new classifier trained on KMNIST and the Metta- Class data is added on top of the existing classifiers.
28

---

Discovering Zero
F. Equivalence set set of Tuples
As mentioned in the section 2 (Characterization of Logits) of the main paper, if the probability density function (PDF) of the data, f(x), is known, a given data point can be assessed for membership in the distribution by comparing its probability to a predefined threshold, ay. The threshold a should be chosen as a non-negative real number, ensuring that data points with high relative probability data. To generalize it further lets take fr()) is the threshold of the PDF ((x), and the tuple (J(x), fr())) defines that class of data. Consider a black- box operator that takes a function as input and returns threshold function suited for that PDF. In fact, an entire set of such tuples can represent the same class. For example, (f(z) + k, ff(()) + k) defines the same class-f for any veal-velued constant k. All such tuples preserve the decision boundary and thus represent the same underlying class.
recognized belonging the class. Therefore tuple (f(e), a4) defines that class (class- ff of
Let M be the set of monotonically non-decreasing functions. Then each tuple in the set
[f(x)] == {(M (x),, Mo (r())) M E M}
(42)
also defines the same class-f. Here, M f(x denotes (f(x)), written this way to reduce bracket clutter.
Definition of set V
Let V be the of sets of the kind shown in the equation 42. Therefore the set V can be written as
V [f(x)), (o(:)), [b(z))] ....}
(43)
Throughout this report, whenever the vector space V is mentioned, it refers to this vector space.
Definition of Addition and Scalar multiplication
As understood in the main paper, the equations for union and complement taken definitions here with slight modifications so that they are suitable for defining vector space. Specifically,
ADDITION
For all [f()), [g(z)] e V, the sum + is defined as [f(=]] + [g(x)] =
(44)
SCALAR MULTIPLICATION
For all X IR and for all [f(=)] E V the product (x) is defined as
A [[(=)) [A [x))
(45)
All the properties of a vector space on the set V, including the definitions of addition and scalar multiplication, can be verified similarly to what has been done in the main paper. Here, we focus specifically on the additive identity (the zero-vector). Let [Z (x)) is the identity of this vector space. Then by the definition of identity for any vector [f(x]] E V
[f()] + [[()) = [f()]
(46)
(/(x) + [(x)] = [/(x)]
(47)
(f+1)) [()1)(z) = [f())
(48)
(M o (f++)(s), Mo + +T(()) M = {(M o f(x), M or(o)) [MEM]
(49)
(50)
If T E M then above equality can be written as,
{(N f(x), N fr(2)) |N E M] = {(M f(x), M fr())) | E
(51)
Therefore, the additive identity (i.e., the zero-vector) of the vector space possesses the property of being monotonically non-decreasing function. Since every vector in V represents class (i.c., probability distribution over data), there must exist a probability density function (PDF) corresponding to the zero-vector that is monotonically non-decra.sing.
To understand the possible form of the zero-vector, we consider the following two extreme cases:
29

---

Discovering Zero
1. When the PDF is constant.
2. When the PDF is strictly monotonically increasing.
Case 1: PDF of the Kero-Vector is Constant
A constant PDF corresponds uniform distribution. Hence, if the zero-vector has constant PDF, the data associated with the zero-vector is uniformly distributed across its support.
Case 2: PDF is Strictly Monotonically Increasing
Let A be the volume spanned by PDF that is strietly monotonically increasing and has zero probability outside the volume V. Since PDF must integrate to one over its support, A - the PDF appears approximately constant within any finite, localized region. Therefore, samples drawn from such a PDF will appear uniformly distributed in small local areas.
From both Case and Case 2, it follows that data sampled from the zero-vector appears uniform within any finite local region. Hence, the Metta-Class (the Vero-Vector Class) effectively behaves as a uniform distribution.
NOTE:
Some technical details, such as the constraints the set M, have been omitted to streamline the intuition. Readers advised not to treat this as a formal proof but rather conceptual hint or intuition behind viewing the Zero- Vector as uniform distribution.
30


Reviews:
[{"review_id": "An4T6MfSqN", "reviewer": "Reviewer_gWUe", "summary": "The paper introduces a novel mathematical framework for understanding class representations in machine learning by defining classes as vectors in a vector space. The core idea revolves around the concept of a Zero-Vector Class, which corresponds to a data class with a uniform distribution. This conceptualization enables various applications, including clear learning, unary class learning, set operations on classes, continual learning, data generation, and equation discovery. The framework is demonstrated through experiments on the MNIST and CIFAR-10 datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the Reviewer for their detailed and constructive review. The questions raised are insightful and have helped us further refine and improve our work.\n\n# Q1. Stability of Zero-Vector Class Training\nWe observed no instability during training with the Zero-Vector Class. The Zero-Vector Class uses identical network architecture and loss function as standard training. Empirically, we find training stability improved when incorporating the Zero-Vector Class:\n - Purity Factor plots (e.g., \"Figure 18 shows MNIST, Figure 29 shows CIFAR-10\") are **more stable** during training when the Zero-Vector Class is included. \n- Test Accuracy plot (e.g., \"Figure 18 shows MNIST, Figure 29 shows CIFAR-10\") is stable irrespective of zero-vector class used in training or not. \n- No vanishing/exploding gradients and convergence issues as training metrics either improved or remained stable.\n\n# Q2. Differences from Energy-Based Models (EBMs)\n\nWe clearly distinguish our method from EBMs as follows:\n\n- **Representation:** EBMs learn a single global energy function representing data likelihood.\\\nOur method explicitly learns separate class logits, each class individually via a standard classification network. This results in distinct, interpretable per-class distributions.\n\n- **Training Stability:** EBMs typically require specialized training procedures (e.g., Langevin dynamics sampling), leading to potential instability or complex hyperparameter tuning. \\\nOur method uses standard supervised classification (cross-entropy loss) with the Zero-Vector Class, providing inherent training stability without extra sampling or specialized optimization.\n\t\n- **Generalization:** EBMs are susceptible to mode collapse, potentially failing to capture the complete data distribution.\\\nOur method explicitly learns the underlying data distribution through class logits (**Equation 27**), converging to the true distribution as the number of data points increases. This ensures comprehensive generalization to the true data manifold, thus avoiding mode collapse.\n\n# Scalability and Computational Efficiency\n\n## Computational Efficiency\nWe analyze computational complexity in two parts:\n\n1. Pre-softmax logits: complexity $O(f(n))$ for $n$ data points.\n2. Softmax calculation: $O(n \\cdot \\text{softmax}(c))$ for $c$ classes.\n\nAdding the Zero-Vector Class introduces one extra output node, adding overhead $O(nL^2)$, where $L$ is the preceding layer size.\n\nComplexities summarized:\n\n| Classifier | Training Cost | Inference Cost |\n|------------|---------------|----------------|\n| Zero-Exclusive | $O(f(n))+O(n\\cdot\\text{softmax}(c))\\approx O(f(n))$ | $O(f(n))+O(n\\cdot\\text{softmax}(c))\\approx O(f(n))$ |\n| Zero-Inclusive ($k$ extra points) | $O(f(n+k)+(n+k)L^2)+O((n+k)\\cdot\\text{softmax}(c+1))\\approx O(f(n+k))$ | $O(f(n)+nL^2)+O(n\\cdot\\text{softmax}(c+1))\\approx O(f(n))$ |\n\nTypically, $O(f(n))$ dominates, making overhead manageable, especially given improvements in purity.\n\n## Scalability\nInspired by successful embedding-based models (VQ-VAE, Stable Diffusion, MAE; typical dimensions: 512–1024), we explicitly tested scalability on embeddings (dimension:768) from Masked Autoencoder (MAE, ImageNet-1K):\n\n- Selected 10 random ImageNet classes, generated embeddings and corresponding Zero-Vector data.\n- Trained Zero-Inclusive (with Zero-Vector data) and Zero-Exclusive classifiers.\n\nResults confirm scalability:\n\n- **Purity Factors:** 0.7–1.0 for Zero-Inclusive Network vs. 0.1–0.9 for Zero-Exclusive Network, demonstrating improved boundaries.\n- **Accuracy:** Both networks achieved similar accuracy (~82\\%), confirming no accuracy degradation.\n\n[Click here to view additional results on ImageNet-1K embeddings.](https://drive.google.com/file/d/1lB6JvipbRdom8n-8U7JFScrIG1LXMhKV/view?usp=sharing)\n\n\nIn NLP Transformers, next-token prediction tasks classify over vocabulary, naturally permitting Zero-Vector Class integration. Typical NLP dimensions (~512) are lower than those already tested (e.g., MNIST:784), confirming practical feasibility. Explicit NLP experiments remain future work.\n\n# Response on Mathematical Rigor & Clarity:\nOur goal was to clearly convey the core intuition behind our framework and support it empirically. We agree formalization is valuable and have provided rigorous mathematical explanations in response to Reviewer **dsGt**, which we encourage you to read for more details.\n\n# Regarding Essential References:\nGiven the novelty of this approach, identifying directly related prior work was challenging for us as well. We hope this contribution opens a new line of exploration.\n\nWe sincerely appreciate your detailed review and hope our responses clarified key concerns. Feedback from other reviewers was positive; if our clarifications resolved your concerns, we would appreciate your reconsideration of the Overall Recommendation. Thank you for your valuable evaluation and time.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence", "methods_and_evaluation_criteria": "The proposed methods and/or evaluation criteria (e.g., benchmark datasets) make sense for the problem or application at hand", "theoretical_claims": "I did not check the correctness of any proofs for theoretical claims", "experimental_designs_or_analyses": "I check the soundness/validity of any experimental designs or analyses.", "relation_to_broader_scientific_literature": "The key contributions of the paper are not related to the broader scientific literature", "essential_references_not_discussed": "I am unfamilar with this area and not sure whether there are essential references that should be discussed", "other_strengths_and_weaknesses": "**Strengths**:\n1. Novel Theoretical Contribution: The paper presents an innovative way of treating classes as vectors, enabling set operations on class regions in a mathematically rigorous manner. The introduction of the Zero-Vector Class provides a fresh perspective on classification and decision boundaries.\n2. Improved Decision Boundaries (Clear Learning): The paper argues that incorporating the Zero-Vector Class leads to clearer and more realistic decision regions in neural networks, avoiding misclassification in empty feature space regions.\n3. Facilitates Unary Class Learning: The proposed approach allows neural networks to learn a single class in isolation, which can be useful in anomaly detection and other real-world applications where only positive examples are available.\n4. Continual Learning Without Retraining: The Zero-Vector Class allows for incremental learning without catastrophic forgetting, as new classes can be integrated without redefining previous decision boundaries.\n5. Potential for Data Generation and Equation Discovery:The method can be used to generate synthetic data by following gradients of logit functions. It can also help discover mathematical equations representing classes through Taylor-series-based learning.\n\n**Weaknesses**:\n1. Mathematical Rigor & Clarity: While the paper provides proofs and derivations, some sections could be presented in a more structured and rigorous manner. Certain notations and definitions (e.g., the validity of the equivalence sets and operations on them) could benefit from clearer formalization.\n2. Scalability to High-Dimensional Data: The framework's applicability to more complex datasets (e.g., ImageNet or NLP tasks) remains unclear. The Zero-Vector Class is primarily tested in lower-dimensional spaces; its efficiency in high-dimensional feature spaces needs further exploration.\n3. Computational Efficiency:The paper does not discuss the computational overhead of incorporating Zero-Vector Classes in training.\nThe potential trade-off between training time and classification accuracy should be analyzed.", "other_comments_or_suggestions": "See weaknesses", "questions_for_authors": "1. Stability of Zero-Vector Class Training: Given that Zero-Vector Class data is sampled from a uniform distribution, does this introduce instability or difficulties during training? Did you observe any vanishing/exploding gradients or convergence issues?\n2. The paper draws parallels between Zero-Vector Class training and Energy-Based Models (EBMs). How does the proposed method differ in terms of learned representations, training stability, and generalization?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bibR2tyYPq", "reviewer": "Reviewer_hzzz", "summary": "This paper introduced a novel technique to regularize neural networks' decision boundary by introducing the so-called \"zero-vector class\". The paper established some mathematical properties of the zero-vector class concept and derived a simple method to improve on the neural networks in different tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and for positively highlighting the theoretical contributions of our work.\n\n# Q. Regarding Sampling from Low-Dimensional Manifolds\nYour concern about sampling from low-dimensional manifolds is valid. Directly sampling from such manifolds is generally impractical. Crucially, however, our method explicitly does **not require manifold sampling**. Instead, we uniformly sample from the entire input space—a process that is both simple and ensures maximal entropy.\n\nWhy is this effective? Uniform sampling guarantees maximal entropy and thus explicitly represents regions of the input space associated with high uncertainty. These samples intentionally represent areas the classifier should not confidently assign to known classes. By introducing the Zero-Vector Class, we encourage the classifier to place decision boundaries precisely around true data manifolds, effectively learning the manifold structure indirectly. This is clearly demonstrated in Figure 6 (bottom row): without the Zero-Vector Class (bottom left), the classifier's boundaries incorrectly extend into empty regions. In contrast, with the Zero-Vector Class (bottom right), decision boundaries closely follow the true data distribution. This result validates our simple yet powerful sampling approach.\n\n# Regarding the Strength of Empirical Results\nWe respectfully disagree with the assertion that improvements from our method are marginal. The gains are substantial, measurable, and practically significant. Figure 6 visually demonstrates how the Zero-Vector Class dramatically improves decision boundary alignment with the true manifold. Moreover, quantitative evidence strongly supports our method:\n\n- In Figure 18 (MNIST), the purity factor for classes 3 and 8 drops below 5% without the Zero-Vector Class, meaning the classifier confidently misclassifies 95% of the region. With the Zero-Vector Class, purity exceeds 95\\% implies misclassifies only 5\\% of the region.\n- A similar pattern occurs in Figure 29 (CIFAR-10).\n\nHigh purity explicitly indicates decision regions closely align with true data manifolds. Achieving this high purity directly enables a variety of applications (e.g., single-class learning, continual learning, equation discovery), which are otherwise infeasible with traditional classification methods.\n\n# Scalability to High-Dimensional Data\nWe appreciate your question about scalability, and explicitly evaluated it through additional experiments inspired by the practical success of embedding-based models (e.g., VQ-VAE, Stable Diffusion, and Masked Autoencoder [MAE]). Specifically, we:\n\n- Selected embeddings (768-dimensional) from 10 random classes in ImageNet-1K using a pretrained MAE model.\n- Generated corresponding Zero-Vector Class data uniformly in this embedding space.\n- Trained two classifiers:\n  - **Zero-Inclusive Network** (with Zero-Vector data)\n  - **Zero-Exclusive Network** (without Zero-Vector data)\n\nResults demonstrate clear scalability and effectiveness:\n- **Purity Factor:**  \n  Zero-Exclusive Network purity fluctuates dramatically (0.1–0.9), indicating misclassification of empty space.  \n  Zero-Inclusive Network purity remains consistently high and stable (0.7–1.0).\n- **Accuracy:** \nBoth classifiers achieve comparable accuracy (~82%), confirming no accuracy penalty from the Zero-Vector Class.\n\nThese experiments clearly validate our method’s scalability to high-dimensional embedding spaces used by modern neural networks. [Click here to view additional results on ImageNet-1K embeddings.](https://drive.google.com/file/d/1lB6JvipbRdom8n-8U7JFScrIG1LXMhKV/view?usp=sharing)\n\n\nFor NLP applications, Transformer models typically operate in embedding spaces with dimensions (~512) lower than those we have already successfully tested (e.g., MNIST: 784). Since next-token prediction in NLP is formulated as a vocabulary classification task, incorporating the Zero-Vector Class as an additional vocabulary token is straightforward and promising. Explicit NLP experiments will be pursued in future work.\n\nWe believe these clarifications resolve the concerns raised. If you find these points convincing, we would appreciate a reconsideration of your Overall Recommendation. Thank you again for your valuable review and time, and the insightful question.", "claims_and_evidence": "The paper claims to have invented the notion of \"zero-vector class\" and applied it on classification problems. The concept is useful in many situations, such as single-class learning and continuous learning, and improving the decision boundary. The paper studied the empirical benefit in the Appendix which I have not been able to check.", "methods_and_evaluation_criteria": "The main part of the paper is consisted of building the formal framework of Valid Logit Functions and the vector space they constitute. There are few empirical claims discussed in the paper but their results were presented in the Appendix, which I have not checked in full.", "theoretical_claims": "The theoretical claims are the valid logit functions can be partitioned into equivalence classes and these classes form a vector space under certain conditions. It is proved in the main paper and I believe it is correct.", "experimental_designs_or_analyses": "This paper have some experimental results showing the empirical properties of adding a \"zero class\" in the classification task. Even though it is not a overwhelmingly better method compared to not adding the class, it did changed the neural network's decision boundary in some ways, and improved on the purity factor defined in the paper. However, it is unsure whether this approach could scale when the domain of tasks change.", "relation_to_broader_scientific_literature": "n/a", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "see above", "other_comments_or_suggestions": "n/a", "questions_for_authors": "In the paper, the author assumed that we could sample the \"zero class\" simply by drawing from a uniform distribution. However, it is quite common that the data distribution we care about lie on a low dimensional manifold (or a neighborhood of that manifold) which we can hardly sample efficiently from. For example, we can not easily sample random images besides Gaussian noises. I am curious how the authors would consider sampling from the \"zero class\" in those cases.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "soSgX2WNIi", "reviewer": "Reviewer_dsGt", "summary": "This paper proposes a mathematical framework for handling classes in datasets as vectors in a vector space, including a Zero-Vector Class which can be regarded as the absence of a class. They introduce all theoretical foundations and discuss two applications of their framework, namely \"clear learning\" and \"unary class learning\". The approach is validated using MNIST and CIFAR-10 datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your careful and meticulous review, especially your identification of small errors with exact line numbers; this level of detail is invaluable for improving the manuscript's clarity and quality.\n\n# Clarification on Training Procedure for Clear Learning\n\nThank you for raising this important point. We acknowledge the training procedure was not fully detailed in the original submission and appreciate the opportunity to clarify.\n\nClear Learning follows a standard supervised classification setup using Cross-Entropy loss and SGD optimization. The only change is the inclusion of the Zero-Vector Class. This is done by uniformly sampling points from the input space, labeling them as the Zero-Vector Class, and adding them to the training data. The classifier’s output layer includes one extra node to represent this class.\n\nAll other components—architecture, loss function, optimizer, and hyperparameters—remain identical to baseline training. This makes the method easy to integrate into existing pipelines with minimal changes.\n\nWe used standard network architectures appropriate for each task:\n- For MNIST and the 2D datasets (e.g., Figure~6), we used fully connected networks.\n- For CIFAR-10, a conventional convolutional neural network.\n- For the equation discovery task, a multidimensional Taylor-series-based classifier.\n\nGiven the range of tasks, we’ve included implementation code in the supplementary material and will release the full codebase upon acceptance. Training details will also be clarified in the final revision.\n\n# Assumptions about Zero-Vector Class (Mathematical Rigor \\& Clarity)\n\nWe appreciate your comment regarding the assumption that the Zero-Vector Class behaves like a uniform distribution and an additive identity. While our main goal was to convey the intuition leading to the framework, we agree that formalization is valuable. Your feedback helped us clarify this aspect more rigorously\n\nAs described in Section 2 (Characterization of Logits) of the main paper, if the data PDF is $f(x)$, a predefined threshold $\\alpha_{f}$, can determine class membership. Thus tuple $(f(x), \\alpha_{f})$ defines that class, say class$-f$. More generally, let $f_T(x)$ be the threshold of the PDF $f(x)$, and the tuple $(f(x), f_T(x))$ defines that class. Consider a black-box operator that takes a function as input and returns a threshold function suited for that PDF. For example, $(f(x) + k, f_T(x) + k)$ defines the same class-$f$ for any real-valued constant $k$. All such tuples preserve the decision boundary and thus represent the same underlying class.\n\n Let 𝓜 be the set of monotonically non-decreasing functions. Then each tuple in the set \n$[f(x)]:= $  {$( M \\circ f(x),\\ M \\circ f_T(x) ) \\big| M \\in 𝓜$ } also defines the same class-$f$. Here, $M \\circ f(x)$ denotes $M(f(x))$, written this way to reduce bracket clutter.\n\n**Definition of set V:** $V = $\\{$[f(x)], [g(x)], [h(x)], .... $}\n\n**Addition:**$\\forall_{[f(x)], [g(x)] \\in V}$ $[f(x)] + [g(x)] := [f(x) + g(x)]$\n\n**Scalar multiplication:**  $\\forall_{\\lambda \\in \\mathbb{R}}$ $\\forall_{[f(x)] \\in V}$ $\\lambda [f(x)] :=[\\lambda f(x)]$\n\nThe remaining vector space properties follow as in the main paper. Here, we focus on the additive identity (the zero-vector).\n\nLet $[I(x)]$ is the identity of this vector space. Then, for any vector $[f(x)]\\in V$\n\n$[f(x)] + [I(x)] = [f(x)] $ ⇒ $[f(x) + I(x)] = [f(x)]  $ ⇒ $[(f + I)(x)] = [f(x)]  $\nwhich implies:\n\n{$ (M ∘ (f + I)(x), M ∘ (f + I)_T(x))    |   M ∈ 𝓜$} $=$ {$ (M ∘ f(x), M ∘ f_T(x))  |  M ∈ 𝓜 $}\n\nIf $I ∈ 𝓜$, then the above holds trivially. Hence, the additive identity must be a monotonically non-decreasing function. Since every vector in $V$ corresponds to a class (i.e., a PDF), there must exist a PDF representing the zero-vector class that is monotonically non-decreasing. We now analyze its form:\n\n## Case 1: PDF of the Zero-Vector is Constant\nA constant PDF corresponds to a uniform distribution. Hence, the zero-vector class corresponds to a uniform distribution over its support.\n\t\n## Case 2: PDF is Strictly Monotonically Increasing\nLet $A$ be the volume spanned by a PDF that is strictly monotonically increasing and has zero probability outside the volume $A$. Since a PDF must integrate to one over its support, as $A \\to \\infty$, the PDF appears approximately constant within any finite, localized region. Thus, samples will appear uniform within any localized region.\n\nFrom both Case 1 and Case 2, it follows that data sampled from the zero-vector appears uniform within any finite local region. Hence, the Zero-Vector Class effectively behaves as a uniform distribution.\n\nWe invite you to look at the computational cost and scalability discussed in response to reviewer **gWUe**. If our clarifications have addressed your concerns, we would be grateful if you would consider updating your overall recommendation. Thank you again for your time and thoughtful review.", "claims_and_evidence": "The main content of this work is the theoretical foundation of the Zero-Vector Class. The authors provide well-supported evidence for their claims in the form of experiments and visualizations of their theoretical concept. However, there are some assumptions that lack evidence, e.g. the assumption that the Zero Vector Class acts like a uniform distribution and an additive identity. However, the emperical experiments show that the learning approaches work.", "methods_and_evaluation_criteria": "The proposed methods make sense for the problem at hand. They demonstrate the theoretical assumptions, but also reveal limitations in scalability, which is left for future work.", "theoretical_claims": "See above.", "experimental_designs_or_analyses": "The experiments and analyses are sound. However, the training procedure for the individual neural networks is not described in much detail.", "relation_to_broader_scientific_literature": "The authors contribute to a broad range of literature on the theoretical foundations of learning algorithms. In the future more experiments should be conducted for more diverse benchmarks beyond MNIST and CIFAR-10.", "essential_references_not_discussed": "There are no essential references not discussed that I know of.", "other_strengths_and_weaknesses": "The idea is very interesting and a great contribution. The theoretical assumptions and limitations concerning scalability should be investigated more in the future.", "other_comments_or_suggestions": "Small errors:\n- line 043: \"many techniques (...) that allow\" instead of allows\n- the alternation between upper and lower case is irritating (e.g. line 044: \"the Neural Network\", in the next column: \"If a neural network exhibits..\", \"Logit\" vs. \"logit\". \n- line 037 (right column): \"of the combined class's\" should be \"classes'\" I think\n- line 139: \"need not be an..\" should be \"does not need to be..\"\n- line 132 (right column): \"in the left-hand side\" should be \"on the left-hand side\"\n- Beginning of Section 3: there is an article missing \"set of Valid Logit..\"\n- When using equations the punctuation is missing\n- On the right column of page 4 the equations are not numbered\n- Instead of \"refer the subsection\" it should be \"see subsection ..\" \n- line 234 (right column): \"the PDF of [0] is considered\" instead of \"consider\"\n- line 310: \"the learning does not make much sense\"", "questions_for_authors": "- How does the training procedure for clear learning look like?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NgkX44uBCC", "reviewer": "Reviewer_vQK1", "summary": "The authors propose a mathematical framework for representing data classes as vectors in a vector space. The goal is to enable operations such as addition (class union) and scalar multiplication (class complement) to improve machine learning classification. The main contributions are: the introduction of the Zero-Vector Class, a novel approach to represent class boundaries, the application of this framework to enhance neural network learning, and improvements in classification accuracy and continual learning. The study shows that the Zero-Vector Class allows networks to learn the true data manifold, leading to clearer decision boundaries and better performance on classification tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your careful and constructive review, which motivated us to perform additional experiments on ImageNet-1K embeddings and to analyze computational complexity explicitly. Your feedback has significantly strengthened our paper.\n\n# Response on Scalability and Computational Efficiency\n\nWe first address computational complexity to support the discussion on scalability.\n\n## Q2. Computational Cost\nTo analyze this clearly, we divide the computational requirements for any $c$-class classifier into two distinct parts:\n\n1. Pre-softmax logit computation\n\n2. Softmax computation\n\nLet a **Zero-Exclusive Network** be a classifier that does not use the zero-vector class during training. Suppose there are $n$ training points, and let the computation needed to compute pre-softmax logits be $O(f(n))$—for instance, $f(n)$ might represent millions of computations. Computing the softmax function then takes $O(n \\cdot \\text{softmax}(c))$ computations, where $\\text{softmax}(c)$ represents the complexity of softmax over $c$ classes.\n\nNow consider modifying the Zero-Exclusive Network by adding an extra node at the output to classify the zero-vector class; we call this the **Zero-Inclusive Network**. In this scenario, we incorporate the zero-vector class in training. The addition of the new node introduces extra calculations dependent on the number of nodes in the preceding layer. Letting $L$ denote the number of nodes in the layer preceding this new node, the additional computation required is $O(nL^2)$.\n\nThe computational complexities during training and inference are summarized in the following table:\n\n|  Classifier  |  Training Cost  |  Inference cost |\n|----------------|-----------------------------------------------|------------------------------------------------|\n|  Zero-Exclusive Network (base classifier)  |  $O(f(n)) + O(n \\cdot \\text{softmax}(c))$ $\\approx O(f(n))$  | $O(f(n)) + O(n \\cdot \\text{softmax}(c)) $ $ \\approx O(f(n))$  |\n|  Zero-Inclusive Network ($k$ zero-vector data points in training)  |  $O(f(n+k) + (n+k)L^2) + O((n+k) \\cdot \\text{softmax}(c+1)) $ $\\approx O(f(n+k))$  |  $O(f(n) + nL^2) + O(n \\cdot \\text{softmax}(c+1)) $ $\\approx O(f(n))$  |\n\nTypically, $O(f(n))$ dominates $O(nL^2)$ since $L$ corresponds only to the final internal layer. There is a trade-off between computational overhead and purity improvement. This trade-off depends significantly on the intended applications leveraging the full potential of the Zero-Vector framework.\n\nWe will include this complexity analysis in the additional page for the final submission.\n## Q1. Scalability\n\nWe acknowledge scalability as a critical concern. Our motivation for exploring high-dimensional embeddings comes from the success of embedding-based models like VQ-VAE, Stable Diffusion, and MAE, where internal dimensions (512–1024) capture complex distributions effectively.\n\nTo test this, we conducted experiments on embeddings (dim: 768) from a pretrained MAE on ImageNet-1K:\n- Selected 10 random ImageNet classes and obtained embeddings using MAE\n- Generated corresponding Zero-Vector Class data within this embedding space.\n- Trained two classifiers:\n\t- **Zero-Inclusive Network** (trained with Zero-Vector Class data)\n\t- **Zero-Exclusive Network** (trained without Zero-Vector Class data)\n\nResults:\n- **Purity Factor:**\n\t- Zero-Exclusive Network purity fluctuated significantly (0.1–0.9), indicating substantial misclassification of empty regions in the feature space.\n\t- Zero-Inclusive Network purity remained consistently high and stable (0.7–1.0), indicating effective suppression of misclassification in empty regions.\n- **Accuracy:**\n\t- Both networks reached similar test accuracy (~82%), showing no degradation from the Zero-Vector Class.\n\nThese experiments demonstrate the practical scalability of our framework to high-dimensional feature spaces, directly addressing the reviewer’s concern. [Click here to view additional results on ImageNet-1K embeddings.](https://drive.google.com/file/d/1lB6JvipbRdom8n-8U7JFScrIG1LXMhKV/view?usp=sharing)\n\nSimilarly, in NLP Transformer models, next-token prediction is a classification over a fixed vocabulary, naturally allowing inclusion of the Zero-Vector Class as an extra token. While NLP experiments are left for future work, typical embedding sizes (~512) are smaller than those already tested (e.g., 784 for MNIST), supporting the method’s practical feasibility in NLP tasks.\n\n# Regarding References\nThis work explores a novel direction, and we found it challenging to identify direct precursors. Thank you for suggesting foundational works like word2vec and SVMs—we agree these are relevant in spirit and will incorporate such references in the final version. We welcome any further suggestions as well.\n\nWe believe these clarifications resolve the concerns raised. If you find these points convincing, we would appreciate a reconsideration of your Overall Recommendation. Thank you again for your valuable review and time.", "claims_and_evidence": "The authors claim that the Zero-Vector Class improves classification by enabling neural networks to learn the true data manifold. The evidence includes mathematical proofs establishing a vector space for class representation, experimental results on MNIST and CIFAR-10, and comparisons between models trained with and without the Zero-Vector Class. The main claims are: the Zero-Vector Class refines decision boundaries, it enables unary class learning, and it facilitates continual learning. The experiments support these claims by demonstrating reduced misclassification in empty feature space, improved performance in single-class training, and effective knowledge transfer in continual learning. However, the scalability claim is less supported, as results on CIFAR-10 indicate challenges in higher-dimensional spaces, suggesting the need for further validation on more complex datasets.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are well-suited for the problem of class representation in a vector space. The introduction of the Zero-Vector Class and the use of set operations on classes align with the goal of improving classification and continual learning. The evaluation criteria are also strong.", "theoretical_claims": "No major flaws were found in the provided proofs, but additional empirical verification on complex datasets would strengthen the theoretical claims.", "experimental_designs_or_analyses": "From my point of view, the experimental design appears sound.  \n\n1. The paper evaluates the Zero-Vector Class using well-known datasets (including MNIST and CIFAR-10) to test its effectiveness in classification.  \n2. The authors compare models trained with and without the Zero-Vector Class to evaluate its impact on decision boundaries and classification performance.  \n3. The introduction of the Occupancy Factor and Purity Factor provides a structured way to evaluate the clarity and accuracy of class boundaries.  \n4. The continual learning experiments demonstrate the potential of the Zero-Vector Class for modular and scalable learning without retraining entire models.", "relation_to_broader_scientific_literature": "​The paper's key contributions align with and extend existing concepts in machine learning and embedded representations. Representing data classes as vectors in a vector space is reminiscent of embedding techniques, which map high-dimensional data into lower-dimensional spaces while preserving meaningful semantic relationships. This approach facilitates operations like addition and scalar multiplication, akin to how embeddings capture semantic similarities in natural language processing.", "essential_references_not_discussed": "​The paper's key contributions are grounded in established concepts within machine learning, particularly vector space models and support vector machines (SVMs). Could you reference foundational works that have significantly influenced this domain, like word2vec: https://arxiv.org/abs/1301.3781 ?", "other_strengths_and_weaknesses": "Strengths:\n1. The paper exhibits notable strengths in its originality and potential significance. \n2. The authors introduce a novel mathematical framework that represents data classes as vectors in a vector space.\n3. This offers a fresh perspective on class representation in machine learning.\n4. The Zero-Vector Class allows neural networks to learn the true data manifold rather than just decision boundaries.\n\nWeakness:\n1. The scalability of the Zero-Vector Class to high-dimensional datasets is not thoroughly explored.\n2. Limited discussion on computational complexity and efficiency of the proposed method.", "other_comments_or_suggestions": "1. Overall the paper is well-written.\n2. While the method is evaluated on MNIST and CIFAR-10, a discussion on its applicability to higher-dimensional datasets would enhance its impact.", "questions_for_authors": "1. How does the Zero-Vector Class perform on high-dimensional datasets, and have you tested it beyond MNIST and CIFAR-10?\n2. What are the computational costs of integrating the Zero-Vector Class, and how does it compare in efficiency to standard classification methods?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces an innovative framework that models classes as vectors, enabling mathematically rigorous set operations on class regions. Building on this formulation, the authors propose the concept of a Zero-Vector Class, which offers a novel perspective on classification and decision boundaries. The incorporation of this zero class results in more compact and representative class acceptance regions and facilitates learning from a single class, making the approach well-suited for anomaly detection. Additionally, the Zero-Vector Class supports incremental learning without catastrophic forgetting, as new classes can be integrated without redefining existing decision boundaries. The method also shows promise for synthetic data generation by following gradients of logit functions and for discovering analytical class representations through Taylor-series-based learning. All of these contributions are substantiated by the experimental results. I strongly recommend the acceptance of this paper.
Decision: Accept (spotlight poster)