Text Classiﬁcation with Born’s Rule
Anonymous Author(s)
AfﬁliationAddress email
Abstract
This paper presents a text classiﬁcation algorithm inspired by the notion of superposition of states in quantum physics.
By regarding text as a superposition of words,we derive the wave function of a document and we compute the transition probability of the document to a target class according to Born’s rule.
Two complementaryimplementations are presented.
In the ﬁrst one, wave functions are calculatedexplicitly.
The second implementation embeds the classiﬁer in a neural networkarchitecture.
Through analysis of three benchmark datasets, we illustrate severalaspects of the proposed method, such as classiﬁcation performance, explainability,and computational efﬁciency.
These ideas are also applicable to non-textual data.
Introduction
In quantum physics, the superposition principle is the idea that a system can be in multiple states at thesame time (e.g., a cat is simultaneously alive and dead in the Schrödinger’s thought experiment [22]).
Here we show that text can be formally treated as a quantum system in a superposition of words.
By exploiting this analogy, we develop a supervised classiﬁer based on key postulates of quantummechanics, namely the Born rule [4].
The Born rule provides a link between the mathematical formalism of quantum theory and experiment,and as such is almost single-handedly responsible for practically all predictions of quantum physics.
In this paper, we represent documents and classes as quantum objects and we compute the probabilityof a document to collapse in a target class by applying the Born rule.
The paper is structured as follows.
Section 2 presents the motivation of this work.
Section 3introduces the notation and some preliminary notions in quantum mechanics.
Section 4 develops ourclassiﬁcation algorithm.
Section 5 embeds the classiﬁer in a neural network architecture.
Section 6presents our empirical results that use a bag-of-words (superposition-of-words) representation of thedocument.
Finally, Section 7 gives our concluding remarks and discusses extensions to semanticsand non-textual data.
To simplify extensions to this work, we implement our classiﬁcation algorithm in scikit-learn [16] andwe embed the classiﬁer in a neural network architecture using pytorch [15].
All the code is availableat <link to repository> (available in the supplemental material for review).
Motivation
In its most general form, the no free lunch theorem [23] implies that only prior knowledge makes itpossible to generalize from the training examples to novel test examples.
As quantum mechanicsrepresents our understanding of nature at the deepest level [13], we wonder whether quantuminspired machine learning may incorporate a fundamental form of prior knowledge.
Our hope is that such knowledge results in more efﬁcient algorithms, with better generalization ability andshorter computational times.
Such algorithms would be also beneﬁcial in terms of explainability andinterpretability [8], as their interpretation is immediately inherited from the physical model [17].
This work should not be confused with quantum computing, which harnesses the properties ofquantum states to perform calculations [1], or with quantum machine learning [3], which exploreshow to devise and implement quantum software that could enable machine learning that is faster thanthat of classical computers [6].
We refer to our work as quantum-inspired machine learning, in thatwe use quantum theory to derive machine learning algorithms that can run on classical computers.
Although this is not the ﬁrst time quantum theory is applied to machine learning, this is, to ourknowledge, the ﬁrst step to develop a new general-purpose algorithm for supervised classiﬁcationbased on key postulates of quantum mechanics, namely the Born rule.
Here we focus on text classiﬁcation as the analogy with quantum systems is straightforward.
However, the formalism we present is general and the implementations we derive can be used inpractice as general-purpose classiﬁers.
The main contribution of this paper is to cast the Born rule asa general input-output transformation for supervised classiﬁcation.
Notation
Let x be a feature vector with elements x j for j = 1 , ..., J .
Let y be a probability vector withelements y k ≥ 0 for k = 1 , ..., K , and such that (cid:80)
k
y k
= 1 .
Our goal is to learn a function g suchthat
y = g ( x ) .
Then, given a test instance x (cid:48) , we predict the probabilities y (cid:48)
=
g ( x (cid:48) ) and select theclass k ∗ = argmax k y (cid:48) k for classiﬁcation.
We start by giving some preliminary notions in quantum mechanics.
Wave function.
In quantum physics, a system is regarded as a superposition of states s and, using
Dirac’s notation [7], it is represented by a wave function | ψ
[Equation]
[Equation]
ψ
[Equation]
Born rule.
In the Copenhagen interpretation, the modulus squared of the inner product is interpretedas the (unnormalized) probability of the wave function | ψ (cid:105) collapsing to a new wave function | ϕ (cid:105)upon measure of an observable:
[Equation]
where ¯ ϕ s denotes the complex conjugate of ϕ s .
This is known as the Born rule, and it is one of thefundamental postulates of quantum mechanics.
Wave coefﬁcients.
From (1) and (2), we notice that the coefﬁcient | ψ s | 2 represents the (unnormalized) probability of the wave function | ψ (cid:105) to collapse in state | s (cid:105) :
[Equation]
Classiﬁcation Algorithm
Let the feature vector x contain only non-negative elements such that x j ≥ 0 for all j (e.g., wordcounts or tf-idf weights).
We regard x j as the (unnormalized) probability of the data instance (e.g.,document) to collapse in the j -th feature (e.g., word).
We represent the j -th feature as a quantum state| j (cid:105) , and we represent the data instance with a superposition of states | ψ (cid:105)
=
(cid:80)
j ψ j | j (cid:105) .
Accordingto (3) we have x j = P ( ψ → j ) =
| ψ j | 2 , which implies the natural choice ψ j = √ x j .
[Equation]
In a similar way, we represent the k -th class with a wave function | ϕ ( k ) (cid:105) =
(cid:80) j ϕ ( k ) j | j (cid:105) , and weobtain the coefﬁcients ϕ ( k ) j by setting the transition probability from | ϕ ( k ) (cid:105) to | j (cid:105) equal to theconditional probability of feature j given class k , which we write as P j | k .
According to (3) we have
P j | k = P ( ϕ ( k ) → j ) =
| ϕ ( k ) j | 2 , which implies the natural choice ϕ ( k )
j
[Equation]
Finally, we obtain the classiﬁcation probability y k by computing the probability of | ψ (cid:105) to collapse in| ϕ ( k ) (cid:105) .
By substituting (4) and (5) in (2), the unnormalized probabilities are:
[Equation]
and the normalized probabilities are y k
= u k / (cid:80) k u k .
4.1 Training
To obtain the conditional probability P j | k in (6) we proceed as follows.
Given a training set{ ( x ( n ) , y ( n ) ) } n =1 ,...,N , we normalize each feature vector x ( n ) such that it sums up to 1:
[Equation]
Then, we compute the conditional probability P j | k from the (unnormalized) joint probability P jk :
[Equation]
4.2 Regularization
We observe that if P j | k is constant for k = 1 , ..., K , then the j -th addend increases the summationin (6) by the same value for all classes.
As u k is a monotonic transformation of the summation,the j -th addend does not alter the ranking of the probabilities u k , thus being irrelevant for theﬁnal classiﬁcation k ∗ = argmax k y k
= argmax k u k .
To regularize the predictions, we shrink thecontribution of irrelevant addends towards zero by re-weighting the summation in (6).
Let us rewrite, for ease of notation,
the probabilities P j | k
as some weights W jk ≥ 0 .
Then, j isirrelevant if W jk is constant for all k .
We normalize W jk such that the weights of the classes k sumup to 1 for each j , that is: [Equation]
[Equation]
An irrelevant j maximizes the entropy H j = − (cid:80) k W k | j ln
( W k | j ) , as W k | j is uniformly distributedacross the classes k .
Thus, we introduce the following weights that range between 0 (irrelevant jwith maximum entropy) and 1 (relevant j with null entropy):
[Equation]
Finally, we use H j in (10) to re-weight the summation in (6), which becomes:
[Equation]
4.3 Generalization
To simplify ablation studies, we generalize (11) as follows:
[Equation]
where H j is given in (9)–(10), P jk is given in (8), and a > 0 , b ≥ 0 , and h ≥ 0 are the modelhyper-parameters.
Here, we are mainly interested in the choice
a = 12 , b = 1 , and h = 1 , whichcorresponds to the original model in (11).
Another interesting conﬁguration is a = 1 , b = 0 , andh = 0 , which corresponds to u k = (cid:80) j P k | j x j where P k | j is the conditional probability of k givenj .
This conﬁguration offers a natural benchmark for our quantum approach in that it computes u kaccording to classical probability theory.
4.4 Explainability
The contribution of the j -th feature to the total probability u k (local explanation) is given by theaddend H hj W ajk x aj in (12).
Therefore, the most inﬂuential feature for the classiﬁcation k ∗ is givenby j ∗ = argmax j H
h j W a jk ∗ x a j .
In general, we use H h j W a jk ∗ x a j to rank the features by the degree inwhich they contribute to the classiﬁcation k ∗ .
The explanation at the class level (global explanation) is obtained by investigating the product H hj W ajkin (12), regardless of the vector x .
The global most inﬂuential feature for each class k is given byj ∗ k = argmax j H hj W ajk .
In general, we use H hj W ajk to rank the features by their global importancewith respect to class k .
4.5 Computational Complexity
In the training phase, the algorithm in (11), and more generally (12), learns the joint probability P jkin (8) by multiplying the J × N matrix of elements z ( n ) j with the N × K matrix of elements y ( n ) k .
Entry P jk is given by the inner product of the j -th row of the left matrix (which has N entries) andthe k -th column of the right matrix (which has N entries), so computing it takes time O ( N ) .
We dothis once per element.
Since the output matrix has dimension J × K , there are O ( JK ) elements toconsider and the total work is done in O ( NJK ) .
That is, the training time is at most linear in thenumber of samples ( N ), in the number of features ( J ), and in the number of classes ( K ).
In the prediction phase, we compute u k (and y k ) in (11), and more generally (12).
As these are allelementwise operations on the J × K matrix of elements P jk , the total work is done in time
O ( JK ) .
That is, the prediction time is at most linear in the number of features ( J ), and in the number ofclasses ( K ), and it does not depend on the number of training samples ( N ).
Finally, we notice that all the operations involved in the training and prediction phases can be easilyparallelized (e.g., on GPUs).
Thus, we expect the method to be highly scalable.
Neural Architecture
A major limitation of the algorithm presented in Section 4 is that it can be applied only when x j ≥ 0 .
Here we embed the method in a more ﬂexible architecture that admits x j ∈ C .
Let us assume that a data instance (e.g., document) can be represented as a superposition of somehidden states | s (cid:105) for s = 1 , ..., S (e.g., word embeddings).
Then, we write its wave function| ψ (cid:105) =
(cid:80) s
ψ s | s (cid:105) where the coefﬁcients ψ s generally depend on the feature vector x (e.g., words).
We represent such coefﬁcients with a neural network ψ s = ψ s ( x ) that maps the feature vectorx ∈ C J to the vector of wave coefﬁcients ψ ∈ C S .
Then, we write the wave function of class k as| ϕ ( k ) (cid:105) =
(cid:80) s ϕ
( k ) s | s (cid:105) where the coefﬁcients ϕ ( k ) s depend on k and s , but not on the feature vector x .
Finally, we use the Born rule in (2) to compute the probability of | ψ (cid:105) to collapse in | ϕ ( k ) (cid:105) : [Equation]
[Equation]
Equation (13) is read as a neural network u
= σ ( Φv ) where Φ is a matrix of elements Φ ks = ¯ ϕ ( k ) s ,v = ψ ( x ) is the output of the previous layer, Φv denotes the matrix product (cid:80) s Φ ks v s , and theactivation function σ ( · ) =
| · | 2 is the modulus squared.
Finally, we apply a normalization layer toobtain the probabilities y k
= u k / (cid:80) k u k ∈ [0 , 1] .
An illustration is given in Figure 1.
Our method is similar to the usual approach where the ﬁrst part of a network is assumed to learnlog-probabilities, and the ﬁnal classiﬁcation layer converts them into probabilities by applying thesoftmax function.
Here, the ﬁrst part of the network is assumed to learn the coefﬁcients of a wavefunction, and the ﬁnal classiﬁcation layer converts them into probabilities by applying the Born rule.
5.1 Initialization
We initialize the weights Φ ks such that the wave function | ϕ ( k ) (cid:105) has equal probability to collapsein any state | s (cid:105) .
To this end, let us write the complex-valued weights Φ ks
= ρ ks e iθ ks where i isthe imaginary unit, ρ ks ≥ 0 , and θ ks ∈ [0 , 2 π ) .
We recall from (3) that the probability of | ϕ ( k ) (cid:105)to collapse in | s (cid:105) is P ( ϕ ( k ) → s ) =
| ϕ
( k ) s | 2 = | Φ ks |
=
ρ 2 ks .
We set ρ 2 ks = ρ 2 so that the(unnormalized) probability is constant for all k and s .
To normalize the probability, we set Sρ 2 = 1 ,where S is the number of states (input dimension of the layer).
We obtain ρ = 1 / √ S , whichresembles the initialization in [9] and [10].
Finally, we sample θ ks from a uniform distribution in theinterval [0 , 2 π ) , such that the weights Φ ks are uniformly distributed in the complex circle (isotropy). [Equation]
[Equation]
When the feature vector x is a (unnormalized) probability vector as in Section 4, then (11) can bewritten as the neural network in (13) where we use S =
J and ψ s ( x ) = √ x s .
In this case, it isinteresting to initialize the weights in (13) with the corresponding weights developed in (11), that isΦ ks
= H s (cid:112) P s | k .
5.2 Explainability
We notice that the probabilities y k are invariant under scaling and rotation of the coefﬁcients ¯ ϕ ( k ) sin (13).
To show that, we multiply ¯ ϕ ( k ) s by a scaling factor ρ and a phase factor e iθ .
Then, wesubstitute ¯ ϕ ( k ) s → ρe iθ ¯ ϕ ( k ) s in (13).
The phase factor e iθ vanishes when computing the modulus andthe scaling factor ρ vanishes when the probabilities are normalized.
The invariance by scaling and rotation implies that the weights Φ ks have no absolute meaning, andthey become meaningful only in relation with each other.
To inspect the relations among states s , fora given class k , it is interesting to visualize the weights Φ ks in the complex plane (with unlabelledaxes).
Here, the length of the vectors can be used to rank the states by importance, while the directionof the vectors produce constructive or destructive interference among the states (see Figure 4 for anillustration).
For local explanations, Φ ks v s is used instead of Φ ks .
Empirical Results
We illustrate several aspects of our classiﬁer using three well-established text classiﬁcation benchmarks: 20Newsgroup 5 , and the R8 and R52 subsets of Reuters 21578 6 .
We perform tokenizationusing the function nltk.word_tokenize 7 and vectorize the text with TﬁdfVectorizer 8 .
No other text transformation or cleaning procedure is performed.
The ﬁnal datasets are composed by ( 20Newsgroup )20 classes, 204 817 words, 11 314 training documents, and 7 532 test documents; ( R8 ) 8 classes,33 593 words, 5 485 training documents, 2 189 test documents; ( R52 ) 52 classes, 38 132 words,6 532 training documents, and 2 568 test documents.
All the results are obtained using Python 3.9 on a Google Cloud Virtual Machine equipped with
CentOS 7, 12 vCPU Intel Cascade Lake 85 GB RAM, 1 GPU NVIDIA Tesla A100, and CUDA 11.5.
6.1 Training time, prediction time, and accuracy score
We compare our methodology against a baseline of six classiﬁers on the 20Newsgroup dataset.
With
Born Classiﬁer ( BC ), we refer to the algorithm presented in Section 4, where weights are computedas in equation (8) and classiﬁcation probabilities are calculated as in equation (11).
The baseline iscomposed by the algorithms Decision Tree (DT) , K-Nearest Neighbors (KNN) , Random Forest (RF) ,
Support Vector Machine (SVM) , Multinomial Naive Bayes (MNB) , and Logistic Regression (LR) .
Forall the algorithms in the baseline, we use the corresponding implementation in scikit-learn .
All theclassiﬁers are executed on CPU with default parameters.
The comparison between BC and the baseline is reported in Figure 2, where we show the trainingtime, prediction time, and accuracy score in function of the fraction of data used for training in 10independent executions.
Figure 2 shows that BC is fast to train, is fast to predict, can be acceleratedon GPU, and it achieves the highest accuracy regardless of the size of the dataset.
Moreover, as theamount of training data decreases, the accuracy gap between BC and the other classiﬁers widens.
6.2 Hyper-parameters and ablation study
We now compare the overall accuracy score and runtime of BC with the results of the ﬁne-tunedbaseline on 20Newsgroup (see Table 1).
We then conduct an ablation study on BC (see Figure 3).
In Table 1, we tune the baseline classiﬁers via grid-search using 5-fold cross-validation.
Exceptfor BC , which needs no tuning, all the other classiﬁers use between 20 and 50 combinations ofhyper-parameters (reported in the replication code).
BC still achieves the highest accuracy by aremarkable margin.
Furthermore, the overall time required to classify the dataset is several orders ofmagnitude smaller than than that required by the baseline algorithms.
This is partly due to the speedof execution of BC (see Figure 2) and partly due to the fact that BC has no hyper-parameters to tune.
[Table]
The ablation study is reported in Figure 3.
Here we display the test accuracy scores of the modelcorresponding to equation (12) for several values of the hyper-parameters a , b , and h .
The conﬁguration a = 12 , b = 1 , h = 1 , corresponds to BC in equation (11) and it achieves an accuracy of 87.3%.
Selecting the hyper-parameters via cross validation achieves a test accuracy of 86.8%, suggesting thatour prior generalizes well.
The classical conﬁguration a = 1 , b = 0 , h = 0 , discussed in Section 4.3,achieves an accuracy of 83.3%.
Removing the regularization ( h = 0 ) from our model leads to a largedrop in performance: from 87.3% to 77.7%.
On the other hand, adding the regularization ( h = 1 )to the classical conﬁguration doesn’t lead to any performance gain: from 83.3% to 82.0%.
Theseresults can be understood by recalling that the classical conﬁguration uses the conditional probabilityof the classes given the features ( P k | j ), while BC uses the conditional probability of the featuresgiven the classes ( P j | k ).
As P j | k gives larger weight to more frequent features, then BC is dominatedby noise when the features irrelevant for classiﬁcation are also the most frequent (as it is typical intext).
Regularization becomes important to reduce the inﬂuence of frequent features that are slightlyskewed.
Instead, in the classical conﬁguration, regularization is redundant as P k | j already limits theinﬂuence of such features.
6.3 Embedding in a neural network architecture
With Born Layer ( BL) , we refer to the neural architecture displayed in Figure 1 that takes in input ψ ( x )and returns the classiﬁcation probabilities according to equation (13).
Here we choose ψ s ( x ) = √ x sso that the transformation in equations (11) and (13) are the same and they can be directly compared(in general, ψ ( x ) may be an arbitrary transformation to exploit e.g., semantics, as discussed in Section 7).
BL is initialized as in (14) and it is optimized on the L 1 -loss using Adam [12] withdefault parameters.
We also initialize BL with the weights computed by BC as discussed at the end of Section 5.1.
We refer to this version with BC+BL .
We stress-test BC and BL (which only rely on word counts) against a variety of deep learningapproaches reported in literature (which exploit more sophisticated representations of the document). In Table 2, we include cooperative neural networks ( CoNN ) [20], models that learn distributedrepresentations of entities and documents from a knowledge base ( TextEnt ) [25], graph convolutionalnetworks ( TextGCN ) [20], the neural attentive bag-of-entities model ( NABoE ) [24], and the diversiﬁedensemble neural network ( DEns ) [19].
Surprisingly, we ﬁnd that BC achieves state-of-the-art accuracy on the 20Newsgroup dataset and itis up to a million times faster than alternative approaches (0.001s).
This conﬁrms that the weightscomputed explicitly in (11) are a good prior.
Also BL matches state-of-the-art performance, but itrequires signiﬁcantly longer computational times (about 34s).
Initializing BL with BC requires few training epochs to further improve performance, suggesting that BC+BL can be ﬁne-tuned relativelyfast (about 3s).
Finally, we notice that, while BL optimizes complex-valued weights, BC+BL usesreal-valued weights. This suggests that the reason for good performance lies in the transformationitself (13), rather than in the usage of complex numbers. [Table] [Table]
[Table]
TextGCN [Table] [Table] NABoE [24]
[Table] [Table] [Table] [Table]
[Table] epochs) [Table] [Table]
[Table]
epoch) 86.
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
The runtime is the (GPU) time to train the model.
The accuracy score is the average score achieved by the model on the test set in 10 independent runs.
Standard deviations are below 0.1 (%) and are omitted from the table.
The runtime for CoNN is taken from the original paper [20].
The runtime for the other methods are obtained by running the corresponding code on our hardware infrastructure.
6.4 Explanation
[Table]
To exemplify the explanation provided by BC , in Table 3, we extract the global weights described in
Section 4.4 and we report the 10 most inﬂuential words for a subset of classes in 20Newsgroup .
Weobserve that the words appear semantically correlated to their respective class and do not containneither noisy words, such as stopwords or punctuation, or words whose meaning is too general to berepresentative of a speciﬁc class.
[Table]
( rec.autos ), Graphics ( comp.graphics ), Macintosh ( comp.sys.mac.hardware ), Windows ( comp.os.mswindows.misc ), Cryptography ( sci.crypt ).
In Figure 4, we train BL for two epochs and we represent the evolution of the complex-valued weightsdescribed in Section 5.2.
The ﬁgure represents the 10 weights that have the largest (and smallest) modulus for the class rec.sport.baseball at the end of the training.
Starting from weights that areinitialized randomly in all directions of the complex circle, BL learns to give larger weights to morerelevant features, smaller weights to less relevant features, and to orientate the features in differentdirections.
The weights converge in the same direction for words that are speciﬁcally related to theconcept of baseball .
Interestingly, the words hockey and NHL (National Hockey League), whichare still relevant for a different class of sports, are associated with a large weight but they are alsocollocated in the opposite direction with respect to the baseball-related words.
Conclusion
By regarding text as a superposition of words, we have introduced a supervised classiﬁcationalgorithm based on key postulates of quantum mechanics, namely the Born rule.
The classiﬁer isself-explainable, admits a closed-form expression (11), and can be embedded in neural networkarchitectures (13).
Our method outperforms traditional algorithms on the 20Newsgroup , R8 , and R52datasets, in terms of classiﬁcation performance and computational time. There are several potential improvements and extensions to this work.
On the theoretical side, it isinteresting to understand the reasons for good performance. Our results suggests that the reason liesin the transformation itself (13), but how this can be explained by learning theory remains an openquestion.
On the empirical side, it is interesting to encode semantics and to investigate the performance ofthe method on non-textual data.
Whenever a data instance can be represented with a probabilitydistribution over some space, then a wave function can be constructed explicitly and the data may beclassiﬁed with the algorithm in Section 4.
More in general, an arbitrary data instance can be classiﬁedwith the network in Section 5, where the ﬁrst part of the network learns the coefﬁcients of a wavefunction and the ﬁnal classiﬁcation layer converts them into probabilities by applying the Born rule.
Using the transformation in (13) on top of existing architectures offers a natural way to push ourframework beyond a bag-of-words (superposition-of-words) representation of the data in order toleverage semantics or extend the method to non-textual data. For instance, an image may be classiﬁedwith the network in (13) where ψ ( x ) is a convolutional layer (this corresponds to regarding the imageas a superposition of the features learnt by the convolution).
A particularly interesting research direction would be constructing deep networks that apply the transformation in (13) repeatedly.
A deep architecture of this kind would ﬁnd its biological foundationsin the quantum brain hypothesis [11], which suggests that quantum events could play a non-trivialrole in neuronal cells, and contribute to an extremely high complexity, variability and computationalpower of neuronal dynamics.
Checklist
For all authors...(
a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Yes] (b) Did you describe the limitations of your work?
[Yes] See Section[-
-]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] We don’t
see any potential negative societal impact other than those that apply to ML in general.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] The code is provided in the supplemental material.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes](d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Datasets (20NG, R8, R52) and software (sklearn, pytorch) are cited properly.(b)
Did you mention the license of the assets?
[N/A](c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The implementation of our classiﬁer(s) is provided in the supplemental material andbelongs to the authors.
We will make the code available on Github upon publication.(d)
Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A](e)
Did you discuss whether the data you are using/curating contains personally identiﬁableinformation or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable? [N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
[Figure]
Appendix
[Figure]
A.1 Imbalanced data
[Figure]
While 20Newsgroup is almost balanced in terms of documents per class, R8 and R52 are not. Inparticular, the most frequent class in R52 contains 2 840 training samples, while the least frequentclass contains only 1 document in the training set. Figure 5 reports the F1-macro score for BC andthe baseline on the three datasets, which are increasingly imbalanced. BC outperforms the baselinemodels and the performance gap widens for more imbalanced data.
The native capability of BC towork with imbalanced data can be traced back to (8), which computes the conditional probability ofthe features given the classes.
Dividing the joint probability by the marginal effectively normalizesby the class imbalance.