Text Classiﬁcation with Born’s Rule
Anonymous Author(s)
AfﬁliationAddress email
Abstract
This paper presents a text classiﬁcation algorithm inspired by the notion of superposition of states in quantum physics.
By regarding text as a superposition ofwords, we derive the wave function of a document and we compute the transitionprobability of the document to a target class according to Born’s rule.
Two complementary implementations are presented.
In the ﬁrst one, wave functions arecalculated explicitly with prior knowledge.
The second implementation embeds theclassiﬁer in a neural network architecture.
Both versions are self-explainable. Ourclassiﬁer achieves state-of-the-art accuracy (87.3%) on the 20Newsgroup datasetwith a training time of about 0.043s on CPU (0.001s on GPU), which is severalorders of magnitude faster than alternative approaches. The accuracy can be furtherimproved by the neural version. Similar results are also obtained on the R8 and
R52 datasets.
We discuss how these ideas are also applicable to non-textual data.
Introduction
In quantum physics, the superposition principle is the idea that a system can be in multiple states at thesame time (e.g., a cat is simultaneously alive and dead in the Schrödinger’s thought experiment [19]).
Here we show that text can be formally treated as a quantum system in a superposition of words.
By exploiting this analogy, we develop a supervised classiﬁer based on key postulates of quantummechanics, namely the Born rule [3].
The Born rule provides a link between the mathematical formalism of quantum theory and experiment,and as such is almost single-handedly responsible for practically all predictions of quantum physics.
In this paper, we represent documents and classes as quantum objects and we compute the probabilityof a document to collapse in a target class by applying the Born rule.
We implement our classiﬁcation algorithm in scikit-learn [14] and we embed the classiﬁer in a neuralnetwork architecture using pytorch [13].
All the code is available at <link to repository> (available inthe supplemental material for review).
The paper is structured as follows.
Section 2 presents the motivation of this work.
Section 3introduces the notation and some preliminary notions in quantum mechanics.
Section 4 develops ourclassiﬁcation algorithm.
Section 5 embeds the classiﬁer in a neural network architecture.
Section 6presents our empirical results.
Finally, Section 7 gives our concluding remarks and discussesextensions to non-textual data.
Motivation
In its most general form, the no free lunch theorem [20] implies that only prior knowledge makes itpossible to generalize from the training examples to novel test examples.
As quantum mechanics represents our understanding of nature at the deepest level [11], we wonder whether quantuminspired machine learning may incorporate a fundamental form of prior knowledge.
Our hopeis that such knowledge results in more efﬁcient algorithms, with better generalization ability andshorter computational times.
Such algorithms would be also beneﬁcial in terms of explainability andinterpretability [6], as their interpretation is immediately inherited from the physical model [15].
This work should not be confused with quantum computing, which harnesses the properties ofquantum states to perform calculations [1], or with quantum machine learning [2], which exploreshow to devise and implement quantum software that could enable machine learning that is faster thanthat of classical computers [4].
We refer to our work as quantum-inspired machine learning, in thatwe use quantum theory to derive machine learning algorithms that can run on classical computers.
Although this is not the ﬁrst time quantum theory is applied to non-microscopic domains, this is, toour knowledge, the ﬁrst supervised classiﬁcation algorithm based on quantum theory.
Here we focus on the representation of text as a superposition of words as the analogy with quantumsystems is straightforward.
The formalism we present is, however, rather general and the implementations we derive can be used in practice as general-purpose classiﬁers.
Extensions to non-textual dataare discussed in the conclusion.
Notation
Let x be a feature vector with elements x j for j = 1 , ..., J .
Let y be a probability vector withelements y k ≥ 0 for k = 1 , ..., K , and such that (cid:80)
k
y k
= 1 .
Our goal is to learn a function g suchthat
y = g ( x ) .
Then, given a test instance x (cid:48) , we predict the probabilities y (cid:48)
=
g ( x (cid:48) ) and select theclass k ∗ = argmax y (cid:48) k for classiﬁcation.
We start by giving some preliminary notions in quantum mechanics.
Wave function.
In quantum physics, a system is regarded as a superposition of states s and, using
Dirac’s notation [5], it is represented by a wave function | ψ (cid:105) :
[Equation]
=
[Equation]
s ψ s | s
[Equation]
with ψ s ∈ C .
Born rule.
In the Copenhagen interpretation, the modulus squared of the inner product is interpretedas the (unnormalized) probability of the wave function | ψ (cid:105) collapsing to a new wave function | ϕ (cid:105)upon measure of an observable:
[Equation]
where ¯ ϕ s denotes the complex conjugate of ϕ s .
This is known as the Born rule, and it is one of thefundamental postulates of quantum mechanics.
Wave coefﬁcients.
From (1) and (2), we notice that the coefﬁcients | ψ s | 2 represent the probabilityof the wave function | ψ (cid:105) to collapse in state | s (cid:105) :
[Equation]
Classiﬁcation Algorithm
Let the feature vector x contain only non-negative elements such that x j ≥ 0 for all j (e.g., wordcounts or tf-idf weights).
We regard x j as the (unnormalized) probability of the data instance (e.g.,document) to collapse in the j -th feature (e.g., word).
We represent the j -th feature as a quantum state| j (cid:105) , and we represent the data instance with a superposition of states | ψ (cid:105)
=
(cid:80)
j ψ j | j (cid:105) .
Accordingto (3) we have x j = P ( ψ → j ) =
| ψ j | 2 , which implies the natural choice ψ j [Equation] [Equation]
[Equation]
In a similar way, we represent each class k with a wave function | ϕ ( k ) (cid:105) =
(cid:80) j ϕ ( k ) j | j (cid:105) , and we obtainthe coefﬁcients ϕ ( k ) j by setting the transition probability from | ϕ ( k ) (cid:105) to | j (cid:105) equal to the conditionalprobability of j given k , which we write as P j | k .
According to (3) we have P j | k
= P ( ϕ ( k ) → j ) =|
ϕ ( k ) j | 2 , which implies the natural choice ϕ ( k ) j
[Equation]
[Equation]
Finally, we obtain the classiﬁcation probability y k by computing the probability of | ψ (cid:105) to collapse in| ϕ ( k ) (cid:105) .
By substituting (4) and (5) in (2), the unnormalized probabilities are:
[Equation]
and the normalized probabilities are y k
= u k / (cid:80) k u k .
4.1 Training
To obtain the conditional probability P j | k in (6) we proceed as follows.
Given a training set{ ( x ( n ) , y ( n ) ) } n =1 ,...,N , we normalize each feature vector x ( n ) such that it sums up to 1:
[Equation]
Then, we compute the conditional probability P j | k from the (unnormalized) joint probability P jk :
[Equation]
4.2 Regularization
We observe that if P j | k is a constant vector for a given j , then the j -th addend increases the summationin (6) by the same value for all classes k .
As u k is a monotonic transformation of the summation,the j -th addend does not alter the ranking of the probabilities u k , thus being irrelevant for theﬁnal classiﬁcation k ∗ = argmax y k
= argmax u k .
To regularize the predictions, we shrink thecontribution of irrelevant addends towards zero by re-weighting the summation in (6).
Let us rewrite, for ease of notation,
the probabilities P j | k
as some weights W jk ≥ 0 .
Then, j isirrelevant if W jk is constant for all k .
We normalize W jk such that the weights of the classes k sumup to 1 for each j , that is:
[Equation]
An irrelevant j maximizes the entropy H j = − (cid:80) k W k | j ln
( W k | j ) , as W k | j is uniformly distributedacross the classes k .
Thus, we introduce the following weights that range between 0 (irrelevant jwith maximum entropy) and 1 (relevant j with null entropy):
[Equation]
Finally, we use H j in (10) to re-weight the summation in (6), which becomes:
[Equation]
4.3 Generalization
To simplify ablation studies, we generalize (11) as follows:
[Equation]
where H j is given in (9)–(10), P jk is given in (8), and a > 0 , b ≥ 0 , and h ≥ 0 are the modelhyper-parameters.
Here, we are mainly interested in the choice
a = 12 , b = 1 , and h = 1 , whichcorresponds to the original model in (11).
Another interesting conﬁguration is a = 1 , b = 0 , andh = 0 , which corresponds to u k = (cid:80) j P k | j x j where P k | j is the conditional probability of k givenj .
This conﬁguration offers a natural benchmark for our quantum approach in that it computes u kaccording to classical probability theory.
4.4 Explainability
The contribution of the j -th feature to the total probability u k (local explanation) is given by theaddend H hj W ajk x aj in (12).
Therefore, the most inﬂuential feature for the classiﬁcation k ∗ is givenby j ∗ = argmax j H
h j W ajk ∗ x a j .
In general, we use H h j W ajk ∗ x a j to rank the features by the degree inwhich they contribute to the classiﬁcation k ∗ .
The explanation at the class level (global explanation) is obtained by investigating the product H hj W ajkin (12), regardless of the vector x .
The global most inﬂuential feature for each class k is given byj ∗ k = argmax j H hj W ajk .
In general, we use H hj W ajk to rank the features by their global importancewith respect to class k .
4.5 Computational Complexity
In the training phase, the algorithm in (11), and more generally (12), learns the joint probability P jkin (8) by multiplying the J × N matrix z ( n ) j with the N × K matrix y ( n ) k .
Entry ( j, k ) of P jk is givenby the inner product of the j -th row of the left matrix (which has N entries) and the k -th columnof the right matrix (which has N entries), so computing it takes time O ( N ) .
We do this once perelement.
Since P jk has dimension J × K , there are O ( JK ) elements to consider and the total workis done in O ( NJK ) .
That is, the training time is at most linear in the number of samples ( N ), in thenumber of features ( J ), and in the number of classes ( K ).
In the prediction phase, we compute u k (and y k ) in (11), and more generally (12).
As these are allelementwise operations on the J × K matrix P jk , the total work is done in time
O ( JK ) .
That is, theprediction time is at most linear in the number of features ( J ), and in the number of classes ( K ), andit does not depend on the number of training samples ( N ).
Finally, we notice that all the operations involved in the training and prediction phases can be easilyparallelized (e.g., on GPUs).
Thus, we expect the method to be highly scalable.
Neural Architecture
A major limitation of the algorithm presented in Section 4 is that it can be applied only when x j ≥ 0 .
Here we embed the method in a more ﬂexible architecture that admits x j ∈ C .
Let us assume that a data instance (e.g., document) can be represented as a superposition of somehidden states | s (cid:105) for s = 1 , ..., S (e.g., word embeddings).
Then, we write its wave function| ψ (cid:105) =
(cid:80) s
ψ s | s (cid:105) where the coefﬁcients ψ s generally depend on the feature vector x (e.g., words).
Werepresent such coefﬁcients with a neural network ψ s = ψ s ( x ) that maps the feature vector x ∈ C Jto the wave function ψ ∈ C S .
Then, we write the wave function of class k as | ϕ ( k ) (cid:105) =
(cid:80) s ϕ
( k ) s | s (cid:105)where the coefﬁcients ϕ ( k ) s depend on k and s , but not on the feature vector x .
Finally, we use the Born rule in (2) to compute the probability of | ψ (cid:105) to collapse in | ϕ ( k ) (cid:105) :
[Equation]
Equation (13) is read as a neural network u
= σ (Φ v ) where Φ = Φ ks = ¯ ϕ ( k ) s is a matrix of weights,v = v s = ψ s ( x ) is the output of the previous layer, Φ v denotes the matrix product (cid:80) s Φ
ks v s , andthe activation function σ ( · ) = | · | 2 is the modulus squared.
Finally, we apply a normalization layerto obtain the probabilities y k
= u k / (cid:80) k u k ∈ [0 , 1] .
An illustration is given in Figure 1.
Our method is similar to the usual approach where the ﬁrst part of a network is assumed to learnlog-probabilities, and the ﬁnal classiﬁcation layer converts them into probabilities by applying thesoftmax function.
Here, the ﬁrst part of the network is assumed to learn the coefﬁcients of a wavefunction, and the ﬁnal classiﬁcation layer converts them into probabilities by applying the Born rule.
5.1 Initialization
We initialize the weights Φ ks such that the wave function | ϕ ( k ) (cid:105) has equal probability to collapsein any state | s (cid:105) .
To this end, let us write the complex-valued weights Φ ks
= ρ ks e iθ ks where i isthe imaginary unit, ρ ks ≥ 0 , and θ ks ∈ [0 , 2 π ) .
We recall from (3) that the probability of | ϕ ( k ) (cid:105)to collapse in | s (cid:105) is P ( ϕ ( k ) → s ) =
| ϕ
( k ) s | 2 = | Φ ks |
=
ρ 2 ks .
We set ρ 2 ks = ρ 2 so that the(unnormalized) probability is constant for all k and s .
To normalize the probability, we set Sρ 2 = 1 ,where S is the number of states (input dimension of the layer).
We obtain ρ = 1 / √ S , whichresembles the initialization in [7] and [8].
Finally, we sample θ ks from a uniform distribution in theinterval [0 , 2 π ) , such that the weights Φ ks are uniformly distributed in the complex circle (isotropy).
[Equation]
When the feature vector x is a probability vector as in Section 4, then (11) can be written as the neuralnetwork in (13) where we use S =
J and ψ s ( x ) = √ x s .
In this case, it is interesting to initialize theweights in (13) with the corresponding weights developed in (11), that is Φ ks
= H s (cid:112) P s | k .
5.2 Explainability
We notice that the probabilities y k are invariant under scaling and rotation of the coefﬁcients ¯ ϕ ( k ) sin (13).
To show that, we multiply ¯ ϕ ( k ) s by a scaling factor ρ and a phase factor e iθ .
Then, wesubstitute ¯ ϕ ( k ) s → ρe iθ ¯ ϕ ( k ) s in (13).
The phase factor e iθ vanishes when computing the modulus andthe scaling factor ρ vanishes when the probabilities are normalized.
The invariance by scaling and rotation implies that the weights Φ ks have no absolute meaning, andthey become meaningful only in relation with each other.
To inspect the relations among states s , fora given class k , it is interesting to visualize the weights Φ ks in the complex plane (with unlabelledaxes).
Here, the length of the vectors can be used to rank the states by importance, while the directionof the vectors produce constructive or destructive interference among the states (see Figure 4 for anillustration).
For local explanations, Φ ks v s is used instead of Φ ks .
Empirical Results
We illustrate the performance of our classiﬁer using three well-established text classiﬁcation benchmarks: 20Newsgroup 4 , and the R8 and R52 subsets of Reuters 21578 5 .
We perform tokenizationusing the function nltk.word_tokenize 6 and vectorize the text with TﬁdfVectorizer 7 .
No other text transformation or cleaning procedure is performed.
The ﬁnal datasets are composed by ( 20Newsgroup )20 classes, 204 817 words, 11 314 training documents, and 7 532 test documents; ( R8 ) 8 classes,33 593 words, 5 485 training documents, 2 189 test documents; ( R52 ) 52 classes, 38 132 words,6 532 training documents, and 2 568 test documents.
All the results are obtained using Python 3.9 on a Google Cloud Virtual Machine equipped with
CentOS 7, 12 vCPU Intel Cascade Lake 85 GB RAM, 1 GPU NVIDIA Tesla A100, and CUDA 11.5.
6.1 Training time, prediction time, and accuracy score
We compare our methodology against a baseline of six classiﬁers on the 20Newsgroup dataset.
With
Born Classiﬁer ( BC ), we refer to the algorithm presented in Section 4, where weights are computedas in equation (8) and classiﬁcation probabilities are calculated as in equation (11).
The baseline iscomposed by the algorithms Decision Tree (DT) , K-Nearest Neighbors (KNN) , Random Forest (RF) ,
Support Vector Machine (SVM) , Multinomial Naive Bayes (MNB) , and Logistic Regression (LR) .
Forall the algorithms in the baseline, we use the corresponding implementation in scikit-learn .
All theclassiﬁers are executed on CPU with default parameters.
The comparison between BC and the baseline is reported in Figure 2, where we show the trainingtime, prediction time, and accuracy score in function of the fraction of data used for training in 10independent executions.
Figure 2 shows that BC is fast to train, is fast to predict, can be acceleratedon GPU, and it achieves the highest accuracy regardless of the size of the dataset.
Moreover, as theamount of training data decreases, the accuracy gap between BC and the other classiﬁers furtherwidens.
6.2 Hyper-parameters and ablation study
We now compare the overall accuracy score and runtime of BC with the results of the ﬁne-tunedbaseline (see Table 1).
We then conduct an ablation study on BC (see Figure 3).
In Table 1, we tune the baseline classiﬁers via grid-search using 5-fold cross-validation.
Exceptfor BC , which needs no tuning, all the other classiﬁers use between 20 and 50 combinations ofhyper-parameters (reported in the replication code).
BC still achieves the highest accuracy by aremarkable margin.
Furthermore, the overall time required to classify the dataset is several orders ofmagnitude smaller than than that required by the baseline algorithms.
This is partly due to the speedof execution of BC (see Figure 2) and partly due to the fact that BC has no hyper-parameters to tune.
[Table]
The ablation study is reported in Figure 3.
Here we display the test accuracy scores of the modelcorresponding to equation (12) for several values of the hyper-parameters a , b , and h .
The conﬁguration a = 12 , b = 1 , h = 1 , corresponds to BC in equation (11) and it achieves an accuracy of 87.3%.
Selecting the hyper-parameters via cross validation achieves a test accuracy of 86.8%, suggesting thatour prior generalizes well.
The classical conﬁguration a = 1 , b = 0 , h = 0 , discussed in Section 4.3,achieves an accuracy of 83.3%.
Removing the regularization ( h = 0 ) from our model leads to a largedrop in performance: from 87.3% to 77.7%.
On the other hand, adding the regularization ( h = 1 )to the classical conﬁguration doesn’t lead to any performance gain: from 83.3% to 82.0%.
Theseresults can be understood by recalling that the classical conﬁguration uses the conditional probabilityof the classes given the features ( P k | j ), while BC uses the conditional probability of the featuresgiven the classes ( P j | k ).
As P j | k gives larger weight to more frequent features, then BC is dominatedby noise when the features irrelevant for classiﬁcation are also the most frequent (as it is typical intext).
Regularization becomes important to reduce the inﬂuence of frequent features that are slightlyskewed.
Instead, in the classical conﬁguration, regularization is redundant as P k | j already limitsthe inﬂuence of such features.
Finally, it is interesting to note that P k | j gives larger weight to morefrequent classes so that the classical conﬁguration is affected by imbalanced data.
As P j | k has theeffect of balancing the sample, we expect BC to work out-of-the-box on imbalanced data.
6.3 Embedding in a neural network architecture
With Born Layer ( BL) , we refer to the neural architecture displayed in Figure 1 that takes in input ψ ( x )and returns the classiﬁcation probabilities according to equation (13).
Here we choose ψ s ( x ) = √ x sso that the transformation in equations (11) and (13) are the same.
BL is initialized as in (14) and it isoptimized on the L 1 -loss using Adam [10] with default parameters.
We also initialize BL with theweights computed by BC as discussed at the end of Section 5.1.
We refer to this version with BC+BL .
In Table 2, we extend the comparison to a variety of deep learning approaches reported in literature,including cooperative neural networks ( CoNN ) [18], models that learn distributed representationsof entities and documents from a knowledge base ( TextEnt ) [22], graph convolutional networks( TextGCN ) [18], the neural attentive bag-of-entities model ( NABoE ) [21], and the diversiﬁed ensembleneural network ( DEns ) [17].
Our results show that BC achieves state-of-the-art accuracy on the20Newsgroup dataset and it is up to a million times faster than alternative approaches (0.001s).
Also BL matches state-of-the-art performance on 20Newsgroup , but it requires signiﬁcantly longercomputational times (about 34s).
Initializing BL with BC requires only few training epochs to furtherimprove performance, suggesting that BC+BL can be ﬁne-tuned relatively fast (about 3s).
While 20Newsgroup is almost balanced in terms of documents per class, R8 and R52 are not.
Inparticular, the most frequent class in R52 contains 2 840 training samples, while the least frequentclass contains only 1 document in the training set.
As BC implicitly balances the sample (seediscussion in Section 6.2), we expect it to achieve lower accuracy (as in Table 2) but higher scoresthat correct for class imbalance.
To illustrate with empirical data, we ﬁnd that TextGCN [23] achievesa macro-averaged F1-score of 67.7% on R52 , while BC achieves 70.1%.
[Table]
[Table]
[Table]
[Table]
The runtime is the (GPU) time to train the model.
The accuracy score is the average score achieved by the model on the test set in 10 independent runs.
Standard deviations are below 0.1 (%) and are omitted from the table.
The runtime for CoNN is taken from the original paper [18].
The runtime for the other methods are obtained by running the corresponding code on our hardware infrastructure.
6.4 Explanation
To exemplify the explanation provided by BC , in Table 3, we extract the global weights described in
Section 4.4 and we report the 10 most inﬂuential words for a subset of classes in 20Newsgroup .
Weobserve that the words appear semantically correlated to their respective class and do not containneither noisy words, such as stopwords or punctuation, or words whose meaning is too general to berepresentative of a speciﬁc class. [Table] cica [Table]
LC
[Table]
key
[Table] C BJDES [Table]
NDIS Amanda [Table] In Figure 4, we train BL for two epochs and we represent the evolution of the complex-valued weightsdescribed in Section 5.2.
The ﬁgure represents the 10 weights that have the largest (and smallest)modulus for the class rec.sport.baseball at the end of the training.
Starting from weights that areinitialized randomly in all directions of the complex circle, BL learns to give larger weights to morerelevant features, smaller weights to less relevant features, and to orientate the features in differentdirections.
The weights converge in the same direction for words that are speciﬁcally related to theconcept of baseball .
Interestingly, the words hockey and NHL (National Hockey League), whichare still relevant for a different class of sports, are associated with a large weight but they are alsocollocated in the opposite direction with respect to the baseball-related words.
Conclusion
By regarding text as a superposition of words, we have introduced a supervised classiﬁcationalgorithm based on key postulates of quantum mechanics.
The classiﬁer is self-explainable, admitsa closed-form expression (11), and can be embedded in neural networks architectures (13).
Ourmodel has achieved, or matched, state-of-the-art accuracy on three well-established text classiﬁcationbenchmarks. Unlike sophisticated approaches that require long computational times, involve nontrivial data pre-processing, exploit semantics, embeddings, and external knowledge bases, our modelis fast to train, avoids any text transformation and it relies on a bag-of-words (superposition-of-words)representation of the document.
We are unable to explain why our method seems to have a betterperformance compared to traditional algorithms, unless we postulate that quantum theory representssome form of prior knowledge that can be successfully exploited in machine learning. In this sense,we ﬁnd quantum-inspired machine learning a promising avenue for future research.
There are several potential improvements and extensions to this work, such as investigating theperformance of the method on non-textual data.
Whenever a data instance can be represented with aprobability distribution over some space, then a wave function can be constructed explicitly and thedata may be classiﬁed with the algorithm in Section 4.
More in general, an arbitrary data instance canbe classiﬁed with the network in Section 5, where the ﬁrst part of the network learns the coefﬁcientsof a wave function and the ﬁnal classiﬁcation layer converts them into probabilities by applyingthe Born rule.
For instance, an image may be classiﬁed with the network in (13) where ψ ( x ) is aconvolutional layer; this corresponds to regarding the image as a superposition of the features learntby the convolution. Using the transformation in (13) as the last classiﬁcation layer of a neural networkoffers a natural way to push our framework beyond a bag-of-words representation of the data in orderto leverage semantics, embeddings, and more sophisticated architectures for textual and non-textualdata.
A particularly interesting research direction would be constructing deep networks that apply the transformation in (13) repeatedly.
A deep architecture of this kind would ﬁnd its biological foundations inthe quantum brain hypothesis [9], which suggests that quantum events could play a non-trivial role inneuronal cells, and contribute to an extremely high complexity, variability and computational powerof neuronal dynamics.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See Section {+
+}
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
We don’tsee any potential negative societal impact other than those that apply to ML in general.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] The code isprovided in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Datasets (20NG, R8,R52) and software (sklearn, pytorch) are cited properly.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The implementation of our classiﬁer(s) is provided in the supplemental material andbelongs to the authors.
We will make the code available on Github upon publication.
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table] using/curating?
[N/A]
[Table] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable [Table] information or offensive content?
[N/A] [Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] ) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A] [Table]
) Did you describe any potential participant risks, with links to Institutional Review
[Table] Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]