Text Classiﬁcation with Born’s Rule
Abstract
This paper presents a text classiﬁcation algorithm inspired by the notion of superposition of states in quantum physics.
By regarding text as a superposition of words, we derive the wave function of a document and we compute the transition probability of the document to a target class according to Born’s rule.
Two complementary implementations are presented.
In the ﬁrst one, wave functions are calculated explicitly.
The second implementation embeds the classiﬁer in a neural network architecture.
Through analysis of three benchmark datasets, we illustrate several aspects of the proposed method, such as classiﬁcation performance, explainability, and computational efﬁciency.
These ideas are also applicable to non-textual data.
Introduction
In quantum physics, the superposition principle is the idea that a system can be in multiple states at the same time (e.g., a cat is simultaneously alive and dead in Schrödinger’s thought experiment [22]).
Here we show that text can be formally treated as a quantum system in a superposition of words.
Our superposition-of-words model is a bag-of-words model where each word is treated as a quantum state.
By exploiting this representation, we develop a supervised classiﬁer based on key postulates of quantum mechanics, namely the Born rule [4].
The Born rule provides a link between the mathematical formalism of quantum theory and experiment, and as such is almost single-handedly responsible for practically all predictions of quantum physics.
In this paper, we represent documents and classes as quantum objects and we compute the probability of a document to collapse in a target class by applying the Born rule.
The paper is structured as follows.
Section 2 presents the motivation of this work.
Sectionintroduces the notation and some preliminary notions in quantum mechanics.
Section 4 develops our classiﬁcation algorithm.
Section 5 embeds the classiﬁer in a neural network architecture.
Sectionpresents our empirical results.
Finally, Section 7 gives our concluding remarks and discusses extensions to semantics and non-textual data.
To simplify extensions to this work, we implement our classiﬁcation algorithm in scikit-learn [17] and we embed the classiﬁer in a neural network architecture using pytorch [16].
All the code is available at https://github.com/eguidotti/bornrule .
Motivation
In its most general form, the no-free-lunch theorem [23] implies that only prior knowledge makes it possible to generalize from the training examples to novel test examples.
As quantum mechanics represents our understanding of nature at the deepest level [13], we wonder whether quantuminspired machine learning may incorporate a fundamental form of prior knowledge.
Our hope is that such knowledge results in more efﬁcient algorithms, with better generalization ability and shorter computational times.
Such algorithms would be also beneﬁcial in terms of explainability and interpretability [8], as their interpretation is immediately inherited from the physical model [18].
This work should not be confused with quantum computing, which harnesses the properties of quantum states to perform calculations [1], or with quantum machine learning [3], which explores how to devise and implement quantum software that could enable machine learning that is faster than that of classical computers [6].
We refer to our work as quantum-inspired machine learning, in that we use quantum theory to derive machine learning algorithms that can run on classical computers.
Although this is not the ﬁrst time quantum theory is applied to machine learning (see e.g., [15]), this is, to our knowledge, the ﬁrst step to develop a new general-purpose algorithm for supervised classiﬁcation based on key postulates of quantum mechanics, namely the Born rule.
Here we focus on text classiﬁcation, as the analogy with quantum systems is straightforward.
However, the formalism we present is general and the implementations we derive can be used in practice as general-purpose classiﬁers.
Notation
Let x be a feature vector with elements x j for j = 1 , ..., J .
Let y be a probability vector with elements y k ≥ 0 for k = 1 , ..., K , and such that (cid:80)
k
y k
= 1 .
Our goal is to learn a function g such that y = g ( x ) .
Then, given a test instance x (cid:48) , we predict the probabilities y (cid:48)
=
g ( x (cid:48) ) and select the class k ∗ = argmax k y (cid:48) k for classiﬁcation.
We start by giving some preliminary notions in quantum mechanics.
Wave function.
In quantum physics, a system is regarded as a superposition of states | s (cid:105) and, using
Dirac’s notation [7], it is represented by a wave function | ψ (cid:105) :
[Equation]
Born rule.
In the Copenhagen interpretation, the modulus squared of the inner product is interpreted as the (unnormalized) probability of the wave function | ψ (cid:105) collapsing to a new wave function | ϕ (cid:105) :
[Equation]
where ¯ ϕ s denotes the complex conjugate of ϕ s .
This is known as the Born rule, and it is one of the fundamental postulates of quantum mechanics.
Wave coefﬁcients.
From (1) and (2), we notice that the coefﬁcient | ψ s | 2 represents the (unnormalized) probability of the wave function | ψ (cid:105) to collapse in state | s (cid:105) :
[Equation]
Classiﬁcation Algorithm
Let the feature vector x contain only non-negative elements such that x j ≥ 0 for all j (e.g., word counts or tf-idf weights).
We regard x j as the (unnormalized) probability of the data instance (e.g., document) to collapse in the j -th feature (e.g., word).
We represent the j -th feature as a quantum state | j (cid:105) , and we represent the data instance with a superposition of states | ψ (cid:105) =
(cid:80)
j ψ j | j (cid:105) .
According to (3) we have x j = P ( ψ → j ) =
| ψ j | 2 , which implies the natural choice ψ j = √ x j .
[Equation]
In a similar way, we represent the k -th class with a wave function | ϕ ( k ) (cid:105) =
(cid:80) j ϕ ( k ) j | j (cid:105) , and we obtain the coefﬁcients ϕ ( k ) j by setting the transition probability from | ϕ ( k ) (cid:105) to | j (cid:105) equal to the conditional probability of feature j given class k , which we write as P j | k .
According to (3) we have
P j | k = P ( ϕ ( k ) → j ) =
| ϕ ( k ) j | 2 , which implies the natural choice ϕ ( k )
j
[Equation]
Finally, we obtain the classiﬁcation probability y k by computing the probability of | ψ (cid:105) to collapse in | ϕ ( k ) (cid:105) .
By substituting (4) and (5) in (2), the unnormalized probabilities are:
[Equation]
and the normalized probabilities are y k
= u k / (cid:80) k u k .
4.1 Training
To obtain the conditional probability P j | k in (6) we proceed as follows.
Given a training set { ( x ( n ) , y ( n ) ) } n =1 ,...,N , we normalize each feature vector x ( n ) such that it sums up to 1:
[Equation]
Then, we compute the conditional probability P j | k from the (unnormalized) joint probability P jk :
[Equation]
4.2 Regularization
We observe that if P j | k is constant for k = 1 , ..., K , then the j -th addend increases the summation in (6) by the same value for all classes.
As u k is a monotonic transformation of the summation, the j -th addend does not alter the ranking of the probabilities u k , thus being irrelevant for the ﬁnal classiﬁcation k ∗ =
argmax k y k
= argmax k u k .
To regularize the predictions, we shrink the contribution of irrelevant addends towards zero by re-weighting the summation in (6).
Let us rewrite, for ease of notation,
the probabilities P j | k
as some weights W jk ≥ 0 .
Then, j is irrelevant if W jk is constant for all k .
We normalize W jk such that the weights of the classes k sum up to 1 for each j , that is:
[Equation]
An irrelevant j maximizes the entropy H j = − (cid:80) k W k | j ln
( W k | j ) , as W k | j is uniformly distributed across the classes k .
Thus, we introduce the following weights that range between 0 (irrelevant j with maximum entropy) and 1 (relevant j with null entropy):
[Equation]
Finally, we use H j in (10) to re-weight the summation in (6), which becomes:
[Equation]
4.3 Generalization
To simplify ablation studies, we generalize (11) as follows:
[Equation]
where H j is given in (9)–(10), P jk is given in (8), and a > 0 , b ≥ 0 , and h ≥ 0 are the model hyper-parameters.
Here, we are mainly interested in the choice
a = 12 , b = 1 , and h = 1 , which corresponds to the original model in (11).
Another special conﬁguration is a = 1 , b = 0 , and h = 0 , which corresponds to u k = (cid:80) j P k | j x j where P k | j is the conditional probability of k given j .
This conﬁguration offers a natural benchmark for our quantum approach in that it computes u k according to classical probability theory.
4.4 Explainability
The contribution of the j -th feature to the total probability u k (local explanation) is given by the addend H hj W ajk x aj in (12).
Therefore, the most inﬂuential feature for the classiﬁcation k ∗ is given by j ∗ = argmax j H
h
j W ajk ∗ x a j .
In general, we use H h j W ajk ∗ x a j to rank the features by the degree in which they contribute to the classiﬁcation k ∗ .
The explanation at the class level (global explanation) is obtained by investigating the product H hj W ajk in (12), regardless of the vector x .
The global most inﬂuential feature for each class k is given by j ∗ k = argmax j H hj W ajk .
In general, we use H hj W ajk to rank the features by their global importance with respect to class k .
4.5 Computational Complexity
In the training phase, the algorithm in (11), and more generally (12), learns the joint probability P jk in (8) by multiplying the J × N matrix of elements z ( n ) j with the N × K matrix of elements y ( n ) k .
Entry P jk is given by the inner product of the j -th row of the left matrix (which has N entries) and the k -th column of the right matrix (which has N entries), so computing it takes time O ( N ) .
We do this once per element.
Since the output matrix has dimension J × K , there are O ( JK ) elements to consider and the total work is done in O ( NJK ) .
That is, the training time is at most linear in the number of samples ( N ), in the number of features ( J ), and in the number of classes ( K ).
In the prediction phase, we compute u k (and y k ) in (11), and more generally (12).
As these are all elementwise operations on the J × K matrix of elements P jk , the total work is done in time O
( JK ) .
That is, the prediction time is at most linear in the number of features ( J ), and in the number of classes ( K ), and it does not depend on the number of training samples ( N ).
Finally, we notice that the computational complexity can be further improved by using sparse matrices, and all the operations involved in the training and prediction phases can be easily parallelized (e.g., on GPUs).
Thus, we expect the method to be highly scalable.
Neural Architecture
A major limitation of the algorithm presented in Section 4 is that it can be applied only when x j ≥ 0 .
Here we embed the method in a more ﬂexible architecture that admits x j ∈ C .
Let us assume that a data instance (e.g., document) can be represented as a superposition of some hidden states | s (cid:105) for s = 1 , ..., S (e.g., word embeddings).
Then, we write its wave function | ψ (cid:105) =
(cid:80) s
ψ s | s (cid:105) where the coefﬁcients ψ s generally depend on the feature vector x (e.g., words).
We represent such coefﬁcients with a neural network ψ s = ψ s ( x ) that maps the feature vector x ∈ C J to the vector of wave coefﬁcients ψ ∈ C S .
Then, we write the wave function of class k as | ϕ ( k ) (cid:105) =
(cid:80) s ϕ
( k ) s | s (cid:105) where the coefﬁcients ϕ ( k
) s depend on k and s , but not on the feature vector x .
Finally, we use the Born rule in (2) to compute the probability of | ψ (cid:105) to collapse in | ϕ ( k ) (cid:105) :
[Equation]
Equation (13) is read as a neural network u
= σ ( Φv ) where Φ is a matrix of elements Φ ks = ¯ ϕ ( k ) s , v = ψ ( x ) is the output of the previous layer, Φv denotes the matrix product (cid:80) s Φ ks v s , and the activation function σ ( · )
= | · | 2 is the modulus squared.
Finally, we apply a normalization layer to obtain the probabilities y k
= u k / (cid:80) k u k ∈ [0 , 1] .
An illustration is given in Figure 1.
Our method is similar to the usual approach where the ﬁrst part of a network is assumed to learn log-probabilities, and the ﬁnal classiﬁcation layer converts them into probabilities by applying the softmax function.
Here, the ﬁrst part of the network is assumed to learn the coefﬁcients of a wave function, and the ﬁnal classiﬁcation layer converts them into probabilities by applying the Born rule.
5.1 Initialization
We initialize the weights Φ ks such that the wave function | ϕ ( k ) (cid:105) has an equal probability to collapse in any state | s (cid:105) .
To this end, let us write the complex-valued weights Φ ks
= ρ ks e iθ ks , where i is the imaginary unit, ρ ks ≥ 0 , and θ ks ∈ [0 , 2 π ) .
We recall from (3) that the probability of | ϕ ( k ) (cid:105) to collapse in | s (cid:105) is P ( ϕ ( k ) → s ) =
| ϕ
( k ) s | 2 = | Φ ks |
=
ρ 2 ks .
We set ρ 2 ks = ρ 2 so that the (unnormalized) probability is constant for all k and s .
To normalize the probability, we set Sρ 2 = 1 , where S is the number of states (input dimension of the layer).
We obtain ρ = 1 / √ S , which resembles the initialization in [9] and [10].
Finally, we sample θ ks from a uniform distribution in the interval [0 , 2 π ) , such that the weights Φ ks are uniformly distributed in the complex circle (isotropy).
[Equation]
When the feature vector x is a (unnormalized) probability vector as in Section 4, then (11) can be written as the neural network in (13) where we use S =
J and ψ s ( x ) = √ x s .
In this case, it is interesting to initialize the weights in (13) with the corresponding weights developed in (11), that is Φ ks
= H s (cid:112) P s | k .[-
-]
5.2 Explainability
We notice that the probabilities y k are invariant under scaling and rotation of the coefﬁcients ¯ ϕ ( k ) s in (13).
To show that, we multiply ¯ ϕ ( k ) s by a scaling factor ρ and a phase factor e iθ .
Then, we substitute ¯ ϕ ( k ) s → ρe iθ ¯ ϕ ( k ) s in (13).
The phase factor e iθ vanishes when computing the modulus and the scaling factor ρ vanishes when the probabilities are normalized.
The invariance by scaling and rotation implies that the weights Φ ks have no absolute meaning, and they become meaningful only in relation with each other.
To inspect the relations among states s , for a given class k , it is interesting to visualize the weights Φ ks in the complex plane (with unlabelled axes).
Here, the length of the vectors can be used to rank the states by importance, while the direction of the vectors produce constructive or destructive interference among the states (see Figure 5 for an illustration).
For local explanations, Φ ks v s is used instead of Φ ks .
Empirical Results
We illustrate several aspects of our classiﬁer using three well-established text classiﬁcation benchmarks: 20Newsgroup 5 , and the R8 and R52 subsets of Reuters 21578 6 .
We perform tokenization using the function nltk.word_tokenize 7 and vectorize the text with TﬁdfVectorizer 8 .
No other text transformation or cleaning procedure is performed.
The ﬁnal datasets are composed by ( 20Newsgroup ) 20 classes, 204 817 words, 11 314 training documents, and 7 532 test documents; ( R8 ) 8 classes, 33 593 words, 5 485 training documents, 2 189 test documents; ( R52 ) 52 classes, 38 132 words, 6 532 training documents, and 2 568 test documents.
All the results are obtained using Python 3.9 on a Google Cloud Virtual Machine equipped with
CentOS 7, 12 vCPU Intel Cascade Lake 85 GB RAM, 1 GPU NVIDIA Tesla A100, and CUDA 11.5.
6.1 Training time, prediction time, and accuracy score
We compare our methodology against a baseline of six classiﬁers on the 20Newsgroup dataset.
With
Born Classiﬁer ( BC ), we refer to the algorithm presented in Section 4, where weights are computed as in equation (8) and classiﬁcation probabilities are calculated as in equation (11).
The baseline is composed by the algorithms Decision Tree (DT) , K-Nearest Neighbors (KNN) , Random Forest (RF) ,
Support Vector Machine (SVM) , Multinomial Naive Bayes (MNB) , and Logistic Regression (LR) .
For all the algorithms in the baseline, we use the corresponding implementation in scikit-learn .
All the classiﬁers are executed on CPU with default parameters.
The comparison between BC and the baseline is reported in Figure 2, where we show the training time, prediction time, and accuracy score in function of the fraction of data used for training inindependent executions.
Figure 2 shows that BC is fast to train, is fast to predict, can be accelerated on GPU, and it achieves the highest accuracy regardless of the size of the dataset.
Moreover, as the amount of training data decreases, the accuracy gap between BC and the other classiﬁers widens.
6.2 Imbalanced data
While 20Newsgroup is almost balanced in terms of documents per class, R8 and R52 are not.
In particular, the most frequent class in R52 contains 2 840 training samples, while the least frequent class contains only 1 document in the training set.
Figure 3 reports the F1-macro score for BC and the baseline on the three datasets, which are increasingly imbalanced.
BC outperforms the baseline models and the performance gap widens for more imbalanced data. The native capability of BC to work with imbalanced data can be traced back to (8), which computes the conditional probability of the features given the classes. Dividing the joint probability by the marginal effectively normalizes by the class imbalance. 6.3 Hyper-parameters and ablation study
We now compare the overall accuracy score and runtime of BC with the results of the ﬁne-tuned baseline on 20Newsgroup (see Table 1).
We then conduct an ablation study on BC (see Figure 4).
[Table]
In Table we tune the baseline classiﬁers via grid-search using 5-fold cross-validation.
Except for BC , which needs no tuning, all the other classiﬁers use between 20 and 50 combinations of hyper-parameters (reported in the replication code).
BC still achieves the highest accuracy by a remarkable margin.
Furthermore, the overall time required to classify the dataset is several orders of magnitude smaller than than that required by the baseline algorithms.
This is partly due to the speed of execution of BC (see Figure 2) and partly due to the fact that BC has no hyper-parameters to tune.
The ablation study is reported in Figure 4.
Here we display the test accuracy scores of the model corresponding to equation (12) for several values of the hyper-parameters a , b , and h .
The conﬁguration a = 12 , b = 1 , h = 1 , corresponds to BC in equation (11) and it achieves an accuracy of 87.3%.
Selecting the hyper-parameters via cross validation also achieves a test accuracy of 87.3%, suggesting that our prior generalizes well.
The classical conﬁguration a = 1 , b = 0 , h = 0 , discussed in Section 4.3, achieves an accuracy of 83.3%.
Removing the regularization ( h = 0 ) from our model leads to a large drop in performance: from 87.3% to 77.7%.
On the other hand, adding the regularization ( h = 1 ) to the classical conﬁguration does not lead to any performance gain: from 83.3% to 82.0%.
These results can be understood by recalling that the classical conﬁguration uses the conditional probability of the classes given the features ( P k | j ), while BC uses the conditional probability of the features given the classes ( P j | k ).
As P j | k gives larger weight to more frequent features, then BC is dominated by noise when the features irrelevant for classiﬁcation are also the most frequent (as is typical in text).
Regularization becomes important to reduce the inﬂuence of frequent features that are slightly skewed.
Instead, in the classical conﬁguration, regularization is redundant as P k | j already limits the inﬂuence of such features.
6.4 Embedding in a neural network architecture
With Born Layer ( BL) , we refer to the neural architecture displayed in Figure 1 that takes in input ψ ( x ) and returns the classiﬁcation probabilities according to equation (13).
Here we choose ψ s ( x ) = √ x s so that the transformation in equations (11) and (13) are the same, and they can be directly compared (in general, ψ ( x ) may be an arbitrary transformation to exploit e.g., semantics, as discussed in
Section 7).
BL is initialized as in (14) and it is optimized on the L 1 -loss using Adam [12] with default parameters.
We also initialize BL with the weights computed by BC as discussed at the end of
Section 5.1.
We refer to this version with BC+BL .
We stress-test BC and BL (which only rely on word counts) against a variety of deep learning approaches reported in literature (which exploit more sophisticated representations of the document).
In Table 2, we include cooperative neural networks ( CoNN ) [20], models that learn distributed representations of entities and documents from a knowledge base ( TextEnt )
[25], graph convolutional networks ( TextGCN ) [20], the neural attentive bag-of-entities model ( NABoE ) [24], and the diversiﬁed ensemble neural network ( DEns ) [29].
We ﬁnd that BC achieves state-of-the-art accuracy on the 20Newsgroup dataset, and it is up to a million times faster than alternative approaches (0.001s).
This conﬁrms that the weights computed explicitly in (11) are a good prior.
Also BL matches state-of-the-art performance, but it requires signiﬁcantly longer computational times (about 34s).
Initializing BL with BC requires few training epochs to further improve performance, suggesting that BC+BL can be ﬁne-tuned relatively fast (about 3s).
Finally, we notice that, while BL optimizes complex-valued weights, BC+BL uses real-valued weights.
This suggests that the reason for good performance lies in the transformation itself (13), rather than in the usage of complex numbers. Accuracy (%) Runtime (s) 20NG
R R 20NG
R R CoNN [20] 83.
N/A
N/A 120. N/A
N/A TextEnt [25] 84.
N/A 923. N/A TextGCN [26] 86.
NABoE [24] 86. N/A 152.
N/A DEns [29] 87. N/A
N/A N/A
BC (11) 87.
BL (1 epoch) 84. BL (10 epochs) 86.
BL (100 epochs) 87.
BC+BL (1 epoch) 86. BC+BL (10 epochs) 87.
BC+BL (100 epochs
[Table] 6.5 Explanation
To exemplify the explanation provided by BC , we extract the global weights described in Section 4.
and we report the 10 most inﬂuential words (out of more than 200 000) for a subset of classes in 20Newsgroup .
Table 3 shows that the words do not contain stopwords or punctuation, or terms that are too general to be representative of a speciﬁc class. The ability of BC to ﬁlter out noise can be traced back to the regularization in Section 4.2 that shrinks to zero the weights of words with little discriminatory power.
[Table]
DES [Table] In Figure [Table]
we train BL for two epochs and we represent the evolution of the complex-valued weights described in Section 5.2.
The ﬁgure represents the 10 weights that have the largest (and smallest) modulus for the class rec.sport.baseball at the end of the training.
Starting from weights that are initialized randomly in all directions of the complex circle, BL learns to give larger weights to more relevant features, smaller weights to less relevant features, and to orientate the features in different directions.
The weights converge in the same direction for words that are speciﬁcally related to the concept of baseball .
The words hockey and NHL (National Hockey League), which are still relevant for a different class of sports, are associated with a large weight but they are also collocated in the opposite direction with respect to the baseball-related words.
Conclusion
By regarding text as a superposition of words, we have introduced a supervised classiﬁcation algorithm based on key postulates of quantum mechanics, namely the Born rule.
The classiﬁer is self-explainable, admits a closed-form expression (11), and can be embedded in neural network architectures (13).
Our method outperforms traditional algorithms on the 20Newsgroup , R8 , and R datasets, in terms of classiﬁcation performance and computational time.
These results suggest that physical principles can be successfully exploited in machine learning and may open a new class of classiﬁcation algorithms.
There are several potential improvements and extensions to this work.
First, our classiﬁer is derived from the Born rule and the explanation for its effectiveness in the view of machine learning remains an open question. Improving the empirical evaluation with thorough experiments would allow to assess the limitations of the method and to describe the beneﬁts and pitfalls in various applications and data types.
Second, whenever a data instance can be represented with a probability distribution over some space, then a wave function can be constructed explicitly and the data may be classiﬁed with the algorithm in Section 4.
More generally, an arbitrary data instance can be classiﬁed with the network in Section 5, where the ﬁrst part of the network learns the coefﬁcients of a wave function and the ﬁnal classiﬁcation layer converts them into probabilities by applying the Born rule.
Using the transformation in (13) on top of existing architectures offers a natural way to push our framework beyond a bag-of-words (superposition-of-words) representation of the document, to encode semantics, and to extend the method to non-textual data.
For instance, an image may be classiﬁed with the network in (13) where ψ ( x ) is a convolutional layer. This corresponds to regarding the image as a superposition of the features learnt by the convolution.
Finally, an interesting research direction would be constructing deep networks that apply the transformation in (13) repeatedly.
A deep architecture of this kind would ﬁnd its biological foundations in the quantum brain hypothesis [11], which suggests that quantum events could play a non-trivial role in neuronal cells, and contribute to an extremely high complexity, variability and computational power of neuronal dynamics. Acknowledgments This work was partially supported by the Google Cloud Research Credits program with the award GCP
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section 7.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] We don’t see any potential negative societal impact other than those that apply to ML in general.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The code to reproduce the results is provided in the supplemental material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] Datasets (20NG, R8, R52) and software (scikit-learn, pytorch) are cited properly.
(b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]