# Short Text Classification with Tolerance-Based Soft Computing Method
## 3 Materials and Methods
### Methods  
In this section, we present our proposed Tolerance Sentiment Classifier (TSC) in terms of the two Algorithms 1 and 2. The TSC algorithm was implemented using Python on a 16 GB RAM, Nvidia RTX 2060 GPU, 512 GB SSD machine using SBERT base vectors (1 \(\times\) 768 dimensional vectors). We considered mean and median values for determining the prototype class vectors for the TSC algorithm. In addition, we experimented using TF-IDF vectors with the TSC algorithm for all the datasets. However, the classification results with TF-IDF vectors were unsatisfactory primarily because the cosine distance values started to \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Dataset** & **Type** & **Size** & **Positive** & **Negative** & **Neutral** & **Irrelevant** \\ \hline \multirow{2}{*}{COVID-Sentiment} & Train & 7000 & 22.02\% & 30.35\% & 47.63\% & - \\  & Test & 1003 & 23.53\% & 37.29\% & 39.18\% & - \\ \hline \multirow{2}{*}{U.S. Airline Sentiment} & Train & 12,000 & 16.79\% & 61.02\% & 22.19\% & - \\  & Test & 1000 & 13\% & 67.5\% & 19.5\% & - \\ \hline \multirow{2}{*}{IMDB Movie Review} & Train & 20,000 & 50.27\% & 49.73\% & - & - \\  & Test & 2000 & 50.35\% & 49.65\% & - & - \\ \hline \multirow{2}{*}{SST-2} & Train & 15000 & 55.37\% & 44.63\% & - & - \\  & Test & 1500 & 55.53\% & 44.47\% & - & - \\ \hline \multirow{2}{*}{Sentiment140} & Train & 15000 & 50\% & 50\% & - & - \\  & Test & 1000 & 50\% & 50\% & - & - \\ \hline \multirow{2}{*}{SemEval 2017} & Train & 17001 & 40.67\% & 15\% & 44.33\% & - \\  & Test & 3546 & 41.54\% & 15.76\% & 42.70\% & - \\ \hline \multirow{2}{*}{Sanders corpus} & Train & 4059 & 10.24\% & 11.38\% & 45.26\% & 33.12\% \\  & Test & 1015 & 9.85\% & 10.54\% & 47.68\% & 31.93\% \\ \hline \multirow{2}{*}{UCI Sentence} & Train & 2700 & 49.11\% & 50.89 & - & - \\  & Test & 300 & 58\% & 42\% & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Dataset Information. \begin{table}
\begin{tabular}{l l l} \hline \hline
**News Category** & **\#Train Documents** & **\# Tolerance Classes** \\ \hline alt.atheism & 442 & 230 \\ comp.graphics & 534 & 238 \\ comp.os.ms-windows.misc & 528 & 498 \\ comp.sys.ibm.pc.hardware & 540 & 445 \\ comp.sys.mac.hardware & 527 & 428 \\ comp.windows.x & 561 & 400 \\ misc.forsale & 535 & 312 \\ rec.autos & 526 & 342 \\ rec.motorcycles & 548 & 189 \\ rec.sport.baseball & 554 & 319 \\ rec.sport.hockey & 551 & 332 \\ sci.crypt & 537 & 437 \\ sci.electronics & 547 & 146 \\ sci.med & 551 & 285 \\ sci.space & 525 & 272 \\ soc.religion.christian & 542 & 494 \\ talk.politics.guns & 499 & 380 \\ talk.politics.mideast & 511 & 335 \\ talk.politics.misc & 421 & 248 \\ talk.religion.misc & 335 & 130 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: 20-Newsgroups dataset for best \(\epsilon\) value of 0.19.  
converge to one and hence resulted in a number of the same values in the distance matrix shown in Figure 2, which restricts the use of TF-IDF vectors. **Vector Embeddings with SBERT:** Sentence-BERT (SBERT) is a modification of the pre-trained BERT network that uses siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. SBERT is fine-tuned on SNLI [30] and the Multi-Genre NLI [31] data, which creates sentence embeddings and significantly outperforms other state-of-the-art sentence embedding methods such as InferSent [32] and and Universal Sentence Encoder [33] in terms of accuracy. **Training Phase: Representative Class Generation Algorithm 1:** In this phase, given a tolerance level \(\varepsilon\), tolerance classes are induced from the training set vectors using the cosine distance, and the representative of each tolerance class is computed as the mean value of the feature vector. The polarity (or category) of the representative vector is determined based on majority voting. **Testing Phase: Polarity Assignment Algorithm 2:** In the classification phase, TSC uses the representative class vectors generated in the training phase and their associated polarity/text category.