# _algorithms_
## 1 Introduction  
Shallow machine learning models may have intuitive interpretability, but they suffer from a performance gap compared with their counterpart deep learning models. With the introduction of BERT [1], pre-training and fine-tuning methods were shown to be excellent in nearly all NLP tasks. These deep learning models achieved superior performance in Chinese NLP tasks, but their black-box nature makes it difficult to explain decisions, which may limit their applicability in certain scenarios.  
In this article, we address those challenges by proposing a novel interpretable framework based on the Tsetlin machine (TM), which was originally proposed by [2]. TM is capable of decomposing the problem into multiple pattern recognition problems [2]. Firstly, the binary representations of texts are obtained as the input features for TM. Further, multiple conjunction clauses are applied to learn the implicit propositional logic of the problem. Finally, the output of clauses are aggregated and the classification result is obtained through a thresholding mechanism. Differing from the current widely-used deep learning models, TM has a relatively simple and universal learning mechanism, namely Tsetlin Automata (TA). The learning and feedback mechanism of TA makes TM efficient in terms of resource consumption [3], and yields faster convergence rate [4; 5], better interpretability [6] and higher reliability than deep learning models.  
Extensive experiments are conducted on three public Chinese datasets, ChnSentiCorp-htl-all, Weibo-Senti and Douban Movie Reviews, for sentiment analysis and spam review detection. The results indicated that our method achieved superior accuracy and a superiorF1 score, compared to state-of-the-art deep learning based methods, e.g., BERT and ER-INE [7]. This suggests that TM is a promising tool in NLPm due to its superior performance and excellent interpretability. (A preliminary and very abridged version of some of these results was presented in [8] at the First International Symposium on the Tsetlin (ISTM 2022), in June, 2022, in Grimstad, Norway.) Our contributions include the following points:  
* We use the clauses in the TM to learn the state transition of Chinese words and to obtain optimal states through the feedback mechanism. Once we find out the mapping relationship between words and target classes, we summarize the results of each clause and capture the overall semantic information. To the best of our knowledge, this is the first work that utilizes TM in Chinese NLP tasks.
* We propose a new framework, based on TM, for spam review detection. Compared to previous work on TM, we add richer input features, including user features and product features, etc., and adopt a conversion mechanism to convert these into the form required for model input. To the best of our knowledge, this is the first work that utilizes TM in spam review detection tasks.
* Our devised framework provides a good F1 score and accuracy close to, or higher than, BERT [1] on some open datasets.
* In the case study experiment, the visualization of the decision-making process in our model is demonstrated, which provides an intuitive and easy way to interpret the final output of the proposed model.  
The rest of the paper is organized as follows. We list some related works in Section 2. Section 3 describes the processing operation and model structure, and we also introduce the learning and feedback mechanism of the TM model. We present the experimental results and provide some interpretability analysis in Section 4. In Section 5, we draw the concluding remarks of our work.