# _algorithms_
## 4 Experiments and Analysis
### Parameter Analysis  
As shown in Table 8, the parameters that needed to be manually adjusted included: the number of clauses (C), the reward or punishment strength (s) during Feedback, and the threshold (\(T\)) for output summation.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Acc** & **F1** \\ \hline SVM & 0.821 & 0.821 \\ \hline LR & 0.823 & 0.814 \\ \hline DPCNN [37] & 0.819 & 0.802 \\ \hline Bi-LSTM & 0.823 & 0.818 \\ \hline RCNN [38] & 0.815 & 0.804 \\ \hline BERT+Attention [36] & 0.855 & 0.850 \\ \hline
**Tsetlin Machine** & **0.869** & **0.866** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Result of Spam Review Detection.  
Figure 7: Ablation experiments of Spam Reviews Detection (/ means without).  
In brief, \(C\) was used to extract semantic features, similar to the convolution kernels in CNN and the recurrent neurons in RNN. If \(C\) was too small, the judgment of the model was affected largely by individual clauses, leading to a large fluctuation and extreme situations. On the other hand, when \(C\) was too large, it caused resource redundancy and more calculation during the training process. A suitable parameter could bring better judgment and faster speed.  
Hyperparameter \(s\) was similar to the learning rate and could affect the speed of convergence and the model's final effect as it had a big affect on the feedback process. When the model triggered Type I Feedback, \(s\) determined the probability of the model state transition. The larger the value of \(s\), the more the TA were stimulated to include literals in their clauses. Clearly, the probability of encountering non-matching input grew as \(s\) increased (more literals were included). It meant that the Feedback was more likely to choose reward (penalty) operation. On the other hand, when \(s\) was small, the feedback mechanism gave Inaction feedback in a greater probability than reward and penalty. Therefore, the model was more inclined to maintain the existing state without changing.  
\(T\) was highly related to \(C\) and it was the threshold for the summation of all the clauses in the output period. The introduction of \(T\) made the clauses distribute themselves according to the sub-patterns of data, so that \(T\) clauses of the correct polarity evaluated to 1 for any particular input \(X\). Additionally, the summation target \(T\) opened up for interplay between the clauses, including rectification of special cases. The purpose of the mechanism was to ensure that the TM only spent a few of the available clauses to represent each specific sub-pattern. This was achieved by randomly selecting clauses to receive feedback, reducing intensity when approaching the summation target \(T\).  
Among them, \(C\) and \(T\) usually used empirical values [24], which were 700 and 9000, respectively, while \(s\) needed to be adjusted according to the task. In the experiment of sentiment analysis, we found that the choice of \(s\) had little effect on the convergence speed and final performance, and the model could easily reach the optimum in about two epochs. In the spam review detection task, we found that \(s\) had a greater impact on the convergence speed and the final effect. As shown in Figure 8, we drew the relationship between \(s\) and the convergence rate and final effect. There, the x-axis represents epoch number and y-axis represents weighted-F1 score.  
Figure 8: Change of Performance with Parameter \(s\).  
Through analysis, it can be seen that when \(s=3\), the model could converge quickly in no more than 15 epochs and achieved the best results. When \(s\) was configured as 15, the model needed more than 40 epochs to achieve similar performance.