# A Rule-Based AI Method for an Agent Playing Big Two
## 4 Experimental Results and Discussion
### Experiment 1: Rule-Based AI versus Randomized AI  
In the first experiment, two rule-based AI agents played 2000 games against two randomized AI agents. In every game, two rule-based AI agents played against two randomized AI agents; therefore, we gathered 4000 play logs from each set--4000 from the two rule-based AI agents and 4000 from the two randomized AI agents. Table 2 shows the statistics of play log data. The rule-based AI won 1792 games (89.60%) and lost 208 games (10.40%). We analyzed the frequency (\(f\)) of scores in the positive zone (when the AI won and achieved a positive score) and the frequency (\(f\)) of scores in the negative zone (when  
Figure 5: The performance curves for (**a**) rule-based AI and randomized AI, (**b**) rule-based AI and conventional AI, (**c**) conventional AI and human players, and (**d**) rule-based AI and human players. the AI lost and achieved a negative score). Rule-based AI finished with a positive score 1792 times and a negative score 2208 times. In contrast, randomized AI finished with a positive score 208 times and a negative score 3792 times. Rule-based AI's mean score when winning was 15.64 (\(SD\) = 6.43) and when losing it was \(-4.01\) (\(SD\) = 2.57). Randomized AI's mean score when winning was 12.32 (\(SD\) = 6.53), and when losing it was \(-4.64\) (\(SD\) = 2.53). When testing the first hypothesis at a confidence level of 95%, we obtained a \(p\)-value of 1.0669 \(\times\) 10\({}^{-283}\). The statistics confirm that the results are statistically significant (\(p\)-value < 0.01). Thus, rule-based AI significantly outperforms randomized AI. The effect size (\(r\)) was 0.5921, which is a large effect (\(r\) > 0.50), according to [PERSON]'s classification of effect sizes. Figure 6 shows the frequency distribution of rule-based AI agents and randomized AI agents from 4000 scores. The \(x\)-axis denotes the score achieved in each game, and the \(y\)-axis denotes the frequency (number of games) of the players that achieved that score. In Big Two, each player receives 13 cards to play. Although each player can only lose 13 cards, they have a chance to win up to 39 cards from the three opponents. The winner's game score is determined by the total number of cards left by the three opponents. For example, if player 1 wins and the three opponents have 2, 6, and 9 cards left, then the scores will be 17; \(-2\); \(-6\); \(-9\). Note that at the end of each game, the winner achieves a positive score and three losers achieve negative scores. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Statistic** & **Rule-Based AI** & **Randomized AI** \\ \hline Win rate & 89.60\% (1792 games) & 10.40\% (208 games) \\ \hline \multicolumn{3}{c}{Scores in the positive zone} \\ \hline \(f\) & 1792 & 208 \\ Mean score & 15.64 & 12.32 \\ \(SD\) score & 6.43 & 6.53 \\ \hline \multicolumn{3}{c}{Scores in the negative zone} \\ \hline \(f\) & 2208 & 3792 \\ Mean score & \(-4.01\) & \(-4.64\) \\ \(SD\) score & 2.57 & 2.53 \\ \hline \multicolumn{3}{c}{Overall statistics concerning all scores} \\ \hline Total & 4000 scores & 4000 scores \\ \(p\)-value & & \(1.0669\times 10^{-283}\) \\ Effect size (\(r\)) & & 0.5921 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics of play log data from 2000 games of rule-based AI versus randomized AI. Figure 6: The frequency distribution of scores for (**a**) rule-based AI, (**b**) randomized AI, (**c**) rule-based AI and randomized AI. The brown bars represent the intersection of purple bars and orange bars. In Figure 6a, rule-based AI agents finished with a positive score in 1792 games and a negative score in 2208 games. In Figure 6b, randomized AI agents finished with a positive score in 208 games and a negative score in 3792 games. We compared the frequency distribution of all game scores between rule-based AI and randomized AI, as shown in Figure 6c. In the positive zone, the rule-based AI scored positively more often than the randomized AI. The percentage of rule-based AI was 89.60%, and the percentage of randomized AI was 10.40%.