# Smart Contract Generation Assisted by AI-Based Word Segmentation
## 6 Evaluation of the SpIn Model
### Robustness Evaluation
The eye-catching OOV recall improvement is **+6.77%** on the PKU dataset. For the AS dataset, there is still a **+2.88%** boost. Such massive promotions on the OOV recall ensure the generalization capability of Spln. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **InterBEST-2010** & \\  & **F1** & R\_oov** \\ \hline \hline \multicolumn{1}{c}{Syllable-based-TWS [67]} & 95.59 & 67.42 \\ \hline \hline \multicolumn{1}{c}{Spln} & **95.61** & **70.83** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison between Spln and previous state-of-the-art methods on the TWS task. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**CITYU**} & \multicolumn{3}{c}{**AS**} & \multicolumn{3}{c}{**PKU**} & \multicolumn{3}{c}{**MSR**} & \multicolumn{3}{c}{**CTB6**} \\  & **F1** & **R\_oov** & **F1** & **R\_oov** & **F1** & **R\_oov** & **F1** & **R\_oov** & **F1** & **R\_oov** \\ \hline DGRNN [69] & - & - & - & - & 96.1 & - & 96.3 & - & 95.8 & - \\ Bi-LSTMs\_CWS [31] & 97.2 & 87.5 & 96.2 & 70.7 & 96.1 & 78.8 & 97.4 & 80.0 & 96.7 & 85.4 \\ Glyce [42] & **97.9** & - & **96.7** & - & **96.7** & - & **98.3** & - & - & - \\ WMSEG [43] & 97.8 & **87.57** & 96.58 & 78.48 & 96.51 & **86.76** & 98.28 & **86.67** & 97.16 & **88.00** \\ \hline Spln & **98.6** & **90.57** & **97.5** & **81.36** & **98.0** & **93.53** & **98.7** & **93.13** & **98.6** & **93.90** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparisons between Spln and previous state-of-the-art results on the CWS task. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **KAIST** & **R\_oov** & **F1** & **GSD** \\ \hline BMES+Unigram\_Feature & 87.62 & 78.34 & 87.12 & 78.27 \\ \hline Spln & **92.37** & **83.81** & **91.19** & **82.24** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison between Spln and previous state-of-the-art methods on the KWS task. Compared with the work [42], which involves rich pictographic feature representations, our features are simple bigram features via concatenation of unigram features. Instead of exploiting rich external knowledge and leveraging wordhood information to incorporate with the framework as in [43], our SpIn is closed without involving additional information. Compared with previous works using the word dictionary and linguistic information (such as character type information), SpIn, without any extra common knowledge, gains state-of-the-art results on the JWS task as well. As there is no related segmentation work about these Korean datasets, we compare SpIn with traditional methods, which widely employ character features and character-based tagging schemas and report the results in Table 5. SpIn achieves a better performance on both datasets. Especially, SpIn boosts up to **+4.75%** F1 score improvement on the GSD dataset. Furthermore, we observe that SpIn performs better for OOV words on both datasets. The evaluation of TWS is the extension work for verifying the effectiveness of SpIn. As opposed to [67], which leverages syllable embeddings to capture linguistic features, our SpIn merely considers text features. We follow the recent SOTA work and conduct comparative experiments. Instead of the work [67] that leverages syllable embeddings to capture linguistic features, our SpIn merely considers text features. Table 6 reports the evaluation results and demonstrates that the SpIn also works for TWS. Although contextual features have been hotly discussed and proved to be effective in existing methods, they may result in a performance loss of the model. As the existing tagging schemas are character-based, employing context features (e.g., bigram, trigram, and quadrigram feature) may introduce noise for the learning model. However, bigram is the natural choice in our proposed SpIn, since we model the connection state of two adjacent characters. For the fixed separation inference, no other context or lexical information is desired, and the connections between characters are fully explored and exploited.