# Smart Contract Generation Assisted by AI-Based Word Segmentation
## 6 Evaluation of the SpIn Model
### Robustness Evaluation  
To prove that SpIn is universal and robust regardless of languages, we evaluate the performance of SpIn on multiple WS tasks involving four languages and nine datasets. **Datasets. ** Five benchmark datasets are evaluated for CWS task, namely Chinese Penn Treebank 6.0 (CTB6) [62] and CITYU, AS, PKU, and MSR from the SIGHAN 2005 bakeoff task [63]. PKU, MSR, and CTB6 are in simplified Chinese whereas AS and CITYU are in traditional Chinese. SpIn is also verified on the JWS dataset BCCW] version 1.1 (short for Balanced Corpus of Contemporary Written Japanese) [64] and KWS dataset UD_Korean-GSD corpora ([[https://github.com/emorynlp/ud-korean/tree/master/google](https://github.com/emorynlp/ud-korean/tree/master/google)]([https://github.com/emorynlp/ud-korean/tree/master/google](https://github.com/emorynlp/ud-korean/tree/master/google)), accessed on 8 April 2018) and Kaist ([[https://github.com/UniversalDependencies/UD_Korean-Kaist](https://github.com/UniversalDependencies/UD_Korean-Kaist)]([https://github.com/UniversalDependencies/UD_Korean-Kaist](https://github.com/UniversalDependencies/UD_Korean-Kaist)), accessed on 3 November 2021). BCCWJ covers various domain data. We follow the same dataset split with the Project Next NLP to attentively avoid data bias. UD_Korean-GSD and Kaist are two widely used datasets in syntactic parsing tasks and are automatically converted from structural trees in the Google UD Treebank [65] and the KAIST Treebank [66]. We extract words according to syntax structure, which carries implicit segmentation information. Moreover, we evaluate SpIn on the Thai Word Segmentation to further demonstrate its effectiveness and robustness. The evaluation of TWS is conducted on the InterBEST-2010 (Benchmark for Enhancing the Standard of Thai language processing) dataset following the previous work [67]. We follow the same data split with the work in [67]. The training and test set are 90% and 10%, respectively. **Parameter settings and evaluation metrics. ** For the SpIn network, we uniformly set the sequence length as 128; the learning rate as 5 \(\times\) 10\({}^{-5}\), the batch size as 32, and the epochs of training as 15. We employ the early stop mechanism to avoid over-fitting and leverage the Adam as the optimizer during the training process. In the experiments, allthe above-mentioned parameters are set still. Furthermore, following the widely accepted evaluation methodologies, the average Micro F1 score of multi-time experimental results is adopted in our experiments as the metric for exhibiting reliability. Moreover, another essential metric, which is the Recall of Out-of-Vocabulary (R_OOV) words, is leveraged to evaluate the generalization of the word segmentation algorithm. **Evaluation results. ** Tables 3-6 presented the results of CWS, JSW, KWS, and TWS, respectively. We adopt the average score of the experiments repeated 10 times for solid evaluations. Following previous works, the results are accurate to one decimal place. A single word segment is considered as the unit for calculating the F1 score and R_OOV. Spln achieves state-of-the-art results on all nine datasets regardless of language. Compared with the previous methods, Spln brought a **+1.3%** (PKU dataset) improvement in the F1 score. The least improvement of Spln on the CWS task is a **+0.4%** F1 score (on the MSR dataset). Moreover, Spln achieves the best result on the JWS task as well. Considering the lofty baselines in CWS and JWS, these promotions are significant enough in proving the universality of Spln on WS tasks. Of note, the switch_LSTMs_CWS [68] in Table 3 also exploited the bigram feature, but are fallen far behind Spln. Therefore, the bigram feature is not the undercovered actuator of Spln. Moreover, Spln also notably escalates the recall rate of OOV words in Tables 3 and 4. On every single dataset for the above four WS tasks, Spln achieves the best OOV performance. The results demonstrate the effectiveness of Spln on OOV words.