# Candidate Term Boundary Conflict Reduction Method for Chinese Geological Text Segmentation
## 4 Experiments and Discussion
### Discussion and Analysis  
Through experiments, it is found that the HFCR method based on information entropy has some outstanding features for both geological and general texts.  
1. The F1-value in the geological text word segmentation is higher than 95%, as shown in Tables 3 and 6, which is obviously better than the SOTA model and other representative tokenizers. Moreover, the segmentation results for general texts are also slightly better than Jieba.
2. Our experiments solve the problems of ambiguity recognition and OOV recognition without any samples, labels, or high-performance devices. As shown in Table 2, place names such as \"Lin Zhou Shi (Linzhou City)\" and \"Ren Cun Zhen (Rencun Town)\", rock names such as \"Zan Huang Yan Qun (Zanhuang Group)\" and \"Xie Chang Shi (plagioclase)\", numerical values, and common words received the correct segmentation.
3. Our algorithm uses dynamic numerical comparison instead of setting a fixed threshold manually, which is good to avoid the randomness in the segmentation based on information entropy. Based on the rapid advantages of statistical methods and the cost advantage without manual labor, large-scale informatization of geological text can be realized in a short time.  
In our experiments, we found that the text length has an effect on accuracy, and two objective facts are of interest.  
1. As the text length increases, most of the words are cut correctly and the accuracy will increase; however, more cases of identical end-word concatenation increase too. For example, the forward connection richness of the suffix \"[PERSON] (Jiazhuang)\" is richer than that of complete place names such as \"[PERSON] ([PERSON])\", \"[PERSON] ([PERSON])\", and \"[PERSON] ([PERSON])\", resulting in error segmentation of \"[PERSON]/[PERSON] ([PERSON]/Jiazhuang)\", \"[PERSON]/[PERSON] ([PERSON])\" and \"[PERSON] ([PERSON])\".
2. There are always terms that appear only once or are monotonically connected in a text of any length, but they appear more often in short texts, which may account for the lower accuracy of shorter texts. For example, sentences such as \"Lue Xian Ya Bian La Chang (slightly flattened and elongated)\" and \"Bu Yi Qu Xiao Huo Geng Ming (not suitable for cancellation or renaming)\" have only a fixed type of connection and are easy to be regarded as fake long terms and be segmented incompletely. However, some long words such as \"[PERSON]/[PERSON]/[PERSON]/[PERSON] (Henan Provincial Department of Land and Resources)\" and \"[PERSON]/[PERSON]/[PERSON]/[PERSON]/[PERSON]/[PERSON]/[PERSON]/[PERSON] (Beijing Dadi Xiyuan Mining Management Technology Co. LTD)\" will be segmented correctly.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Data** & **Tools** & **Accuracy** & **Precision** & **Recall** & **F1-Value** \\ \hline \multirow{3}{*}{Rencun} & HFCR & 92.38\% & 95.85\% & 95.90\% & 95.88\% \\  & WMSEG & 68.34\% & 79.27\% & 72.68\% & 75.84\% \\  & GPT-3 & & & 92.45\% * & \\ \hline \multirow{3}{*}{Nanpanjiang} & HFCR & 93.91\% & 96.89\% & 96.62\% & 96.76\% \\  & WMSEG & 80.03\% & 92.83\% & 81.33\% & 86.70\% \\  & GPT-3 & & & 94.59\% * & \\ \hline \multirow{3}{*}{[PERSON]} & HFCR & 93.98\% & 97.18\% & 96.39\% & 96.78\% \\  & WMSEG & 76.25\% & 86.05\% & 82.16\% & 84.06\% \\ \cline{1-1}  & GPT-3 & & & 92.24\% * & \\ \hline \hline \end{tabular} The underlined part indicates the optimal segmentation performance and the asterisk * indicates the average accuracy of running 20 repeated experiments.  
\end{table}
Table 6: Comparison of accuracy between HFCR algorithm and SOTA models.