# Candidate Term Boundary Conflict Reduction Method for Chinese Geological Text Segmentation
## 2 Related Work  
### Technology of CWS in the General Field  
Throughout the development of word segmentation technology, approaches to CWS can be mainly divided into four categories according to different principles: dictionary-based approaches, statistics-based approaches, deep-learning-based approaches, and multi-mode integration approaches. Dictionary-based approaches, also known as mechanical segmentation, refer to the mechanical segmentation of Chinese text based on a relatively complete dictionary and matching segmentation method to obtain term segmentation sequences. [PERSON] compared the spatial efficiency and processing speed of binary-seek-by-word, TRIE indexing tree, and binary-seek-by-characters. The experimental results proved that the processing speed of binary-seek-by-characters was increased by more than 15 times and the effect was obvious [8]. [PERSON] proposed a mechanical statistical word segmentation based on hash structure, which combined the advantages of mechanical word segmentation and statistical word segmentation and had a strong recognition ability of OOV. However, there are significant differences between the speed and accuracy of word segmentation when using dictionaries with different amounts of words [9]. [PERSON] proposed an improved positive maximal matching algorithm with a double hash structure, but there were still some deficiencies in the recognition of ambiguous terms and OOV [10]. [PERSON] improved the recognition of OOV and ambiguous term through a bidirectional matching algorithm, but the covering ambiguity could not be dealt with [7]. Dictionary-based approaches have the advantages of simplicity, quickness, and easy implementation, and they can improve OOV recognition and ambiguity recognition to some extent. Although high-quality dictionaries are often used in related fields, there are some limitations to their use. With the emergence of large amounts of new words, a solidified dictionary has obvious defects in the performance of specific word segmentation, which requires constant updating and manual maintenance, and it is difficult to keep a balance between the accuracy rate and manual cost. Statistics-based approaches use statistics to obtain the mutual information of words based on the adjacent relationships between words and establish a word segmentation model or algorithm, so as to recognize new words and cut sentences into words. Its main advantage is that it makes use of semantic connection relationships, balances the recognition of different characters, and has a good recognition effect on OOV. Machine learning is a typical representative of statistical methods. The widely used algorithms are the Hidden Markov Model (HMM) [11], Maximum Entropy Markov Model (MEMM) [12; 13], the Conditional Random Field (CRF) [14], and their improved algorithms. For example, [PERSON] used basic segmentation and named entity recognition based on CRF to generate initial segmentation results and corrected the word segmentation results with statistical methods and grammatical rules, which achieved good word segmentation results [14]. Although statistics-based approaches are effective, their limitation is that the machine learning model relies on excellent feature engineering defined manually and a lot of work is required to verify the effectiveness of these features, which is relatively laborious. With the development of machine learning, segmentation algorithms based on neural networks and deep learning have emerged, which can train a neural network to automatically learn features. Therefore, the labor cost and time costs will be greatly reduced in the feature engineering stage and the efficiency of feature engineering extraction will be improved. In 2011, [PERSON] introduced a deep learning algorithm into the natural language processing task for the first time [15]. This method can learn original characteristics and context representations through the final word segmentation training set.