# Research and Implementation of Chinese Automatic Word Segmentation System Based on Complex Network Features

Authors: Huihui Si
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

In the field of Chinese information processing, automatic word segmentation is a fundamental concept. Complex networks are widely used in modeling and analysis of complex systems across a wide range of disciplines, prompting an increase in interest in the structure of complex networks and the relationship between structure and function among academics. In contrast to Western countries, while there is a syllabic change in spoken Chinese that has formed a degree of boundary, it appears in writing as continuous Chinese character strings with no obvious word segmentation marks and no morphological segmentation marks. In summary, an in-depth study of network news information processing through complex networks can provide effective means to purify the network environment and reduce massive waste of communication resources and users’ time, which is of significant research importance. The characteristics of complex networks are used to study the Chinese automatic word segmentation system in this paper. A new interdisciplinary subject, complex networks and complex systems, has emerged. Researchers from various disciplines are attempting to study complex systems through the lens of system complexity theory and complex network theory in order to better understand the mechanism of network fault diffusion.


---

## Body

## 1. Introduction

With the rapid expansion of Chinese web pages on the Internet and the rapid popularization of Chinese electronic publications and Chinese numerals Library, the research on Chinese natural language processing with unrestricted texts as the main object is becoming more and more important [1]. There has not been a final segmentation standard for a long time, and it has gradually become one of the hot issues in Chinese information processing academia. Different from Western countries, although there is a certain change in syllables in spoken Chinese, which forms a certain degree of boundary, it appears in the form of continuous Chinese character strings in writing, without obvious segmentation marks, and even lacking morphological segmentation marks [2]. Therefore, Chinese natural language processing must first segment the text and divide the Chinese character string into correct word strings, which is called Chinese automatic segmentation. As we know, when a Chinese text is input into a computer, there is no distinguishing character between words in the text, usually one word and one yard, arranged at equal intervals. However, in the process of combining words with words, there is a certain combination rule between them [3, 4]. Some of these rules are relatively definite, and some have different segmentation rules in different artistic conception. The research of Chinese automatic word segmentation is mainly from the word level. This problem has received extensive attention since the late 1970s. At present, many word segmentation methods have been implemented. In this long-term research and practice process, the determination of word segmentation units, ambiguous field processing, and unknown word recognition have become three major problems that perplex us [5]. How to design a practical, high-performance Chinese automatic word segmentation system with high segmentation speed, ideal segmentation accuracy, and good maintainability has attracted much attention, and many of them have become the research focus in the field of computer application. Complex network is a network composed of a large number of nodes and edges, which can be used to describe various complex systems, structures, and organizations in the real world, such as social networks, communication networks, and biological networks [6, 7].Complex networks are widely used in the modeling and analysis of complex systems in various disciplines, which also makes more and more scholars begin to pay attention to the structure of complex networks and the relationship between structure and function. Therefore, it is both a challenge and an opportunity to study and improve the complex network and apply it to network news information processing. Through in-depth research in this field, we can automatically identify reactionary, pornographic, and other bad websites. It can automatically discover and track recent news and public opinion hotspots. It can automatically identify some duplicate information or spam information on the network. Complex networks, which can provide more effective means for personalized services, are often represented by graphs, so the classification of complex networks usually draws on some methods of graph theory, including graph similarity and graph matching. At present, the proposed complex network classification algorithms can be divided into three categories: graph isomorphism, iterative method, and feature extraction [8]. In short, the in-depth study of complex network in network news information processing can provide an effective means to purify the network environment and reduce the huge waste of communication resources and user time, which has important research significance [9, 10].The dynamic behavior of the network, such as the influence of network structure on its dynamic properties [11], is a very important research point for complex systems. The propagation mechanism of dynamic behavior in the process of network operation is the focus of research, such as how network node failures propagate in the network, what parameters are related to the scale of chain collapse caused by network node or edge failures, the community structure of complex networks, such as self-organizing criticality phenomenon in the network, and dynamic propagation characteristics, which will bring brand-new research. It introduces new solutions to hot issues such as network news hotspot discovery, public opinion hotspot discovery and dissemination, identification of bad websites such as reactionary or pornographic websites, personalized search engine, multiagent cooperation, and group communication behavior from the perspective of complex networks [12]. The study of complex networks and systems is a new interdisciplinary field. Researchers from various disciplines are attempting to study complex systems from the perspectives of system complexity theory and complex network theory in order to better understand the mechanism of network fault diffusion. The essence of dynamic behavior of complex systems is analyzed from a different angle, and the internal mechanism of network cascading failures and other accidents is analyzed [13], using the holistic analysis method of complex systems, combined with complex systems, statistical physics, and risk analysis.


## 2. Related Work

Document [14] proposed that the “cseg and tag system” developed by the Artificial Intelligence Laboratory of Tsinghua University in the late 1990s is such a representative. Its design is based on such a basic idea: there is mutual feedback between them, which is conducive to the improvement of their performance. Literature [5] through the big data analysis method, at present, Chinese automatic word segmentation, has made gratifying achievements in the application fields of information retrieval, automatic indexing, automatic summarization, machine translation, language research, search engine, natural language understanding, and Chinese information processing. In literature [15], text word segmentation is composed of several sentences in the original text, and text word segmentation is composed of new sentences summarizing the full text. Because the latter requires a lot of accumulation of grammar rules for different languages and many complex algorithm operations, the research on it at home and abroad is still very limited. Therefore, at present, most of the research objects of text automatic word segmentation are abstracts. In literature [16], most systems using this method alone lack the ability of ambiguity resolution and the segmentation accuracy is relatively low. CDWs is the first practical automatic word segmentation system in China. The word segmentation accuracy of CDWs is about 1/626. In literature [17], the system is divided into four core modules: initial segmentation, part-of-speech tagging intersection ambiguity field resolution, combined ambiguity field resolution, and unknown word recognition. Literature [18] shows that with the development of research, many word segmentation systems have successively increased the ability of rule disambiguation. However, because rule-based word segmentation systems need to provide a large amount of segmentation knowledge manually, it is difficult to meet people’s requirements for system openness, and the limitations of rule systems are exposed. Literature [19] points out that there are three types of ambiguity in automatic word segmentation, and different types of ambiguity have different origins and elimination methods. Therefore, different solutions should be adopted for different ambiguity types. In literature [20], through the method of big data analysis, the main difficulty of automatic word segmentation is that the meaning of Chinese automatic word segmentation is ambiguous, but ambiguous automatic segmentation is very common. With the research of automatic segmentation, the research of fuzzy processing of a word has also made great progress. The previous disambiguation methods can be roughly divided into two categories: regular methods and statistical methods. Literature [21] shows that with the development of Chinese automatic word segmentation technology, the further development of this research will be more widely applied to more fields, such as content analysis, cognitive psychology, concept analysis, word frequency statistics, linguistics, and Chinese. Literature [22] points out that the number of word segmentation systems using statistical methods is increasing gradually in China, and the statistical information is not limited to simple word frequency information but uses higher-level information such as part of speech to correct segmentation ambiguity, which is a new idea of Chinese word segmentation.This paper investigates the Chinese automatic word segmentation system and develops a practical modern Chinese word segmentation system based on complex network characteristics. The system’s word segmentation accuracy is high, and its applicability is broad. It has a user-friendly interface, is simple to use, and is simple to expand, maintain, and upgrade. The system excels at processing ambiguous fields and recognizing words that have not been registered. In other words, complex networks will become a new starting point for research in complexity science. Networks will be better applied in various research fields due to their universality.


## 3. Principle and Model of Complex Network Characteristics

Although the time of complex network research is only a few decades, it has become a hotspot in many fields such as mathematics, biology, economics, and computer science. Complex network is an important means of complex system research. Scholars have conducted modeling, description, and demonstration of complex network from various fields. The research of complex network theory originated from graph theory, which is essentially a graph structure composed of a large number of nodes and their interactions, in which the nodes are abstracted from the basic elements or some phenomena in the system, and the edges are defined by the relationship between the two elements of the research object.The construction of network model is very important for the research of complex networks. Many years ago, scholars have established regular network model and random network model. These network models are introduced below. Chinese automatic word segmentation system mainly organizes and analyzes language text information and uses some existing technologies, such as information retrieval, information filtering, information extraction, data mining, and natural language processing, to provide effective core information to users. The processing hierarchy is shown in Figure1.Figure 1
Structure diagram of Chinese automatic word segmentation system based on complex network features.The concept of hierarchical network information processing is introduced in Figure1, and its high-level analysis is based on low-level processing, which is based on the underlying data source. To deal with the Chinese automatic word segmentation system, we must first obtain the Chinese automatic word segmentation data, then preprocess the data using low-level processing such as web page filtering and grammar analysis, and finally perform keyword extraction, abstract generation, clustering, hotspot discovery, and tracking of the Chinese automatic word segmentation using high-level analysis, among other things. People can quickly and accurately get the current hotspots that people pay attention to by using Chinese automatic word segmentation, as shown in Figure 2.Figure 2
General schematic diagram of Chinese automatic word segmentation.Through the related methods of news web page information processing, keywords are extracted from the news web page information, and the news web pages are clustered and then matched with the user’s news browsing log or user-defined related information to realize personalized news information recommendation, so as to realize intelligent service and reduce the time for users to browse news web pages.Complex networks are essentially the result of abstracting complex systems in the real world. A complex network can be represented as a graphG = (V, e), where V = {VI, Li = 1, ... ,n} represents the set of nodes, and N is the number of nodes; E represents the set of edges and m=E represents the number of edges. According to whether the connection edge between nodes has direction, the network can be divided into undirected network and directed network, as shown in Figures 3 and 4.Figure 3
Undirected graph.Figure 4
Directed graph.On the contrary, if there is no weight on all connection edges in the network, this network is called unauthorized network. If the connecting edge of any two nodes in a network has both direction and weight, the network is called directed weighted network, which is undirected weighted network and directed weighted network, respectively, as shown in Figures5 and 6.Figure 5
Undirected weighted graph.Figure 6
Directed weighted graph.G=(V, E), generally expressed by adjacency matrix, which is denoted as A. In an undirected graph, use (vi, vj) to represent an undirected edge; in a directed graph, vi,vj is used to represent a directed edge. For undirected and unweighted networks, if there is an edge connection between node I and node J, then Aij = 1; otherwise, it is 0. For a directed unauthorized network, if there is a directed edge from the starting node I to the ending node J, then Aij = 1; otherwise, it is 0. For the weighted network, the weighted network can be expressed as G = (V, E, W), Wij is the weight of the connection edge between node I and node J. In the adjacency matrix, if there is a connection edge between node I and node J, then Aij = Wij is 0 otherwise.Generally, it is defined as a set of nodes and edges with “tight inside and loose outside” topology. To describe this feature, Newman and Girvan put forward the definition of modularityQ, as shown in the following:(1)Q=12m∑ijAij−kikj2mδCi,Cj,where Ki and kj represent the degrees of node i and node j, Ci and Cj represent the communities of node i and node j, respectively, and M represents the total number of edges of the network. If Ci = Cj, then δCi,Cj=1, otherwise 0. In a given complex network, different partition results lead to different modularity. The larger the Q value, the more accurate the algorithm partition result is. The quality of community partition result can be evaluated by modularity.The modularity degree is applied to the weighted network, and the degreeki of node I is replaced by the sum of weights of all connected edges of node si, and the total number of edges of the network M is replaced by the sum of total weights of all connected edges. Finally, the weighted network’s modularity degree is calculated, and its definition is given in the following:(2)Qw=12W∑ijWij−SiSj2WδCi,Cj.In order to detect overlapping communities, Shen et al. defined overlapping modularity, as shown in the following:(3)Q=12m∑ij1OiOjAij−kikj2mδCi,Cj,where Oi represents the number of communities to which node I belongs.Use an algorithm to calculate the distance, whereI is the identity matrix of nn, and B(j) represents the transfer matrix.(4)dij=∑l=1N1I−Bjil.The algorithm is applied to community detection with attribute characteristics of nodes. The algorithm uses random walk strategy to give the distance function of neighbor random walk, as shown in the following:(5)dij=∑τ:i˗jlengthτ≤lpτc1−clengthτ,where l is the length of random walk and c is the restart probability, which is a path from node I to node J when the transition probability is p (τ) and the path length is length (τ).The location of nodes is judged by the peak-valley structure presented in the topology structure, and the overlapping points are in the peak-valley structure, which is in the valley position. The position of each node in the network in the topology is obtained by the following formula:(6)ϕvi=∑l=1kmvl×e−dil/σ2.Vi indicates the nodes within the influence range of node vi, k indicates the total number of nodes within the influence range of node vi, and m(vl) indicates the importance of node vl.Given the modelλ and the output observation sequence O = o1AoT, the probability of obtaining a set of state sequences Q = q1Aqt is P(Q | 0), which is known from the following formula:(7)pQ|O=pO∣QpQ∑QpO∣QpQ.Further derivation can be obtained.(8)pO∣Q=∏t=1TpOt∣qt=bq1O1bq2O2AbqrOT,pQ=πq1aq1q2aq2q3AaqT−1qT.In this way, the probability of any state sequence of a given observation sequence can be obtained from the above formula.


## 4. Design and Implementation of Chinese Automatic Word Segmentation System

### 4.1. Design of Chinese Automatic Word Segmentation System Based on Complex Network Features

Complex networks are ubiquitous. Almost all complex systems contain multiple agents, and there are various relationships and constraints among agents, thus forming a complex network system. Based on the strategy of multistep processing, considering maintainability and upgradability, the module division and system structure design are carried out. The system is divided into four core modules: initial segmentation, part-of-speech tagging intersection ambiguity field resolution, combined ambiguity field resolution, and unknown word recognition. In order to analyze the complex characteristics of the power grid, it is necessary to understand the complex characteristicsW of the complex network and the propagation characteristics of cascading failures in the complex system. Chinese is a combination of words in form. In a corpus large enough, the probability of meaningful words often reflects certain statistical rules, that is, the higher the frequency of adjacent words, the greater the possibility of forming words. Therefore, the adjacent relationship between Chinese characters can explain the possibility of some Chinese characters becoming words. The vocabulary in the dictionary is always much smaller than the vocabulary encountered in practical application.The research ideas for complex networks are as follows: empirical research on complex systems and analysis of the statistical characteristics of real networks; build a complex system model in line with the statistical characteristics of the real network, study the formation mechanismW and internal mechanism of the network, and study the dynamic behavior on the network, such as the robustness of the network and the propagation behavior of cascading faults. After the unregistered word recognition, a time word recognition module is added. The reason for this arrangement is as follows: first, the time word recognition needs the result of part-of-speech tagging; secondly, more accurate contextual part-of-speech information can be provided after unregistered word recognition; another important reason is that time words may introduce very complex ambiguous fields. Using the characteristic parameters of complex networks, the performance of static networks is analyzed, such as characteristic path length, node degree and degree cumulative distribution, node intermediate number and intermediate cumulative distribution, and clustering coefficient. Through the distribution of these parameters, we can make a basic evaluation of the network structure and then use the way of removing nodes to artificially cause chain faults and large-scale accidents. Mechanical word segmentation based on thesaurus has small time and space overhead and simple algorithm, but it is difficult to deal with the words not included in the thesaurus. The number of sentence segmentation paths is in geometric progression with the length of the sentence. When segmenting the input text, because the length of the text and the number of sentences contained in the text are not known in advance, if sentence breaking processing is not carried out, the actual processing string may be very long and occupy a lot of time and space.


#### 4.1.1. Word Segmentation Method Based on String Matching

At present, many word segmentation methods have been implemented. In this long-term research and practice process, the determination of word segmentation units, ambiguous field processing, and unknown word recognition have become three major problems that perplex us.


#### 4.1.2. Word Segmentation Method Based on Understanding

This method of word segmentation can recognize words by making computers simulate people’s understanding of sentences.Breaking the input text with punctuation as the boundary is a reasonable approach. The binary or ternary model is commonly used because it does not lose too much context information and greatly reduces time and space complexity. The sentences in the article must be divided into strings using only Chinese characters before formal word segmentation based on complex network features can be performed. Punctuation marks, numbers, letters, numerical symbols, and characters from other languages may appear in complex network sentences. These symbols should be recognized and processed separately from the rest of the sentence. Strings containing only Chinese characters are segmented.


### 4.2. Experimental Results and Analysis

In English writing, words are naturally delimited by spaces, so it is intuitive in word understanding, while Chinese simply demarcates words only by punctuation or paragraphs between sentences, but there is no such delimiter between words. The complex network feature algorithm proposed in this section is compared with the evaluation results of three other algorithms, namely, text rank and K-means clustering and decision tree algorithm for Chinese automatic word segmentation based on complex network features, and three experiments are conducted, respectively, as shown in Figures7, 8, and 9.Figure 7
Comparison of the first experimental results of the evaluation of Chinese automatic word segmentation based on complex network features.Figure 8
Comparison of the second experimental results of evaluation of Chinese automatic word segmentation based on complex network features.Figure 9
Comparison of the third experimental results of evaluation of Chinese automatic word segmentation based on complex network features.This demonstrates that the Chinese automatic word segmentation algorithm suggested in this part, which is based on a combinatorial optimization problem, has certain benefits in Chinese automatic word segmentation extraction. Without specific word segmentation, the corpus information is counted regularly and the frequent sequences are obtained to update the word segmentation dictionary so that the dictionary can correctly match new words, proper nouns, terms, and so on. The dictionary processed in this way not only overcomes the defects of the traditional mechanical word segmentation thesaurus but also has the characteristics of fast speed and high efficiency.As experimental data, 600 articles ranging in length from 600 to 10,000 words were extracted from Sina, Baidu, Tencent, Tianya, and other websites. These included news, blog posts, forum posts, and technical papers. This paper compares the method of complex network features proposed in this section to three other effective methods of Chinese automatic segmentation, namely, text rank, K-means clustering algorithm, and decision tree algorithm, the optimization-based algorithm proposed in Section3, and two algorithms, Com-D′ and Com-Pa', which were chosen in Antiqueira and others’ research. The following is a summary of the experimental findings and analysis. The results of four different experiments were compared. Figures 10, 11, 12, and 13 show a comparison of the evaluation results of Chinese automatic word segmentation based on complex network features at various compression rates.Figure 10
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 11
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 12
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 13
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.The experimental results show that from Figures10 to 13, it can be seen that the complex network feature algorithm proposed in this section has obvious advantages in the price results of Chinese automatic word segmentation. In addition, three conclusions can be drawn from the experimental data in the figure: (1) among the four different summary extraction methods proposed in this section, the complex network feature algorithm and K-means algorithm are better than other extraction methods in effect, which can show that the position of nodes in the community and the degree of nodes are two important feature attributes for evaluating sentences; (2) the experimental results of complex network feature algorithm and K-means algorithm are higher than those of text rank and decision tree algorithm, which proves that the combination of community division and sentence statistical attributes to extract Chinese automatic word segmentation is more effective than using sentence statistical attributes alone to extract Chinese automatic word segmentation; (3) the complex network feature algorithm proposed in this section has more obvious advantages in compression rate, which shows that the Chinese automatic word segmentation generated by the complex network feature algorithm proposed in this section is closer to the reference Chinese automatic word segmentation, which proves that using the complex network community division theory to divide the article topic for text automatic Chinese automatic word segmentation has certain research value.


## 4.1. Design of Chinese Automatic Word Segmentation System Based on Complex Network Features

Complex networks are ubiquitous. Almost all complex systems contain multiple agents, and there are various relationships and constraints among agents, thus forming a complex network system. Based on the strategy of multistep processing, considering maintainability and upgradability, the module division and system structure design are carried out. The system is divided into four core modules: initial segmentation, part-of-speech tagging intersection ambiguity field resolution, combined ambiguity field resolution, and unknown word recognition. In order to analyze the complex characteristics of the power grid, it is necessary to understand the complex characteristicsW of the complex network and the propagation characteristics of cascading failures in the complex system. Chinese is a combination of words in form. In a corpus large enough, the probability of meaningful words often reflects certain statistical rules, that is, the higher the frequency of adjacent words, the greater the possibility of forming words. Therefore, the adjacent relationship between Chinese characters can explain the possibility of some Chinese characters becoming words. The vocabulary in the dictionary is always much smaller than the vocabulary encountered in practical application.The research ideas for complex networks are as follows: empirical research on complex systems and analysis of the statistical characteristics of real networks; build a complex system model in line with the statistical characteristics of the real network, study the formation mechanismW and internal mechanism of the network, and study the dynamic behavior on the network, such as the robustness of the network and the propagation behavior of cascading faults. After the unregistered word recognition, a time word recognition module is added. The reason for this arrangement is as follows: first, the time word recognition needs the result of part-of-speech tagging; secondly, more accurate contextual part-of-speech information can be provided after unregistered word recognition; another important reason is that time words may introduce very complex ambiguous fields. Using the characteristic parameters of complex networks, the performance of static networks is analyzed, such as characteristic path length, node degree and degree cumulative distribution, node intermediate number and intermediate cumulative distribution, and clustering coefficient. Through the distribution of these parameters, we can make a basic evaluation of the network structure and then use the way of removing nodes to artificially cause chain faults and large-scale accidents. Mechanical word segmentation based on thesaurus has small time and space overhead and simple algorithm, but it is difficult to deal with the words not included in the thesaurus. The number of sentence segmentation paths is in geometric progression with the length of the sentence. When segmenting the input text, because the length of the text and the number of sentences contained in the text are not known in advance, if sentence breaking processing is not carried out, the actual processing string may be very long and occupy a lot of time and space.


### 4.1.1. Word Segmentation Method Based on String Matching

At present, many word segmentation methods have been implemented. In this long-term research and practice process, the determination of word segmentation units, ambiguous field processing, and unknown word recognition have become three major problems that perplex us.


### 4.1.2. Word Segmentation Method Based on Understanding

This method of word segmentation can recognize words by making computers simulate people’s understanding of sentences.Breaking the input text with punctuation as the boundary is a reasonable approach. The binary or ternary model is commonly used because it does not lose too much context information and greatly reduces time and space complexity. The sentences in the article must be divided into strings using only Chinese characters before formal word segmentation based on complex network features can be performed. Punctuation marks, numbers, letters, numerical symbols, and characters from other languages may appear in complex network sentences. These symbols should be recognized and processed separately from the rest of the sentence. Strings containing only Chinese characters are segmented.


## 4.1.1. Word Segmentation Method Based on String Matching

At present, many word segmentation methods have been implemented. In this long-term research and practice process, the determination of word segmentation units, ambiguous field processing, and unknown word recognition have become three major problems that perplex us.


## 4.1.2. Word Segmentation Method Based on Understanding

This method of word segmentation can recognize words by making computers simulate people’s understanding of sentences.Breaking the input text with punctuation as the boundary is a reasonable approach. The binary or ternary model is commonly used because it does not lose too much context information and greatly reduces time and space complexity. The sentences in the article must be divided into strings using only Chinese characters before formal word segmentation based on complex network features can be performed. Punctuation marks, numbers, letters, numerical symbols, and characters from other languages may appear in complex network sentences. These symbols should be recognized and processed separately from the rest of the sentence. Strings containing only Chinese characters are segmented.


## 4.2. Experimental Results and Analysis

In English writing, words are naturally delimited by spaces, so it is intuitive in word understanding, while Chinese simply demarcates words only by punctuation or paragraphs between sentences, but there is no such delimiter between words. The complex network feature algorithm proposed in this section is compared with the evaluation results of three other algorithms, namely, text rank and K-means clustering and decision tree algorithm for Chinese automatic word segmentation based on complex network features, and three experiments are conducted, respectively, as shown in Figures7, 8, and 9.Figure 7
Comparison of the first experimental results of the evaluation of Chinese automatic word segmentation based on complex network features.Figure 8
Comparison of the second experimental results of evaluation of Chinese automatic word segmentation based on complex network features.Figure 9
Comparison of the third experimental results of evaluation of Chinese automatic word segmentation based on complex network features.This demonstrates that the Chinese automatic word segmentation algorithm suggested in this part, which is based on a combinatorial optimization problem, has certain benefits in Chinese automatic word segmentation extraction. Without specific word segmentation, the corpus information is counted regularly and the frequent sequences are obtained to update the word segmentation dictionary so that the dictionary can correctly match new words, proper nouns, terms, and so on. The dictionary processed in this way not only overcomes the defects of the traditional mechanical word segmentation thesaurus but also has the characteristics of fast speed and high efficiency.As experimental data, 600 articles ranging in length from 600 to 10,000 words were extracted from Sina, Baidu, Tencent, Tianya, and other websites. These included news, blog posts, forum posts, and technical papers. This paper compares the method of complex network features proposed in this section to three other effective methods of Chinese automatic segmentation, namely, text rank, K-means clustering algorithm, and decision tree algorithm, the optimization-based algorithm proposed in Section3, and two algorithms, Com-D′ and Com-Pa', which were chosen in Antiqueira and others’ research. The following is a summary of the experimental findings and analysis. The results of four different experiments were compared. Figures 10, 11, 12, and 13 show a comparison of the evaluation results of Chinese automatic word segmentation based on complex network features at various compression rates.Figure 10
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 11
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 12
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.Figure 13
Comparison of experimental results of Chinese automatic word segmentation evaluation based on complex network features.The experimental results show that from Figures10 to 13, it can be seen that the complex network feature algorithm proposed in this section has obvious advantages in the price results of Chinese automatic word segmentation. In addition, three conclusions can be drawn from the experimental data in the figure: (1) among the four different summary extraction methods proposed in this section, the complex network feature algorithm and K-means algorithm are better than other extraction methods in effect, which can show that the position of nodes in the community and the degree of nodes are two important feature attributes for evaluating sentences; (2) the experimental results of complex network feature algorithm and K-means algorithm are higher than those of text rank and decision tree algorithm, which proves that the combination of community division and sentence statistical attributes to extract Chinese automatic word segmentation is more effective than using sentence statistical attributes alone to extract Chinese automatic word segmentation; (3) the complex network feature algorithm proposed in this section has more obvious advantages in compression rate, which shows that the Chinese automatic word segmentation generated by the complex network feature algorithm proposed in this section is closer to the reference Chinese automatic word segmentation, which proves that using the complex network community division theory to divide the article topic for text automatic Chinese automatic word segmentation has certain research value.


## 5. Conclusions

Complex networks are everywhere. Almost all complex systems have multiple agents, each with their own set of relationships and constraints, resulting in a complex network system. The module division and system structure design are carried out using a multistep processing strategy that considers maintainability and upgradability. The word segmentation methods used in the word segmentation system currently make use of some natural language processing technologies, but they are unable to fully solve the problem of Chinese grammar analysis. The study of various word segmentation algorithms is presented in this paper. Because people’s current rule reasoning and state transition mechanisms do not fully express the mechanism of human brain thinking, the processing of complex and fuzzy semantic information still appears powerless, and the construction of a word segmentation knowledge base and the selection of information in dictionaries both have significant human factors. The complex network feature algorithm can better resolve the conflict between word segmentation speed and accuracy, as well as meet the needs of natural language processing systems for Chinese word segmentation speed and accuracy. At the same time, the system lacks effective self-learning and relearning mechanisms, so future grammar and semantics research in natural language processing will inevitably gravitate toward neural network segmentation and understanding systems.



---

*Source: 3812643-2022-02-09.xml*