# Knowledge Graph Question and Answer System for Mechanical Intelligent Manufacturing Based on Deep Learning

Authors: Miaoyuan Shi
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

With the development of deep learning and its wide application in the field of natural language, the question and answer research of knowledge graph based on deep learning has gradually become the focus of attention. After that, the natural language query is converted into a structured query sentence to identify the entities and attributes in the user’s natural language query and the specified entities and attributes are used to retrieve answers to the knowledge graph. Using the advantage of deep learning in capturing sentence information, it incorporates the attention mechanism to obtain the semantic vector of the relevant attributes in the query and uses the parameter sharing mechanism to insert candidate attributes into the triple in the same model to obtain the semantic vector of typical candidates. The experiment measured that under the 100,000 RDF dataset, the single entity query of the MIQE model does not exceed 3 seconds, and the connection query does not exceed 5 seconds. Under the one-million RDF dataset, the single entity query of the MIQE model does not exceed 8 seconds, and the connection query will not be more than 10 seconds. Experimental data show that the system of knowledge-answering questions of engineering of intelligent construction based on deep learning has good horizontal scalability.


---

## Body

## 1. Introduction

### 1.1. Background and Significance

With the rapid development of Internet information technology and the rapid growth of information volume, storing and retrieving massive amounts of data are a difficult task. Using search engines to find the information people need from massive amounts of data has been a research in the field of information retrieval. However, the search engine still has its limitations. It cannot understand the real needs of the user and returns an accurate answer to the user, but only checks and sorts the information about the user. Designing a knowledge-based answering system for intelligent engineering based on deep learning is a very popular research direction in the field of natural language processing. It is an information retrieval system that covers many areas of technology, such as language processing, machine learning, and data mining.The question understanding method of answering system based on knowledge graph is divided into three levels: article, entity, and relationship. Sort question sentences from the sentence level to get the user’s intention or answer category. From this, you can better understand the questions in this article and conduct the next survey. From the entity level and relationship level, the natural language question entity performance is correctly linked to the explanatory diagram to determine the relationship of the corresponding question and use the explanatory diagram to appropriately answer the user’s question, which can be answered faster and correctly [1, 2].


### 1.2. Related Work

With the rise of artificial intelligence, question answering systems have been fully studied. Zeng proposed the definition of random variables and integration. He combined deep learning algorithms and proposed a new unified definition of mutual information, which defines the joint distribution of two random variables by considering the marginal probability [3]. An integrated cause-effect graph is also proposed, which has the process knowledge extracted from the text and can be used to determine visual answers based on information retrieval techniques [4]. However, his selected literature is limited, and the theory is not systematically mature enough. Sawant et al. believe that in web search, queries that seek entities often trigger special question answering (QA) systems. It can use a parser to interpret the problem as a structured query, execute it on a knowledge graph (KG), and then return a direct entity response. QA systems based on precise parsing are often very fragile: subtle changes in grammar can greatly change the response. In addition, the coverage of the query is fragmented. At the other extreme, a large corpus can provide wider coverage, but the form is irregular and unreliable. They introduced AQQUCN, a quality inspection system that cleverly combines KG and corpus evidence. AQQUCN accepts a wide range of query syntax, including correctly formatted questions and short telegram keyword sequences. Faced with the inherent query ambiguity, AQQUCN aggregates the signals from KG and large corpora to directly rank KG entities, rather than perform a semantic interpretation of the query. AQQUCN models ideal interpretations as unobservable or latent variables [5]. Hu et al. found that RDF questions/answers (Q/A) in the research allow users to ask questions in natural language on the knowledge base represented by RDF. In order to answer natural language questions, the existing work uses a two-stage approach: problem understanding and query evaluation. Their focus is on understanding the problem to solve the ambiguity of natural language phrases. The most common technique is joint disambiguation, which has an exponential search space. He proposed a systematic framework for answering natural language questions on the RDF repository (RDF Q/A) from a graphical data-driven perspective. A semantic query graph is also proposed to model the query intent of natural language problems in a structured way, and on this basis, RDF Q/A is simplified to the subgraph matching problem. More importantly, when a query match is found, the ambiguity of the natural language problem is solved. If no match is found, the cost of disambiguation will be saved [6].


### 1.3. Innovation in This Article

The main innovations of this paper include the following aspects: (1) this paper proposes a deep learning-based intelligent manufacturing knowledge graph problem generation model, introduces the details of the model in detail, and compares it with the template-based model. (2) Using the idea of conflicting genetic networks, a semisupervised educational framework is designed that combines the creation of knowledge graph problems and the matching of questions and answers. When the number of educational data is small, the results of both tasks are achieved.


## 1.1. Background and Significance

With the rapid development of Internet information technology and the rapid growth of information volume, storing and retrieving massive amounts of data are a difficult task. Using search engines to find the information people need from massive amounts of data has been a research in the field of information retrieval. However, the search engine still has its limitations. It cannot understand the real needs of the user and returns an accurate answer to the user, but only checks and sorts the information about the user. Designing a knowledge-based answering system for intelligent engineering based on deep learning is a very popular research direction in the field of natural language processing. It is an information retrieval system that covers many areas of technology, such as language processing, machine learning, and data mining.The question understanding method of answering system based on knowledge graph is divided into three levels: article, entity, and relationship. Sort question sentences from the sentence level to get the user’s intention or answer category. From this, you can better understand the questions in this article and conduct the next survey. From the entity level and relationship level, the natural language question entity performance is correctly linked to the explanatory diagram to determine the relationship of the corresponding question and use the explanatory diagram to appropriately answer the user’s question, which can be answered faster and correctly [1, 2].


## 1.2. Related Work

With the rise of artificial intelligence, question answering systems have been fully studied. Zeng proposed the definition of random variables and integration. He combined deep learning algorithms and proposed a new unified definition of mutual information, which defines the joint distribution of two random variables by considering the marginal probability [3]. An integrated cause-effect graph is also proposed, which has the process knowledge extracted from the text and can be used to determine visual answers based on information retrieval techniques [4]. However, his selected literature is limited, and the theory is not systematically mature enough. Sawant et al. believe that in web search, queries that seek entities often trigger special question answering (QA) systems. It can use a parser to interpret the problem as a structured query, execute it on a knowledge graph (KG), and then return a direct entity response. QA systems based on precise parsing are often very fragile: subtle changes in grammar can greatly change the response. In addition, the coverage of the query is fragmented. At the other extreme, a large corpus can provide wider coverage, but the form is irregular and unreliable. They introduced AQQUCN, a quality inspection system that cleverly combines KG and corpus evidence. AQQUCN accepts a wide range of query syntax, including correctly formatted questions and short telegram keyword sequences. Faced with the inherent query ambiguity, AQQUCN aggregates the signals from KG and large corpora to directly rank KG entities, rather than perform a semantic interpretation of the query. AQQUCN models ideal interpretations as unobservable or latent variables [5]. Hu et al. found that RDF questions/answers (Q/A) in the research allow users to ask questions in natural language on the knowledge base represented by RDF. In order to answer natural language questions, the existing work uses a two-stage approach: problem understanding and query evaluation. Their focus is on understanding the problem to solve the ambiguity of natural language phrases. The most common technique is joint disambiguation, which has an exponential search space. He proposed a systematic framework for answering natural language questions on the RDF repository (RDF Q/A) from a graphical data-driven perspective. A semantic query graph is also proposed to model the query intent of natural language problems in a structured way, and on this basis, RDF Q/A is simplified to the subgraph matching problem. More importantly, when a query match is found, the ambiguity of the natural language problem is solved. If no match is found, the cost of disambiguation will be saved [6].


## 1.3. Innovation in This Article

The main innovations of this paper include the following aspects: (1) this paper proposes a deep learning-based intelligent manufacturing knowledge graph problem generation model, introduces the details of the model in detail, and compares it with the template-based model. (2) Using the idea of conflicting genetic networks, a semisupervised educational framework is designed that combines the creation of knowledge graph problems and the matching of questions and answers. When the number of educational data is small, the results of both tasks are achieved.


## 2. Deep Learning-Based Knowledge Question Answering System for Mechanical Intelligent Manufacturing

### 2.1. Deep Learning

Deep learning is a popular research direction in the computer field in the recent years. A deep network structure based on multiple hidden layers is formed by combining low-level features to form more abstract high-level features [7, 8]. Traditional machine learning is difficult to directly process the original data, because it is usually extracted manually when extracting features. At this time, professional knowledge needs to be used to design a feature extraction that can convert the original data into a feature representation for the machine learning system [9, 10]. Deep learning is a method that uses raw data to automatically discover internal feature expressions. Deep learning is a method that uses raw data to automatically discover internal expressions of features. It converts raw data into higher-level and more abstract feature expressions through a large number of nonlinear transformation combinations, so that deep neural networks can learn very complex operation. The idea of deep learning is to directly extract features from the original data without using artificial features to extract features [11].Convolutional neural network is a well-known deep learning framework inspired by the mechanism of natural visual perception. It is a multilevel deep feedforward artificial neural network composed of multiple two-dimensional planes [12, 13]. Its neurons can respond to some surrounding units within the coverage area and have achieved very good results in image processing problems. Due to its characteristics, it is widely used in image recognition and image classification tasks [14, 15]. Thus, the framework of the convolutional neural network model was established, and a multilayer artificial neural network for recognizing handwritten digits was designed [16]. The network can effectively represent image features, and at that time, it directly recognized visual patterns from original pixels. This makes image visual recognition possible and also provides help for subsequent image recognition research [17, 18].The repetitive neural network is a typical model of machine learning supervision. It contains loops that change over time and is a simple ring connection simulation. In RNN, the output of a sequence at the current time is not only related to the current time state but also related to the output of the previous time. Specifically, when calculating the output of the current time, the RNN will apply the previously memorized information to the current calculation [19, 20].


### 2.2. Characteristics of Mechanical Intelligent Manufacturing

Mechanical intelligent manufacturing is a comprehensive application of AI (artificial intelligence), an intelligent system integrating man and machine, and a manifestation of knowledge and intelligence. Knowledge is the basis of intelligence and intelligence is the ability to apply knowledge [21, 22]. Intelligent manufacturing is a technology and system that enables the entire manufacturing system to have self-awareness, self-diagnosis, adaptation, self-learning, self-cooperation, and self-organization capabilities. It is a man-machine integrated intelligent system built by smart equipment and industry experts. In the manufacturing process, let the manufacturing system have independent thinking and be able to carry out intelligent activities such as analysis, reasoning, judgment, conception, and decision-making [23, 24]. From the background of manufacturing, today’s excess capacity is a common problem in manufacturing, and the new market model has transformed manufacturing into a service industry. The original model can no longer meet the increasingly personalized needs and service experience. The final argument can provide a more personalized and considerate service, who can win the market [25, 26]. From the perspective of the national government, many manufacturing countries have formulated a number of advanced manufacturing development strategies and plans such as adjusting the industrial structure, such as the United States’ advanced manufacturing partnership program, Germany’s Industry 4.0, and the European Union's digital Europe. They promote industrial transformation, provide listings for companies, provide financial support, and promote the improvement of enterprise informationization. Now is the right time for manufacturing to build intelligent manufacturing [27, 28].Mechanical intelligent manufacturing includes intelligent manufacturing technology (IMT) and intelligent manufacturing system (IMS). From a microperspective, as an advanced manufacturing technology, intelligent manufacturing technology mainly realizes intelligent control of product design and production, packaging and distribution, inspection and transportation, information feedback, and other processes through the intelligent control of computer and other hardware information equipment. The production process reduces a large number of manpower operations, extends the brain wisdom of experts, and realizes the integration of man and machine [29]. From a macroeconomic point of view, the intelligent manufacturing system is a systematic and integrated application of various intelligent manufacturing technologies at various stages and modules of product production, so that the manufacturing industry is an integrated and interconnected fully automated production system. Intelligent manufacturing is based on the combination of a new generation of information physics system and advanced manufacturing technology. It can implement perceptual analysis, self-decision, and self-execution of the entire process of data and information and realize a new type of manufacturing mode with the most optimized benefits. Compared with the traditional manufacturing model, the core content of the smart manufacturing model at this stage includes smart production, smart products, smart factories, and smart logistics. The specific model is shown in Figure 1.Figure 1
The core content model of intelligent manufacturing.According to the abovementioned definition of mechanical intelligent manufacturing, it can be seen that mechanical intelligent manufacturing is a product of the combination of information technology and manufacturing technology in the 21st century, and its realization must be based on the improvement of information technology. Under the requirements and guidance of the “Made in China 2025” strategy, smart equipment manufacturers use smart equipment, production, products, management, operation, and maintenance as a starting point and use emerging information, networks, and information to improve their core competitiveness.


### 2.3. Knowledge Graph Question and Answer System Based on Deep Learning

Knowledge Graph A graph-based structured knowledge database describes the concepts and their relationships in the real world in the form of symbols. Its nodes represent entities or concepts, corresponding to the semantic ontology in the real world, and edges represent the mutual relationship between entities and concepts, connecting different types of entities. Triples in the form of {entity, relationship, entity} ({subject, relation, object}) facts are the basic unit of the knowledge graph, and the knowledge graph also contains the attribute description of the entity and its basic structure, as shown in Figure2.Figure 2
Schematic diagram of the basic components of the knowledge graph.The knowledge-based question and answer system is the way to answer questions according to the questions asked by users. There are currently two main types: one is to realize the structured representation of problems through semantic analysis. Methods based on query templates, problem templates, and dependency relationships are typical methods. Starting from the questions raised by users, the user questions are converted into a structured representation of the relationship path, and the fuzzy relationship matching method based on the statistical model is used to understand the question. The question answering system based on knowledge graph is to preclassify and analyze the problem first and then convert the problem into the form of problem triplet to understand the problem. One is to structure the problem through vector representation. First, the unstructured user question is converted into a structured knowledge graph query problem. By using two trained neural networks, the appropriate result is found for the query problem.Most existing KBQA in-depth learning methods follow a coder comparison framework. Use deep learning to pick answers to a survey and open a project. To implement a simple question and answer based on a convergent neural network, first use the n-gram query text to search the knowledge base to create a set of KB candidate items, and then map the query and KB items. The answer to the question is obtained by calculating the correlation between the similarity of the two embedded vectors and the facts in the knowledge base. First of all, the representation learning method of word vectors is used for question and answer based on knowledge graphs. From the analysis of the structure of data stored in the knowledge base, the structure of the triples stored in the knowledge base is always <subject, relation, object>. Looking for the answer to the question becomes the process of finding the most relevant triples in the knowledge base. Through natural language processing methods such as entity recognition (NER), the model is divided into two modules: entity recognition and relationship recognition. The high accuracy of the two recognition modules improves the accuracy of question answering.


### 2.4. Back Propagation Algorithm

In deep learning, the convergent neural network optimization algorithm is generally a stochastic slope descent (SSD). SSD requires the partial derivative of the loss functionC in relation to each weight parameter W and b, but the network model has many weights and compensations. Therefore, the reverse propagation algorithm is proposed for the most efficient calculation of the partial derivative. The process of calculating the reverse propagation algorithm will be explained as follows:Assume that the loss function of the network is(1)C=12n∑yx−aLx2.In formula (1), y(x) is the expected output value, n is the total number of training samples, aL(x) is the output vector of the network, and L is the number of layers of the network. Hypothesis 1: the total loss function can be expressed by averaging the single loss functions:(2)C=1n∑Cx,Cx=12y−aL.Hypothesis 2: the loss function can be expressed as the network output function, so the loss function of a single sample can be expressed as follows:(3)Cx=12y−aL2=12∑jyj−ajL2.Error equation of output layer:(4)δjL=∂C∂ZjL=∂C∂ajL∂ajL∂ZjL=∂C∂ajLσ′ZjL.In formula (4), δjL represents the error of the jth unit of the Lth layer. ∂C/∂ZjL measures the speed of the change of the loss function with the output of the network. σ′LjLc measures the speed of the change of the output of the activation function with ZjL. When σ′ZjL≈0, regardless of how big the value of ∂C/∂ZjL is, the value of δjL will be about 0. At this time, the output neuron enters the saturation region and the network stops learning. Error transfer equation:(5)δL=WL+1TδL+1Θσ′ZL.Equation (5) shows that the error δ′ of the Lth layer can be obtained by calculating the error δL+1 of the L + 1 layer, and the error of any layer can be calculated by combining equations (4) and (5).


## 2.1. Deep Learning

Deep learning is a popular research direction in the computer field in the recent years. A deep network structure based on multiple hidden layers is formed by combining low-level features to form more abstract high-level features [7, 8]. Traditional machine learning is difficult to directly process the original data, because it is usually extracted manually when extracting features. At this time, professional knowledge needs to be used to design a feature extraction that can convert the original data into a feature representation for the machine learning system [9, 10]. Deep learning is a method that uses raw data to automatically discover internal feature expressions. Deep learning is a method that uses raw data to automatically discover internal expressions of features. It converts raw data into higher-level and more abstract feature expressions through a large number of nonlinear transformation combinations, so that deep neural networks can learn very complex operation. The idea of deep learning is to directly extract features from the original data without using artificial features to extract features [11].Convolutional neural network is a well-known deep learning framework inspired by the mechanism of natural visual perception. It is a multilevel deep feedforward artificial neural network composed of multiple two-dimensional planes [12, 13]. Its neurons can respond to some surrounding units within the coverage area and have achieved very good results in image processing problems. Due to its characteristics, it is widely used in image recognition and image classification tasks [14, 15]. Thus, the framework of the convolutional neural network model was established, and a multilayer artificial neural network for recognizing handwritten digits was designed [16]. The network can effectively represent image features, and at that time, it directly recognized visual patterns from original pixels. This makes image visual recognition possible and also provides help for subsequent image recognition research [17, 18].The repetitive neural network is a typical model of machine learning supervision. It contains loops that change over time and is a simple ring connection simulation. In RNN, the output of a sequence at the current time is not only related to the current time state but also related to the output of the previous time. Specifically, when calculating the output of the current time, the RNN will apply the previously memorized information to the current calculation [19, 20].


## 2.2. Characteristics of Mechanical Intelligent Manufacturing

Mechanical intelligent manufacturing is a comprehensive application of AI (artificial intelligence), an intelligent system integrating man and machine, and a manifestation of knowledge and intelligence. Knowledge is the basis of intelligence and intelligence is the ability to apply knowledge [21, 22]. Intelligent manufacturing is a technology and system that enables the entire manufacturing system to have self-awareness, self-diagnosis, adaptation, self-learning, self-cooperation, and self-organization capabilities. It is a man-machine integrated intelligent system built by smart equipment and industry experts. In the manufacturing process, let the manufacturing system have independent thinking and be able to carry out intelligent activities such as analysis, reasoning, judgment, conception, and decision-making [23, 24]. From the background of manufacturing, today’s excess capacity is a common problem in manufacturing, and the new market model has transformed manufacturing into a service industry. The original model can no longer meet the increasingly personalized needs and service experience. The final argument can provide a more personalized and considerate service, who can win the market [25, 26]. From the perspective of the national government, many manufacturing countries have formulated a number of advanced manufacturing development strategies and plans such as adjusting the industrial structure, such as the United States’ advanced manufacturing partnership program, Germany’s Industry 4.0, and the European Union's digital Europe. They promote industrial transformation, provide listings for companies, provide financial support, and promote the improvement of enterprise informationization. Now is the right time for manufacturing to build intelligent manufacturing [27, 28].Mechanical intelligent manufacturing includes intelligent manufacturing technology (IMT) and intelligent manufacturing system (IMS). From a microperspective, as an advanced manufacturing technology, intelligent manufacturing technology mainly realizes intelligent control of product design and production, packaging and distribution, inspection and transportation, information feedback, and other processes through the intelligent control of computer and other hardware information equipment. The production process reduces a large number of manpower operations, extends the brain wisdom of experts, and realizes the integration of man and machine [29]. From a macroeconomic point of view, the intelligent manufacturing system is a systematic and integrated application of various intelligent manufacturing technologies at various stages and modules of product production, so that the manufacturing industry is an integrated and interconnected fully automated production system. Intelligent manufacturing is based on the combination of a new generation of information physics system and advanced manufacturing technology. It can implement perceptual analysis, self-decision, and self-execution of the entire process of data and information and realize a new type of manufacturing mode with the most optimized benefits. Compared with the traditional manufacturing model, the core content of the smart manufacturing model at this stage includes smart production, smart products, smart factories, and smart logistics. The specific model is shown in Figure 1.Figure 1
The core content model of intelligent manufacturing.According to the abovementioned definition of mechanical intelligent manufacturing, it can be seen that mechanical intelligent manufacturing is a product of the combination of information technology and manufacturing technology in the 21st century, and its realization must be based on the improvement of information technology. Under the requirements and guidance of the “Made in China 2025” strategy, smart equipment manufacturers use smart equipment, production, products, management, operation, and maintenance as a starting point and use emerging information, networks, and information to improve their core competitiveness.


## 2.3. Knowledge Graph Question and Answer System Based on Deep Learning

Knowledge Graph A graph-based structured knowledge database describes the concepts and their relationships in the real world in the form of symbols. Its nodes represent entities or concepts, corresponding to the semantic ontology in the real world, and edges represent the mutual relationship between entities and concepts, connecting different types of entities. Triples in the form of {entity, relationship, entity} ({subject, relation, object}) facts are the basic unit of the knowledge graph, and the knowledge graph also contains the attribute description of the entity and its basic structure, as shown in Figure2.Figure 2
Schematic diagram of the basic components of the knowledge graph.The knowledge-based question and answer system is the way to answer questions according to the questions asked by users. There are currently two main types: one is to realize the structured representation of problems through semantic analysis. Methods based on query templates, problem templates, and dependency relationships are typical methods. Starting from the questions raised by users, the user questions are converted into a structured representation of the relationship path, and the fuzzy relationship matching method based on the statistical model is used to understand the question. The question answering system based on knowledge graph is to preclassify and analyze the problem first and then convert the problem into the form of problem triplet to understand the problem. One is to structure the problem through vector representation. First, the unstructured user question is converted into a structured knowledge graph query problem. By using two trained neural networks, the appropriate result is found for the query problem.Most existing KBQA in-depth learning methods follow a coder comparison framework. Use deep learning to pick answers to a survey and open a project. To implement a simple question and answer based on a convergent neural network, first use the n-gram query text to search the knowledge base to create a set of KB candidate items, and then map the query and KB items. The answer to the question is obtained by calculating the correlation between the similarity of the two embedded vectors and the facts in the knowledge base. First of all, the representation learning method of word vectors is used for question and answer based on knowledge graphs. From the analysis of the structure of data stored in the knowledge base, the structure of the triples stored in the knowledge base is always <subject, relation, object>. Looking for the answer to the question becomes the process of finding the most relevant triples in the knowledge base. Through natural language processing methods such as entity recognition (NER), the model is divided into two modules: entity recognition and relationship recognition. The high accuracy of the two recognition modules improves the accuracy of question answering.


## 2.4. Back Propagation Algorithm

In deep learning, the convergent neural network optimization algorithm is generally a stochastic slope descent (SSD). SSD requires the partial derivative of the loss functionC in relation to each weight parameter W and b, but the network model has many weights and compensations. Therefore, the reverse propagation algorithm is proposed for the most efficient calculation of the partial derivative. The process of calculating the reverse propagation algorithm will be explained as follows:Assume that the loss function of the network is(1)C=12n∑yx−aLx2.In formula (1), y(x) is the expected output value, n is the total number of training samples, aL(x) is the output vector of the network, and L is the number of layers of the network. Hypothesis 1: the total loss function can be expressed by averaging the single loss functions:(2)C=1n∑Cx,Cx=12y−aL.Hypothesis 2: the loss function can be expressed as the network output function, so the loss function of a single sample can be expressed as follows:(3)Cx=12y−aL2=12∑jyj−ajL2.Error equation of output layer:(4)δjL=∂C∂ZjL=∂C∂ajL∂ajL∂ZjL=∂C∂ajLσ′ZjL.In formula (4), δjL represents the error of the jth unit of the Lth layer. ∂C/∂ZjL measures the speed of the change of the loss function with the output of the network. σ′LjLc measures the speed of the change of the output of the activation function with ZjL. When σ′ZjL≈0, regardless of how big the value of ∂C/∂ZjL is, the value of δjL will be about 0. At this time, the output neuron enters the saturation region and the network stops learning. Error transfer equation:(5)δL=WL+1TδL+1Θσ′ZL.Equation (5) shows that the error δ′ of the Lth layer can be obtained by calculating the error δL+1 of the L + 1 layer, and the error of any layer can be calculated by combining equations (4) and (5).


## 3. Knowledge Graph Question and Answer System Design

### 3.1. Main Framework of Q&A Knowledge Graph

The corpus mainly includes question and answer pairs, entity annotation corpus, and attribute annotation corpus. The question-answer corpus consists of question and answer triples <question, (entity, attribute, answer)>, and the entity tagging corpus consists of <question, entity>, which is mainly used for training of deep learning entity recognition models; attribute tagging. The corpus is composed of <questions, attributes> and is mainly used for the training of attribute classification models. Question entity recognition refers to the identification of entity references from input questions. First, the semantic representation of the question needs to be processed, and then the entity references in the question are extracted through the entity recognition model; the knowledge retrieval needs to link the identified entities go to the knowledge base entity, search the entity in the knowledge graph through the knowledge graph index, and return the entity-related triples; attribute classification first performs joint semantic characterization of the question and attribute, question and answer, and then choose from the candidate ternary. In the group, the triple with the highest score is selected as the answer according to the attribute model.


### 3.2. Knowledge Graph Question and Answer System Processing Flow

After creating the knowledge graph of this system using a convergent neural network, users can gain knowledge of the graph through front-end interaction. The front-end application uses html, javascript and CSS technologies. Through the @Controller annotation of the spring-boot box, the URL and html are correlated at the controller level. Intelligent machine building technology is used to classify the query, and to a specific template, it will correspond to a query sentence. For the entire knowledge graph answering system, the processing flow is as follows:(1)
Entity identification and entity correction:First of all, we need to perform word segmentation and entity recognition to facilitate the semantic understanding later. The purpose of named entity recognition is to recognize the pointed objects of natural language questions. At present, named entity recognition mostly adopts algorithms such as conditional random field, hidden Markov, and long and short memory network LSTM.(2)
Vectorization of sentences:A template-based question and answer is used. In the naive Bayes method, the bag of words model is used to express the question, each sentence is converted into a vector, and then the naive Bayes algorithm is used to classify the question.(3)
Problem understanding based on naive Bayes:Naive Bayes algorithm can be divided into three stages: preparation stage, classifier training stage, and application stage. For this system, the three stages are as follows: (1) the preparation stage. Design the question template manually and use the template as training data. (2) Training phase. Read the template file and use the bag of words model to represent each sentence of the template. (3) Application phase. First, perform word segmentation and named entity recognition.(4)
Knowledge base queryThe form that this system can understand is named “entity + template label”; the named entity corresponds to the entity of knowledge base, and template label corresponds to the attribute or relationship of query.


### 3.3. Knowledge Graph Data

This article uses freebase as the basic knowledge graph. There are 18 relationship categories and 42 named entity categories in this dataset, including 362,000 sentences in the training set and 256,000 triples and 143,000 sentences in the test set and 5372 triples. The label in most sentences is NA.


## 3.1. Main Framework of Q&A Knowledge Graph

The corpus mainly includes question and answer pairs, entity annotation corpus, and attribute annotation corpus. The question-answer corpus consists of question and answer triples <question, (entity, attribute, answer)>, and the entity tagging corpus consists of <question, entity>, which is mainly used for training of deep learning entity recognition models; attribute tagging. The corpus is composed of <questions, attributes> and is mainly used for the training of attribute classification models. Question entity recognition refers to the identification of entity references from input questions. First, the semantic representation of the question needs to be processed, and then the entity references in the question are extracted through the entity recognition model; the knowledge retrieval needs to link the identified entities go to the knowledge base entity, search the entity in the knowledge graph through the knowledge graph index, and return the entity-related triples; attribute classification first performs joint semantic characterization of the question and attribute, question and answer, and then choose from the candidate ternary. In the group, the triple with the highest score is selected as the answer according to the attribute model.


## 3.2. Knowledge Graph Question and Answer System Processing Flow

After creating the knowledge graph of this system using a convergent neural network, users can gain knowledge of the graph through front-end interaction. The front-end application uses html, javascript and CSS technologies. Through the @Controller annotation of the spring-boot box, the URL and html are correlated at the controller level. Intelligent machine building technology is used to classify the query, and to a specific template, it will correspond to a query sentence. For the entire knowledge graph answering system, the processing flow is as follows:(1)
Entity identification and entity correction:First of all, we need to perform word segmentation and entity recognition to facilitate the semantic understanding later. The purpose of named entity recognition is to recognize the pointed objects of natural language questions. At present, named entity recognition mostly adopts algorithms such as conditional random field, hidden Markov, and long and short memory network LSTM.(2)
Vectorization of sentences:A template-based question and answer is used. In the naive Bayes method, the bag of words model is used to express the question, each sentence is converted into a vector, and then the naive Bayes algorithm is used to classify the question.(3)
Problem understanding based on naive Bayes:Naive Bayes algorithm can be divided into three stages: preparation stage, classifier training stage, and application stage. For this system, the three stages are as follows: (1) the preparation stage. Design the question template manually and use the template as training data. (2) Training phase. Read the template file and use the bag of words model to represent each sentence of the template. (3) Application phase. First, perform word segmentation and named entity recognition.(4)
Knowledge base queryThe form that this system can understand is named “entity + template label”; the named entity corresponds to the entity of knowledge base, and template label corresponds to the attribute or relationship of query.


## 3.3. Knowledge Graph Data

This article uses freebase as the basic knowledge graph. There are 18 relationship categories and 42 named entity categories in this dataset, including 362,000 sentences in the training set and 256,000 triples and 143,000 sentences in the test set and 5372 triples. The label in most sentences is NA.


## 4. Data Analysis of Knowledge Graph Question Answering System

### 4.1. Question Classification Experiment Results

In order to evaluate the stability and applicability of the method in this paper, the experiment was conducted by randomly dividing both datasets into 10 subsets with different sizes. These sets contain a subset of 500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, and 5000 questions, respectively. After that, the question classification model (Cap-net) proposed in this paper is run on each subdataset. In order to prevent the overfitting problem and obtain the best error estimate, this paper chooses ten-fold cross-validation to train these models. Use of the macroprecision to measure the experimental results is shown in Figure3.Figure 3
The effect of different dataset sizes on the accuracy of question classification macros.From the results, when the query total is increased from 500 to 3000, the classification macroaccuracy of both query datasets is significantly improved, indicating that in the case of fewer datasets, increasing the dataset size will have a significant effect on the classification results. As the dataset continues to grow, although the accuracy of the classification macros decreases slightly in some cases, the overall trend increases. This shows that the question classification model in this paper can better fit the data and complete the question classification task under a larger dataset. In view of the characteristics of questions, this paper combines two-way LSTM and attention mechanism and then adds a capsule network to complete the question classification task. In order to prove the effectiveness of the model, the model is ablated (that is, a certain part of the model is removed) to verify the validity of the combination of the question classification model in this paper. First, the attention mechanism is removed, and the output of the bidirectional LSTM layer is directly used as the input of the capsule layer as a comparative ablation experiment. Then, the capsule layer is removed, and the result of the combination of the two-way LSTM and the attention mechanism is directly connected to the output layer as a comparative ablation experiment. The experimental results are shown in Table1, and the specific images are shown in Figure 4.Table 1
Ablation experiment of question classification model.
Experimental modelRemove the capsule layerRemove the attention mechanismCap-netCCKS20180.6450.6890.732TREC0.6670.7120.752Figure 4
Ablation experiment of question classification model.It can be seen that when the model removes the capsule layer, the classification effect is greatly reduced, which shows that adding a capsule layer to the model in this paper can effectively improve the question classification results. When the model removes the attention mechanism, although it can produce better classification results than removing the capsule layer, it still fails to achieve the results of the complete model in this article. This proves that adding attention mechanism can better complete the classification task. By comparing with two ablation experiments, it is further proved that the question classification model in this paper combines bidirectional LSTM with attention mechanism and then adding a capsule network is a reasonable and feasible improvement.


### 4.2. Query Performance Comparison of Knowledge Graph Question Answering System

In this paper, the query performance of the query engine MIQE using memory iteration technology and the query engine IIQE and Franke query model using inverted index technology are compared. This section is based on the standard LUBM dataset under the 100,000 RDF dataset and the million RDF dataset to perform single entity query and connection query performance experiments, in which a single entity query uses commonly used one query condition and two query condition queries. The connection query uses commonly known queries between two entities and three entities. Since the data loading module and SPARQL statement parsing module of the knowledge graph storage access system have been implemented above, the query performance experiments for the two models directly use SPARQL statements for querying. In order to ensure the accuracy of the experimental results, each experiment in this paper conducted ten queries corresponding to the same query conditions but different query contents and the average query time were taken at the end. The experimental results are shown in Tables2 and 3, and the specific images are shown in Figures 5 and 6.Table 2
Query time results for one hundred thousand RDF dataset.
Single entity queryConnect queryA query conditionTwo query conditionsTwo entitiesThree entitiesFranke775575794332MIQE830759322368HIQE520420494876Table 3
Query time results for one million RDF dataset.
Single entity queryConnect queryA query conditionTwo query conditionsTwo entitiesThree entitiesFranke786324836643MIQE922607857712HIQE565567431552Figure 5
Query time results for one hundred thousand RDF dataset.Figure 6
Query time results for one million RDF dataset.It can be seen from the abovementioned two tables that under each RDF dataset, the query time of the MIQE model of each query mode is significantly longer than the query time of the Franke query model and the IIQE model. The query time of MIQE model under single entity query and connection query are both in minutes. The research found that the reason for the longer query time of the MIQE model is the Spark initialization time. Since MIQE in this article needs to load all the data in HBase in advance, and the time to load the data is calculated in the actual query time, the query time of MIQE is longer and as the amount of data increases, as MIQE loads more data, the query time will also increase. Excluding the time to load the data, the experiment measured that under the 100,000 RDF dataset, the single entity query of the MIQE model does not exceed 3 seconds, and the connection query does not exceed 5 seconds. Under the one million RDF dataset, the single entity query of the MIQE model will be not more than 8 seconds, not more than 10 seconds for connection query; the performance is weaker than the Franke query model and IIQE model. The query performance of the IIQE model is generally better than the Franke query model and the MIQE model. The query time of the IIQE model under a query condition and a single entity query of two query conditions does not exceed 5 seconds. The query time of the query does not exceed 8 seconds. The query performance of the Franke model is better than the query performance of the MIQE model, but compared with the IIQE model, although the query performance of a single entity query under two query conditions and the connected entity query between two entities are close, but based on querying a single entity under one query condition, the query performance of IIQE is significantly better than the Franke query model; when three entities are queried at the same time, the query performance of the connection query between the IIQE models is about twice that of the Franke query model. In a single entity query, the query time based on the Franke query model and the IIQE model under two query conditions is faster than the query time based on one query condition. The reason is that multiple query conditions will limit the number of query results. The query conditions will increase the query time due to the large number of results that will cause multiple data transmissions over the network. In summary, the knowledge graph storage access system based on how distributed aggregates are stored and the distributed parallel query mechanism has good horizontal scalability, and the IIQE model is more suitable for quickly searching large-scale knowledge graphs than the other two question models. Compared to IIQE, MIQE is simpler to implement and supports more integrated features. If query time is not high and you want to support more query features, the MIQE model is a good choice.


## 4.1. Question Classification Experiment Results

In order to evaluate the stability and applicability of the method in this paper, the experiment was conducted by randomly dividing both datasets into 10 subsets with different sizes. These sets contain a subset of 500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, and 5000 questions, respectively. After that, the question classification model (Cap-net) proposed in this paper is run on each subdataset. In order to prevent the overfitting problem and obtain the best error estimate, this paper chooses ten-fold cross-validation to train these models. Use of the macroprecision to measure the experimental results is shown in Figure3.Figure 3
The effect of different dataset sizes on the accuracy of question classification macros.From the results, when the query total is increased from 500 to 3000, the classification macroaccuracy of both query datasets is significantly improved, indicating that in the case of fewer datasets, increasing the dataset size will have a significant effect on the classification results. As the dataset continues to grow, although the accuracy of the classification macros decreases slightly in some cases, the overall trend increases. This shows that the question classification model in this paper can better fit the data and complete the question classification task under a larger dataset. In view of the characteristics of questions, this paper combines two-way LSTM and attention mechanism and then adds a capsule network to complete the question classification task. In order to prove the effectiveness of the model, the model is ablated (that is, a certain part of the model is removed) to verify the validity of the combination of the question classification model in this paper. First, the attention mechanism is removed, and the output of the bidirectional LSTM layer is directly used as the input of the capsule layer as a comparative ablation experiment. Then, the capsule layer is removed, and the result of the combination of the two-way LSTM and the attention mechanism is directly connected to the output layer as a comparative ablation experiment. The experimental results are shown in Table1, and the specific images are shown in Figure 4.Table 1
Ablation experiment of question classification model.
Experimental modelRemove the capsule layerRemove the attention mechanismCap-netCCKS20180.6450.6890.732TREC0.6670.7120.752Figure 4
Ablation experiment of question classification model.It can be seen that when the model removes the capsule layer, the classification effect is greatly reduced, which shows that adding a capsule layer to the model in this paper can effectively improve the question classification results. When the model removes the attention mechanism, although it can produce better classification results than removing the capsule layer, it still fails to achieve the results of the complete model in this article. This proves that adding attention mechanism can better complete the classification task. By comparing with two ablation experiments, it is further proved that the question classification model in this paper combines bidirectional LSTM with attention mechanism and then adding a capsule network is a reasonable and feasible improvement.


## 4.2. Query Performance Comparison of Knowledge Graph Question Answering System

In this paper, the query performance of the query engine MIQE using memory iteration technology and the query engine IIQE and Franke query model using inverted index technology are compared. This section is based on the standard LUBM dataset under the 100,000 RDF dataset and the million RDF dataset to perform single entity query and connection query performance experiments, in which a single entity query uses commonly used one query condition and two query condition queries. The connection query uses commonly known queries between two entities and three entities. Since the data loading module and SPARQL statement parsing module of the knowledge graph storage access system have been implemented above, the query performance experiments for the two models directly use SPARQL statements for querying. In order to ensure the accuracy of the experimental results, each experiment in this paper conducted ten queries corresponding to the same query conditions but different query contents and the average query time were taken at the end. The experimental results are shown in Tables2 and 3, and the specific images are shown in Figures 5 and 6.Table 2
Query time results for one hundred thousand RDF dataset.
Single entity queryConnect queryA query conditionTwo query conditionsTwo entitiesThree entitiesFranke775575794332MIQE830759322368HIQE520420494876Table 3
Query time results for one million RDF dataset.
Single entity queryConnect queryA query conditionTwo query conditionsTwo entitiesThree entitiesFranke786324836643MIQE922607857712HIQE565567431552Figure 5
Query time results for one hundred thousand RDF dataset.Figure 6
Query time results for one million RDF dataset.It can be seen from the abovementioned two tables that under each RDF dataset, the query time of the MIQE model of each query mode is significantly longer than the query time of the Franke query model and the IIQE model. The query time of MIQE model under single entity query and connection query are both in minutes. The research found that the reason for the longer query time of the MIQE model is the Spark initialization time. Since MIQE in this article needs to load all the data in HBase in advance, and the time to load the data is calculated in the actual query time, the query time of MIQE is longer and as the amount of data increases, as MIQE loads more data, the query time will also increase. Excluding the time to load the data, the experiment measured that under the 100,000 RDF dataset, the single entity query of the MIQE model does not exceed 3 seconds, and the connection query does not exceed 5 seconds. Under the one million RDF dataset, the single entity query of the MIQE model will be not more than 8 seconds, not more than 10 seconds for connection query; the performance is weaker than the Franke query model and IIQE model. The query performance of the IIQE model is generally better than the Franke query model and the MIQE model. The query time of the IIQE model under a query condition and a single entity query of two query conditions does not exceed 5 seconds. The query time of the query does not exceed 8 seconds. The query performance of the Franke model is better than the query performance of the MIQE model, but compared with the IIQE model, although the query performance of a single entity query under two query conditions and the connected entity query between two entities are close, but based on querying a single entity under one query condition, the query performance of IIQE is significantly better than the Franke query model; when three entities are queried at the same time, the query performance of the connection query between the IIQE models is about twice that of the Franke query model. In a single entity query, the query time based on the Franke query model and the IIQE model under two query conditions is faster than the query time based on one query condition. The reason is that multiple query conditions will limit the number of query results. The query conditions will increase the query time due to the large number of results that will cause multiple data transmissions over the network. In summary, the knowledge graph storage access system based on how distributed aggregates are stored and the distributed parallel query mechanism has good horizontal scalability, and the IIQE model is more suitable for quickly searching large-scale knowledge graphs than the other two question models. Compared to IIQE, MIQE is simpler to implement and supports more integrated features. If query time is not high and you want to support more query features, the MIQE model is a good choice.


## 5. Conclusions

In this study, we explain how to transform the human intention recognition problem into a classification problem and then explain the mathematical model based on the knowledge base in two steps: the first is to use the established knowledge correlation graph to adjust the weight of the entity words given the problem and use the contribution value and contribution multiplier to calculate the weight of the entity words. Then, it introduces the Bayesian model and how the Bayesian model is applied to the classification problem. Finally, the Bayesian model and the weight of entity words are combined to establish a weighted Bayesian algorithm. This algorithm believes that the weight of words will also affect the final accuracy rate.In this study, through the analysis of the biological entities, the experimental entities, combining entities and other special entities contained in the biological entities, were found. By studying the principles of the R-WMD model and optimizing it for the semantic matching of biology entities, the optimization content includes resetting the weight coefficient of the normalized word bag model through richer semantic information, and introducing length-related parametersK, allows longer entities to return first. Finally, the experiment verifies that the algorithm has the best effect on entity semantic matching compared with other text similarity algorithms.This work designs and implements a design of knowledge question answering systems for intelligent engineering based on deep learning. The system is divided into three main sections: data loading section, SPARQL sentence analysis section, and data query section. The knowledge graph is first uploaded to HBase via the data upload module. During the query, the SPARQL query statement obtains the query structure through the parser module and then creates a query tree in the data module to execute a query based on the HBase query method according to the query structure obtained. The experiments verify that the query rendering of the IIQE query engine is superior to the MIQE query engine. The knowledge graph storage access system implemented in this document can not only store knowledge graphs efficiently with a load-balanced balance but also ensure the query performance of knowledge graphs.



---

*Source: 6627114-2021-02-16.xml*