# Military Chain: Construction of Domain Knowledge Graph of Kill Chain Based on Natural Language Model

Authors: Yanfeng Wang; Tao Wang; Junhui Wang; Xin Zhou; Ming Gao; Runmin Liu
Journal: Mobile Information Systems
 (2022)

---

## Abstract

With the advent of the Big Data era, the specialized data in the kill chain domain has increased dramatically, and the engine-based method of retrieving information can hardly meet the users' need for more accurate answers. The kill chain domain includes four components: control equipment, sensor equipment, strike equipment (weapon and platform), and evaluator equipment, as well as related data which contain a large amount of valuable information such as the parameter information contained in each component. If these fragmented and confusing data are integrated and effective query methods are established, they can help professionals complete the military kill chain knowledge system. The knowledge system constructed in this paper is based on the Neo4j graph database and the US Command simulation system to establish a target-oriented knowledge map of kill chain, aiming to provide data support for the Q&A system. Secondly, in order to facilitate the query, this paper establishes entity and relationship/attribute mining based on the continuous bag-of-words (CBOW) encoding model, bidirectional long short-term memory–conditional random field (BiLSTM-CRF) named entity model, and bidirectional gated recurrent neural network (BiGRU) intent recognition model for Chinese kill chain question and answer; returns the corresponding entity or attribute values in combination with the knowledge graph triad form; and finally constructs the answer return. The constructed knowledge map of the kill chain contains 2767 items (including sea, land, and air), and the number of parameters involved is 30124. The number of model parameters of the deep learning network is 27.9 M for the Q&A system built this time, and the accuracy rate is 85.5% after 200 simulated queries.


---

## Body

## 1. Introduction

“Soldiers can be used for a thousand days, but a country cannot be without defense for a day.” For any country, its own military power is its foundation. Only when it has strong militarily can it ensure that it will not be invaded by other countries. With the development of human civilization, all major countries aspire to peace. The probability of large-scale wars becomes extremely low, but local wars, military conflicts, and other low-intensity military operations are becoming more and more frequent [1]. This poses a great test to the military power of each country, which needs to meet the requirements of durability, accuracy, and other weapons performance, as well as economic factors such as the cost of weapons and manufacturing sources. However, with the rise of the military power of each country, the power and technology of weapons and equipment are rising rapidly, but the construction cost and the research and development cost are also rising. Hence, the military spending of countries have increased, coupled with international influence, personnel training, and other factors, so that many countries are constrained in taking military action and are unable to exert their most powerful force [2]. This requires each country to evaluate both the enemy and “us” and to prepare more accurately and efficiently for the military strength of the enemy country, so as to achieve the result of victory at a minimum cost. To achieve this goal, Gro`Z1ssman E. M. [3] proposed at the Air Force Symposium the kill chain concept—an orderly chain of interdependent links in the process of striking a target, consisting of four components: control equipment, sensor equipment, strike equipment (weapon and platform), and evaluator equipment, with the operations divided into six components in six phases: find, fix, track, target, engage, and assess, or F2T2EA.In general, the earlier the kill chain link stops the attack, the better the protection will be. The main way of kill chain assessment in various countries at this stage is still through manual analysis, which has initially obtained more significant results. For example, Brickner, William K [1] conducted a study on kill chain analysis for time-sensitive target strikes. The baseline model of the time-sensitive target kill chain established by the US Naval Air Systems Command (NAVAIR) was used as the basis for the study, and a modeling simulation approach was used in conjunction with a typical example of time-sensitive target detection and strike of a theater ballistic missile, in which the closure of the kill chain in the Extend system simulation software was analyzed in detail. Based on the evaluation results, a more effective detection, operational concept was proposed compared to the NAVAIR force structure at that time. Brad Bloye [4] investigated the optimization of the air-to-ground time-sensitive strike kill chain, compensating for the deficiencies of Brickner by proposing to translate the advantage of network-centric information into the efficiency advantage of the kill chain closure and based on the development of “KCAT” kill chain evaluation tool, the quantitative analysis, evaluation, and optimization of the kill chain had been completed, providing support for the combatants to make decisions on existing and newly developed equipment systems and operational concepts based on the effectiveness of the kill chain. However, the traditional model of assessing only one indicator, the time to closure of the kill chain, can no longer meet the requirements for assessing the effectiveness of the kill chain in a complex environment. O. Thomas Holland SEW [5] addressed this problem by proposing a methodology that included timeliness, appropriateness, precision, discrimination, orchestration, and survivability (TAPDOS) of the kill chain. Although the above methods are more complete for the exploration of the kill chain and have achieved better evaluation results, along with the increasing number of military equipment, the professional data in the field of kill chain is increasing day by day, and it has been difficult to analyze and filter the huge amount of data only by hand.With the development and progress of science and technology, the application of artificial intelligence is becoming more and more widespread, playing an increasingly wide range of roles in real life [6]. In particular, deep learning methods, which have gained vigorous development in recent years, have enabled the progress of AI from simple theory to practical applications in various fields. At present, artificial intelligence technology has accelerated its penetration to the military field, and information and intelligent warfare has gradually become a high-profile topic [7]. In the current era when information-based warfare is developing in depth and intelligent warfare is beginning to emerge, the world's major militaries have made great efforts to promote military informatization and intelligence, integrating a large amount of fragmented and confusing military equipment data and relationships to establish knowledge maps and using deep learning methods for assistance [8].In order to integrate the scattered and confusing kill chain data and establish the knowledge map of the kill chain, this paper firstly writes a crawler based on Scrapy framework to crawl the kill chains existing in Wikipedia, Military News Network, and other web pages, which are usually written by professionals and reviewed by the editorial board and have certain reliability. Secondly, since military data must be highly accurate, this paper verifies whether the obtained kill chains are valid based on the American Command simulation system, and searches for some relevant parameters of the components of the kill chains in the simulation system for data expansion. Finally, all the acquired data are imported into the Neo4j graph database [9] to establish a target-oriented knowledge map of the kill chain.In order to facilitate the querying of the established kill chain knowledge graph, this paper addresses the triadic form consisting of knowledge graph entities, relations, and attributes; decomposes entities and relations/attributes for query interrogatives based on the BiLSTM-CRF [10] named entity model and the BiGRU [11] user intent recognition model; and constructs the corresponding answer returns in combination with the knowledge graph, as shown in Figure 1. Meanwhile, in order to satisfy users more conveniently, this project developed a front end based on Uni-app with UI design, using green as the theme color, and then used Flask back-end framework for front- and back-end connection.Figure 1
The overall flow chart of the question-and-answer system.


## 2. Materials and Methods

### 2.1. Data Acquisition

Data collection is an important component of building domain-specific knowledge graphs to provide data support for the Q&A system. The data of the kill chain collected this time are from the Internet and the US Command simulation system, which is a self-made dataset.


#### 2.1.1. Implementation of Kill Chain Packet Capture Based on Scrapy Framework

A large number of web pages about military kill chains (consisting of control equipment, sensor equipment, strike equipment, and evaluator equipment) exist on the Internet, but most of them exist in the form of unstructured data. In order to obtain these data more efficiently and enrich the kill chain database, we write a crawler based on the Scrapy framework to crawl relevant data on relevant web pages such as Wikipedia, Baidu Encyclopedia, and Military News Network.Scrapy is an application framework developed in Python language for crawling web data, which is used to crawl and extract structured data from pages, and its operation principle is shown in Figure2. This tool has the advantages of fast crawling speed, automatic adjustment of crawling mechanism, and high throughput in crawling data and is the most well-known and widely applicable framework among all crawler frameworks. This Scrapy-based design of the kill chain data crawler is divided into three main modules: rule presetting, web crawling, and data storage.Figure 2
Scrapy framework operation principle.Rule presetting module. The rule presetting module internally contains parameter settings such as user agent, log level, and robot protocol compliance. The field properties of the targets captured this time (i.e., control equipment, sensor equipment, strike equipment, and evaluator equipment) are also set.Web crawling module. The web crawler module first requests the URL of the initial web page, parses the request data using LXML to extract the URL of the web page containing the kill link data, then requests and parses the URL, and uses XPath to locate and obtain information about the kill link.Data storage module. The data storage module stores the crawled data persistently. In order to view the crawled data more intuitively and subsequently use the data to build a knowledge graph, the project exports the crawled kill chain data in CSV format, so that the data can be viewed intuitively through Excel software.


#### 2.1.2. Parameter Acquisition of Kill Chain Components Based on Command Simulation System

For the captured kill chains, we search the names in each kill chain for parameters in the US Command simulation system and import them into an Excel sheet for summary saving, and some parameters of the kill chain components are shown in Table1.Table 1
Kill chain component parameters.
SensorControlStrike equipmentEvaluatorPlaneSurface shipsWeaponPlatformPlaneSubmarinesSurface shipsMinimum heightLengthLengthLengthLengthLengthLengthFrequency limitMaximum heightExtensibilityWidthExtensibilityExtensibilityWidthWidthLower frequency limitScan intervalHeightAirflowDiameterHeightAirflowAirflowScan intervalResolution heightLoad capacityHeightWeightLoad capacityHeightHeightMinimum heightResolution rangeMembersMembersCEPMembersMembersMembersMaximum heightResolution angleSensitivityDamage pointsClimbing rateSensitivityMaximum depthDamage pointsContact area


### 2.2. Neo4j-Based Knowledge Graph Construction for Military Kill Chains

The current ways of storing the crawled data are mainly based on RDF storage (RDF4J), traditional relational database (MySQL) based storage, and graph database-based storage (Neo4j). Due to the problems of large storage space, long algorithm time, and high complexity of RDF storage, the traditional relational database based on RDF storage cannot well support the real-time query of relationships. Combining the situation of the acquired kill chain data and the actual situation of the final deployment of the project in mobile, for more excellent read/write performance and scalability, we adopt the way of graph database storage and select the mainstream Neo4j in graph database for the construction of the knowledge graph. The part of the knowledge graph is shown in Figure3.Figure 3
Knowledge graph section display.The military kill chain knowledge graph is a structured triad consisting of nodes and directed edges in the form of entity-relationship-entity and entity-attribute-attribute values. Nodes are used to represent entities, which in our project represent the components of the kill chain (control equipment, sensor equipment, strike equipment, and evaluator equipment), and attributes of nodes are entity parameters. The entity classes in our knowledge graph are shown in Table2, and the node attributes are shown in Table 3. Directed edges are used to represent the relationships between entities, which represent the correspondence between the components of the kill chain in our knowledge graph, such as sensor equipment and control equipment. The entity relationships are shown in Table 4.Table 2
Knowledge graph entity type display (number of entries of kill chains and their constituents).
Entity typeNumber of entitiesExamplesFacility3293Radar (Tin Shield A [5N59])GroundUnit127AAVP-7A1Magazine1477RB 90 Nightrider [RBS 70 Mk2]Mount3186SS-25 Sickle [MAZ-7310] TELAircraft5124S–3B VikingComm313SA-10 Missile DatalinkSenor6189DRBN 30Ship3325R 05 InvincibleSubmarine639SSBN 726 OhioWeapon3684AGM-158B JASSM-ERKill_Chain2767Link 1Total3012430000 entities represent a variety of equipmentThe bold values represent the total number of entity mentioned above (3293 + 127+... + 3684 + 2767 = 30124).Table 3
Aircraft property type section display.
Property definitionExamplesCategoryFixed-wing aircraftTypeFighterOperator country2101 (corresponding country code)Operator service2002 (corresponding service code)Length15.7Span13.1Height4.54Weight empty14700Weight max27300Weight payload6810Crew1Climb rate194.7OODA detection cycle15OODA targeting cycle10OODA evasive cycle2Physical size codeMedium-sized aircraft (length: 12.1–18 m)Table 4
Knowledge graph entity relationship part type display.
Entity relationship typeNumber of relationshipsExamplesSensor equipment3128<Link 1, reconnaissance equipment, S-3A Viking>Weapon2854<Link 1, Combat Equipment Ammunition, ZUNI 127 mm HVAR Rocket>Platform12910<Link 1, strike equipment delivery platform, A-4N Skyhawk [Ahit]>Evaluator equipment3106<Link 1, effect evaluation equipment, S-3A Viking>Control equipment7859<Link 1, communication command equipment, CV 63 Kitty Hawk>Total29857About 30000 relationships to build a complete kill chainThe bold values represent the total number of entity relationship mentioned above (3128 + 2854 + ... + 3106 + 7859 = 29857).


### 2.3. Kill Chain Data Labeling

A word-by-word named entity annotation of the interrogative sentence is performed for entity recognition and intent recognition by the deep learning model, and due to the excessive number of annotated individuals of the required named entities this time, only some of the annotations are shown, as indicated in Table5.Table 5
Named entity labeling.
EntityMeaningOOthersB-weaBeginning of the weapon entityI-weaMiddle of the weapon entityE-weaEnd of weapon entityB-plaBeginning of the launch platform entityI-plaMiddle of the launch platform entityE-plaEnd of the launch platform entity


### 2.4. A Kill Chain Q&A System Based on BiLSTM-CRF Entity Recognition Model and BiGRU Intent Recognition Network

#### 2.4.1. Word2Vec Encoding Model

The user input words that express the intended meaning need to be word-vectorized in order to transform the abstract human symbols into a mathematical language for computer use. The current mainstream approach is to embed abstract words into a numerical space, i.e., word embedding. In this project, we use the mainstream Word2Vec word embedding method to transform words into vector form.Word2Vec encodes words into a vector based on the anticipation of information co-occurrence, which mainly includes two neural network models, Skip-gram model and continuous bag-of-words model (CBOW) [12], where Skip-gram model [13] predicts the words around the central word by the central word, while continuous bag-of-words model predicts the central word by the surrounding words. Due to the limited types of problems in the kill chain domain and the small number of datasets required for training, the CBOW model is chosen since it has a theoretically better performance with better results, and it is shown in Figure 4.Figure 4
CBOW model.CBOW model is generally a three-layer neural network structure, which is divided into input layer, hidden layer, and Softmax layer (output layer). The input of the model is the one-hot word vector of the adjacent words of the current position words (C in total; each dimension is V), and the data dimension is C∗V. The hidden layer multiplies the word vector of the adjacent vocabulary by the weight matrix Wdimension:V∗N to obtain C∗1∗N vectors and then adds and averages them (as shown in (1)) to obtain the hidden layer vector h with dimension 1∗N. The Softmax layer processes the output vector of the hidden layer through Softmax and selects the word with the highest probability as the word of the predicted current position. The predicted word is compared with the real word of the current position to obtain the error value, and the error value is continuously reduced through backpropagation. The training parameters of the CBOW model are shown in Table 6.(1)h=1Cx1+x2+⋯+xc∗W.Table 6
CBOW model training parameters.
Training parameterParameter valueLearning rate0.0001Number of contextual words10Word vector dimension250Batch size200Minimum word frequency5OptimizerAdam


#### 2.4.2. Named Entity Recognition Module of Military Kill Chain Based on BiLSTM-CRF Model

Entity recognition is the main foundation of intelligent question-and-answer system, and the extraction effect of entity recognition more directly influences the quality and efficiency of subsequent natural processing. The current research methods for named entity extraction include rule-based, dictionary-based, machine learning-based, and statistical-based methods. However, due to the fact that there are mostly proper nouns in the field of kill chain and the problem of complex data relationships, entity recognition using the above methods is prone to errors and poor portability and requires a large-scale corpus to learn annotation, which requires manual participation in feature extraction and makes it difficult to avoid generating human errors.In recent years, with the continuous upgrading of computer hardware, deep learning has been widely developed and achieved good results in entity recognition tasks. The BERT-BiLSTM-CRF structural model has been widely used in various named entity recognition methods because of its high recognition rate, where BERT [14] is able to obtain contextualized word vectors to improve the performance of subsequent entity recognition and has a strong robustness. However, since BERT is based on transformer structure and training model, its number of parameters is huge (e.g., 110M for BERT-base and 330M for BERT-large), which is a big test for the deployed mobile computing power and is not conducive to its promotion. And, in this case, the number of entities named for the kill chain neighborhood is limited, and the phenomenon of multiple meanings of the word is small.In summary, this entity recognition model uses CBOW in Word2Vec as the word vector transformation layer and uses a bidirectional long short-term memory network model (BiLSTM) as the word vector encoding layer, as well as a conditional random field model layer (CRF) as the final output layer, and the overall model is shown in Figure5.Figure 5
BiLSTM-CRF entity recognition model.Bidirectional long short-term memory (BiLSTM). The long short-term memory network (LSTM) [15] can better solve the gradient explosion and gradient disappearance problems that occur because the sequence is too long, and can relate the dependencies over a long period of time. However, LSTM can only consider the information of the previous text, but in many cases, the output of the current moment is not only related to the previous text but also to the later text. Therefore, the BiLSTM, which is based on the improvement of LSTM, is proposed. The BiLSTM [16] model is a forward LSTM (dealing with the previous information) followed by a reverse LSTM to deal with the later information, capturing the contextual dependencies; i.e., the BiLSTM is composed of two unidirectional LSTMs concatenated together, and its structural model is shown in Figure 6.Figure 6
BiLSTM structure.x1,x2,...,xN is the word vector transformed by the CBOW model; Wff=1,2,…,N represents the forward LSTM hidden layer output vector, which is jointly determined by the word vector of the current input xf and the forward LSTM output of the previous moment Wf−1. Similarly, Wbb=1,2,…,N represents the output vector of the inverse LSTM hidden layer, which is jointly determined by the word vector of the current input xb and the forward LSTM output of the previous moment Wb−1. o1,o2,…,oN is the output of the BiLSTM model, which is jointly determined by Wf and Wb, and the mathematical expression is as follows:(2)oN=wfWf+wbWb+bt,where wf represents the weight matrix of the forward LSTM output, wb represents the weight matrix of the reverse LSTM output, and bt is the bias.Conditional random fields (CRF). BiLSTM can predict the probability of each word corresponding to the predicted label, and then the label with the highest probability can be obtained by Softmax, but this will ignore the correlation between the labels, resulting in the label with the highest probability not conforming to the semantic logic, such as producing sentences with obvious errors like noun + verb + verb. Therefore, it is necessary to add some conditions that can constrain the sentences after the output layer of BiLSTM to ensure the validity of the final prediction results. The conditional random field (CRF) [17] model is a serialized labeling algorithm that can automatically learn some useful constraints to reduce erroneous prediction sequences when training data, and the structure of CRF is shown in Figure 7.Figure 7
CRF structure.The core principle is as follows:(3)ScoreX,y=∑iEi,yi+∑iTyi,yi+1,where E is the given emission matrix, T is the CRF parameter matrix that needs to be inverted and optimized by the loss function in the calculation, and ScoreX,y is the matching score for the given input and output. X is the input word sequence, and y is the predicted label sequence. Then, the following objective is maximized:(4)Pygold|X=expsX,ygold∑y∈YsexpsX,y.


#### 2.4.3. Intent Recognition Module Based on BiGRU Model

Classifying questions from user input natural language and recognizing the user's intention constitute one of the indispensable tasks in Chinese natural language processing. The natural language question sentences input by users of military kill chain Q&A system are usually short texts, so the problem to be solved in this project is the classification problem of short texts.The bidirectional gated recurrent neural network (BiGRU) is selected for this intention recognition module. The gated recurrent neural network (GRU) [18] is an improvement of LSTM (with minimum unit consisting of input gate, forget gate, and output gate), and the reset gate of GRU merges the incoming gate and forget gate in LSTM, which makes the model structure simpler, reduces the number of parameters, and saves the training time. BiGRU, on the other hand, solves the problem that the input behind GRU is more important than the front when it advances from left to right, takes into account the future context information, and captures the complete context. Its structural model is shown in Figure 8.Figure 8
BiGRU structure.GRU detailed workflow is as follows:Step 1.
The reset gate controls which of the old cell states and inputs are discarded and which are retained, and its mathematical expression is as follows:(5)rt=σWrxt+Urht−1,where σ represents the Sigmoid activation function, xt represents the input information, and ht−1 represents the output of the hidden layer at the previous moment.Step 2.
Decide which new information is saved to the state of the metacell:(1)
The operation of forgetting the previous information and adding new information is done by the update gate and the mathematical expression is as follows:(6)zt=σWzxt+Uzht−1.(2)
Create a new candidate valueh˜t=tanhWhxt+rtUhht−1 from the tanh layer.Step 3.
The combination of Step 1 and Step 2 is used to update the old cell stateht−1 to the new cell state ht; the mathematical expression is as follows:(7)ht=1−ztht−1+zth˜t.


#### 2.4.4. Answer Generation Module

Currently, there is no publicly available dataset for the types of interrogative sentences about the kill chain, and it is difficult to collect valid interrogative sentences about the military kill chain on military websites. Therefore, an interrogative template about military kill chains was designed for this project, and some of the interrogatives are displayed in Table7.Table 7
Question template.
Question templateExamplesWhat are the parameters of a particular weapon?What are the parameters of the E-3A Sentry?What platforms can a weapon be mounted on?What platforms can the AGM-158B JASSM-ER be equipped with?Which kill chains can a weapon apply to?What kill chains can the F-14B Tomcat be applied to?Composition of each link of a linkWhat is the equipment composition of each link in link 1?Since the knowledge graph triad has two representations: <entity-relationship-entity> and <entity-attribute-attribute value>, after the BiLSTM-CRF named entity model, we can get the entities in the triad, and after the BiGRU intent recognition model, we get the relationships/attributes in the triad. The obtained two parameters are input into the Neo4j graph database to get the required entity/attribute values and construct the corresponding answers to be returned.


## 2.1. Data Acquisition

Data collection is an important component of building domain-specific knowledge graphs to provide data support for the Q&A system. The data of the kill chain collected this time are from the Internet and the US Command simulation system, which is a self-made dataset.


### 2.1.1. Implementation of Kill Chain Packet Capture Based on Scrapy Framework

A large number of web pages about military kill chains (consisting of control equipment, sensor equipment, strike equipment, and evaluator equipment) exist on the Internet, but most of them exist in the form of unstructured data. In order to obtain these data more efficiently and enrich the kill chain database, we write a crawler based on the Scrapy framework to crawl relevant data on relevant web pages such as Wikipedia, Baidu Encyclopedia, and Military News Network.Scrapy is an application framework developed in Python language for crawling web data, which is used to crawl and extract structured data from pages, and its operation principle is shown in Figure2. This tool has the advantages of fast crawling speed, automatic adjustment of crawling mechanism, and high throughput in crawling data and is the most well-known and widely applicable framework among all crawler frameworks. This Scrapy-based design of the kill chain data crawler is divided into three main modules: rule presetting, web crawling, and data storage.Figure 2
Scrapy framework operation principle.Rule presetting module. The rule presetting module internally contains parameter settings such as user agent, log level, and robot protocol compliance. The field properties of the targets captured this time (i.e., control equipment, sensor equipment, strike equipment, and evaluator equipment) are also set.Web crawling module. The web crawler module first requests the URL of the initial web page, parses the request data using LXML to extract the URL of the web page containing the kill link data, then requests and parses the URL, and uses XPath to locate and obtain information about the kill link.Data storage module. The data storage module stores the crawled data persistently. In order to view the crawled data more intuitively and subsequently use the data to build a knowledge graph, the project exports the crawled kill chain data in CSV format, so that the data can be viewed intuitively through Excel software.


### 2.1.2. Parameter Acquisition of Kill Chain Components Based on Command Simulation System

For the captured kill chains, we search the names in each kill chain for parameters in the US Command simulation system and import them into an Excel sheet for summary saving, and some parameters of the kill chain components are shown in Table1.Table 1
Kill chain component parameters.
SensorControlStrike equipmentEvaluatorPlaneSurface shipsWeaponPlatformPlaneSubmarinesSurface shipsMinimum heightLengthLengthLengthLengthLengthLengthFrequency limitMaximum heightExtensibilityWidthExtensibilityExtensibilityWidthWidthLower frequency limitScan intervalHeightAirflowDiameterHeightAirflowAirflowScan intervalResolution heightLoad capacityHeightWeightLoad capacityHeightHeightMinimum heightResolution rangeMembersMembersCEPMembersMembersMembersMaximum heightResolution angleSensitivityDamage pointsClimbing rateSensitivityMaximum depthDamage pointsContact area


## 2.1.1. Implementation of Kill Chain Packet Capture Based on Scrapy Framework

A large number of web pages about military kill chains (consisting of control equipment, sensor equipment, strike equipment, and evaluator equipment) exist on the Internet, but most of them exist in the form of unstructured data. In order to obtain these data more efficiently and enrich the kill chain database, we write a crawler based on the Scrapy framework to crawl relevant data on relevant web pages such as Wikipedia, Baidu Encyclopedia, and Military News Network.Scrapy is an application framework developed in Python language for crawling web data, which is used to crawl and extract structured data from pages, and its operation principle is shown in Figure2. This tool has the advantages of fast crawling speed, automatic adjustment of crawling mechanism, and high throughput in crawling data and is the most well-known and widely applicable framework among all crawler frameworks. This Scrapy-based design of the kill chain data crawler is divided into three main modules: rule presetting, web crawling, and data storage.Figure 2
Scrapy framework operation principle.Rule presetting module. The rule presetting module internally contains parameter settings such as user agent, log level, and robot protocol compliance. The field properties of the targets captured this time (i.e., control equipment, sensor equipment, strike equipment, and evaluator equipment) are also set.Web crawling module. The web crawler module first requests the URL of the initial web page, parses the request data using LXML to extract the URL of the web page containing the kill link data, then requests and parses the URL, and uses XPath to locate and obtain information about the kill link.Data storage module. The data storage module stores the crawled data persistently. In order to view the crawled data more intuitively and subsequently use the data to build a knowledge graph, the project exports the crawled kill chain data in CSV format, so that the data can be viewed intuitively through Excel software.


## 2.1.2. Parameter Acquisition of Kill Chain Components Based on Command Simulation System

For the captured kill chains, we search the names in each kill chain for parameters in the US Command simulation system and import them into an Excel sheet for summary saving, and some parameters of the kill chain components are shown in Table1.Table 1
Kill chain component parameters.
SensorControlStrike equipmentEvaluatorPlaneSurface shipsWeaponPlatformPlaneSubmarinesSurface shipsMinimum heightLengthLengthLengthLengthLengthLengthFrequency limitMaximum heightExtensibilityWidthExtensibilityExtensibilityWidthWidthLower frequency limitScan intervalHeightAirflowDiameterHeightAirflowAirflowScan intervalResolution heightLoad capacityHeightWeightLoad capacityHeightHeightMinimum heightResolution rangeMembersMembersCEPMembersMembersMembersMaximum heightResolution angleSensitivityDamage pointsClimbing rateSensitivityMaximum depthDamage pointsContact area


## 2.2. Neo4j-Based Knowledge Graph Construction for Military Kill Chains

The current ways of storing the crawled data are mainly based on RDF storage (RDF4J), traditional relational database (MySQL) based storage, and graph database-based storage (Neo4j). Due to the problems of large storage space, long algorithm time, and high complexity of RDF storage, the traditional relational database based on RDF storage cannot well support the real-time query of relationships. Combining the situation of the acquired kill chain data and the actual situation of the final deployment of the project in mobile, for more excellent read/write performance and scalability, we adopt the way of graph database storage and select the mainstream Neo4j in graph database for the construction of the knowledge graph. The part of the knowledge graph is shown in Figure3.Figure 3
Knowledge graph section display.The military kill chain knowledge graph is a structured triad consisting of nodes and directed edges in the form of entity-relationship-entity and entity-attribute-attribute values. Nodes are used to represent entities, which in our project represent the components of the kill chain (control equipment, sensor equipment, strike equipment, and evaluator equipment), and attributes of nodes are entity parameters. The entity classes in our knowledge graph are shown in Table2, and the node attributes are shown in Table 3. Directed edges are used to represent the relationships between entities, which represent the correspondence between the components of the kill chain in our knowledge graph, such as sensor equipment and control equipment. The entity relationships are shown in Table 4.Table 2
Knowledge graph entity type display (number of entries of kill chains and their constituents).
Entity typeNumber of entitiesExamplesFacility3293Radar (Tin Shield A [5N59])GroundUnit127AAVP-7A1Magazine1477RB 90 Nightrider [RBS 70 Mk2]Mount3186SS-25 Sickle [MAZ-7310] TELAircraft5124S–3B VikingComm313SA-10 Missile DatalinkSenor6189DRBN 30Ship3325R 05 InvincibleSubmarine639SSBN 726 OhioWeapon3684AGM-158B JASSM-ERKill_Chain2767Link 1Total3012430000 entities represent a variety of equipmentThe bold values represent the total number of entity mentioned above (3293 + 127+... + 3684 + 2767 = 30124).Table 3
Aircraft property type section display.
Property definitionExamplesCategoryFixed-wing aircraftTypeFighterOperator country2101 (corresponding country code)Operator service2002 (corresponding service code)Length15.7Span13.1Height4.54Weight empty14700Weight max27300Weight payload6810Crew1Climb rate194.7OODA detection cycle15OODA targeting cycle10OODA evasive cycle2Physical size codeMedium-sized aircraft (length: 12.1–18 m)Table 4
Knowledge graph entity relationship part type display.
Entity relationship typeNumber of relationshipsExamplesSensor equipment3128<Link 1, reconnaissance equipment, S-3A Viking>Weapon2854<Link 1, Combat Equipment Ammunition, ZUNI 127 mm HVAR Rocket>Platform12910<Link 1, strike equipment delivery platform, A-4N Skyhawk [Ahit]>Evaluator equipment3106<Link 1, effect evaluation equipment, S-3A Viking>Control equipment7859<Link 1, communication command equipment, CV 63 Kitty Hawk>Total29857About 30000 relationships to build a complete kill chainThe bold values represent the total number of entity relationship mentioned above (3128 + 2854 + ... + 3106 + 7859 = 29857).


## 2.3. Kill Chain Data Labeling

A word-by-word named entity annotation of the interrogative sentence is performed for entity recognition and intent recognition by the deep learning model, and due to the excessive number of annotated individuals of the required named entities this time, only some of the annotations are shown, as indicated in Table5.Table 5
Named entity labeling.
EntityMeaningOOthersB-weaBeginning of the weapon entityI-weaMiddle of the weapon entityE-weaEnd of weapon entityB-plaBeginning of the launch platform entityI-plaMiddle of the launch platform entityE-plaEnd of the launch platform entity


## 2.4. A Kill Chain Q&A System Based on BiLSTM-CRF Entity Recognition Model and BiGRU Intent Recognition Network

### 2.4.1. Word2Vec Encoding Model

The user input words that express the intended meaning need to be word-vectorized in order to transform the abstract human symbols into a mathematical language for computer use. The current mainstream approach is to embed abstract words into a numerical space, i.e., word embedding. In this project, we use the mainstream Word2Vec word embedding method to transform words into vector form.Word2Vec encodes words into a vector based on the anticipation of information co-occurrence, which mainly includes two neural network models, Skip-gram model and continuous bag-of-words model (CBOW) [12], where Skip-gram model [13] predicts the words around the central word by the central word, while continuous bag-of-words model predicts the central word by the surrounding words. Due to the limited types of problems in the kill chain domain and the small number of datasets required for training, the CBOW model is chosen since it has a theoretically better performance with better results, and it is shown in Figure 4.Figure 4
CBOW model.CBOW model is generally a three-layer neural network structure, which is divided into input layer, hidden layer, and Softmax layer (output layer). The input of the model is the one-hot word vector of the adjacent words of the current position words (C in total; each dimension is V), and the data dimension is C∗V. The hidden layer multiplies the word vector of the adjacent vocabulary by the weight matrix Wdimension:V∗N to obtain C∗1∗N vectors and then adds and averages them (as shown in (1)) to obtain the hidden layer vector h with dimension 1∗N. The Softmax layer processes the output vector of the hidden layer through Softmax and selects the word with the highest probability as the word of the predicted current position. The predicted word is compared with the real word of the current position to obtain the error value, and the error value is continuously reduced through backpropagation. The training parameters of the CBOW model are shown in Table 6.(1)h=1Cx1+x2+⋯+xc∗W.Table 6
CBOW model training parameters.
Training parameterParameter valueLearning rate0.0001Number of contextual words10Word vector dimension250Batch size200Minimum word frequency5OptimizerAdam


### 2.4.2. Named Entity Recognition Module of Military Kill Chain Based on BiLSTM-CRF Model

Entity recognition is the main foundation of intelligent question-and-answer system, and the extraction effect of entity recognition more directly influences the quality and efficiency of subsequent natural processing. The current research methods for named entity extraction include rule-based, dictionary-based, machine learning-based, and statistical-based methods. However, due to the fact that there are mostly proper nouns in the field of kill chain and the problem of complex data relationships, entity recognition using the above methods is prone to errors and poor portability and requires a large-scale corpus to learn annotation, which requires manual participation in feature extraction and makes it difficult to avoid generating human errors.In recent years, with the continuous upgrading of computer hardware, deep learning has been widely developed and achieved good results in entity recognition tasks. The BERT-BiLSTM-CRF structural model has been widely used in various named entity recognition methods because of its high recognition rate, where BERT [14] is able to obtain contextualized word vectors to improve the performance of subsequent entity recognition and has a strong robustness. However, since BERT is based on transformer structure and training model, its number of parameters is huge (e.g., 110M for BERT-base and 330M for BERT-large), which is a big test for the deployed mobile computing power and is not conducive to its promotion. And, in this case, the number of entities named for the kill chain neighborhood is limited, and the phenomenon of multiple meanings of the word is small.In summary, this entity recognition model uses CBOW in Word2Vec as the word vector transformation layer and uses a bidirectional long short-term memory network model (BiLSTM) as the word vector encoding layer, as well as a conditional random field model layer (CRF) as the final output layer, and the overall model is shown in Figure5.Figure 5
BiLSTM-CRF entity recognition model.Bidirectional long short-term memory (BiLSTM). The long short-term memory network (LSTM) [15] can better solve the gradient explosion and gradient disappearance problems that occur because the sequence is too long, and can relate the dependencies over a long period of time. However, LSTM can only consider the information of the previous text, but in many cases, the output of the current moment is not only related to the previous text but also to the later text. Therefore, the BiLSTM, which is based on the improvement of LSTM, is proposed. The BiLSTM [16] model is a forward LSTM (dealing with the previous information) followed by a reverse LSTM to deal with the later information, capturing the contextual dependencies; i.e., the BiLSTM is composed of two unidirectional LSTMs concatenated together, and its structural model is shown in Figure 6.Figure 6
BiLSTM structure.x1,x2,...,xN is the word vector transformed by the CBOW model; Wff=1,2,…,N represents the forward LSTM hidden layer output vector, which is jointly determined by the word vector of the current input xf and the forward LSTM output of the previous moment Wf−1. Similarly, Wbb=1,2,…,N represents the output vector of the inverse LSTM hidden layer, which is jointly determined by the word vector of the current input xb and the forward LSTM output of the previous moment Wb−1. o1,o2,…,oN is the output of the BiLSTM model, which is jointly determined by Wf and Wb, and the mathematical expression is as follows:(2)oN=wfWf+wbWb+bt,where wf represents the weight matrix of the forward LSTM output, wb represents the weight matrix of the reverse LSTM output, and bt is the bias.Conditional random fields (CRF). BiLSTM can predict the probability of each word corresponding to the predicted label, and then the label with the highest probability can be obtained by Softmax, but this will ignore the correlation between the labels, resulting in the label with the highest probability not conforming to the semantic logic, such as producing sentences with obvious errors like noun + verb + verb. Therefore, it is necessary to add some conditions that can constrain the sentences after the output layer of BiLSTM to ensure the validity of the final prediction results. The conditional random field (CRF) [17] model is a serialized labeling algorithm that can automatically learn some useful constraints to reduce erroneous prediction sequences when training data, and the structure of CRF is shown in Figure 7.Figure 7
CRF structure.The core principle is as follows:(3)ScoreX,y=∑iEi,yi+∑iTyi,yi+1,where E is the given emission matrix, T is the CRF parameter matrix that needs to be inverted and optimized by the loss function in the calculation, and ScoreX,y is the matching score for the given input and output. X is the input word sequence, and y is the predicted label sequence. Then, the following objective is maximized:(4)Pygold|X=expsX,ygold∑y∈YsexpsX,y.


### 2.4.3. Intent Recognition Module Based on BiGRU Model

Classifying questions from user input natural language and recognizing the user's intention constitute one of the indispensable tasks in Chinese natural language processing. The natural language question sentences input by users of military kill chain Q&A system are usually short texts, so the problem to be solved in this project is the classification problem of short texts.The bidirectional gated recurrent neural network (BiGRU) is selected for this intention recognition module. The gated recurrent neural network (GRU) [18] is an improvement of LSTM (with minimum unit consisting of input gate, forget gate, and output gate), and the reset gate of GRU merges the incoming gate and forget gate in LSTM, which makes the model structure simpler, reduces the number of parameters, and saves the training time. BiGRU, on the other hand, solves the problem that the input behind GRU is more important than the front when it advances from left to right, takes into account the future context information, and captures the complete context. Its structural model is shown in Figure 8.Figure 8
BiGRU structure.GRU detailed workflow is as follows:Step 1.
The reset gate controls which of the old cell states and inputs are discarded and which are retained, and its mathematical expression is as follows:(5)rt=σWrxt+Urht−1,where σ represents the Sigmoid activation function, xt represents the input information, and ht−1 represents the output of the hidden layer at the previous moment.Step 2.
Decide which new information is saved to the state of the metacell:(1)
The operation of forgetting the previous information and adding new information is done by the update gate and the mathematical expression is as follows:(6)zt=σWzxt+Uzht−1.(2)
Create a new candidate valueh˜t=tanhWhxt+rtUhht−1 from the tanh layer.Step 3.
The combination of Step 1 and Step 2 is used to update the old cell stateht−1 to the new cell state ht; the mathematical expression is as follows:(7)ht=1−ztht−1+zth˜t.


### 2.4.4. Answer Generation Module

Currently, there is no publicly available dataset for the types of interrogative sentences about the kill chain, and it is difficult to collect valid interrogative sentences about the military kill chain on military websites. Therefore, an interrogative template about military kill chains was designed for this project, and some of the interrogatives are displayed in Table7.Table 7
Question template.
Question templateExamplesWhat are the parameters of a particular weapon?What are the parameters of the E-3A Sentry?What platforms can a weapon be mounted on?What platforms can the AGM-158B JASSM-ER be equipped with?Which kill chains can a weapon apply to?What kill chains can the F-14B Tomcat be applied to?Composition of each link of a linkWhat is the equipment composition of each link in link 1?Since the knowledge graph triad has two representations: <entity-relationship-entity> and <entity-attribute-attribute value>, after the BiLSTM-CRF named entity model, we can get the entities in the triad, and after the BiGRU intent recognition model, we get the relationships/attributes in the triad. The obtained two parameters are input into the Neo4j graph database to get the required entity/attribute values and construct the corresponding answers to be returned.


## 2.4.1. Word2Vec Encoding Model

The user input words that express the intended meaning need to be word-vectorized in order to transform the abstract human symbols into a mathematical language for computer use. The current mainstream approach is to embed abstract words into a numerical space, i.e., word embedding. In this project, we use the mainstream Word2Vec word embedding method to transform words into vector form.Word2Vec encodes words into a vector based on the anticipation of information co-occurrence, which mainly includes two neural network models, Skip-gram model and continuous bag-of-words model (CBOW) [12], where Skip-gram model [13] predicts the words around the central word by the central word, while continuous bag-of-words model predicts the central word by the surrounding words. Due to the limited types of problems in the kill chain domain and the small number of datasets required for training, the CBOW model is chosen since it has a theoretically better performance with better results, and it is shown in Figure 4.Figure 4
CBOW model.CBOW model is generally a three-layer neural network structure, which is divided into input layer, hidden layer, and Softmax layer (output layer). The input of the model is the one-hot word vector of the adjacent words of the current position words (C in total; each dimension is V), and the data dimension is C∗V. The hidden layer multiplies the word vector of the adjacent vocabulary by the weight matrix Wdimension:V∗N to obtain C∗1∗N vectors and then adds and averages them (as shown in (1)) to obtain the hidden layer vector h with dimension 1∗N. The Softmax layer processes the output vector of the hidden layer through Softmax and selects the word with the highest probability as the word of the predicted current position. The predicted word is compared with the real word of the current position to obtain the error value, and the error value is continuously reduced through backpropagation. The training parameters of the CBOW model are shown in Table 6.(1)h=1Cx1+x2+⋯+xc∗W.Table 6
CBOW model training parameters.
Training parameterParameter valueLearning rate0.0001Number of contextual words10Word vector dimension250Batch size200Minimum word frequency5OptimizerAdam


## 2.4.2. Named Entity Recognition Module of Military Kill Chain Based on BiLSTM-CRF Model

Entity recognition is the main foundation of intelligent question-and-answer system, and the extraction effect of entity recognition more directly influences the quality and efficiency of subsequent natural processing. The current research methods for named entity extraction include rule-based, dictionary-based, machine learning-based, and statistical-based methods. However, due to the fact that there are mostly proper nouns in the field of kill chain and the problem of complex data relationships, entity recognition using the above methods is prone to errors and poor portability and requires a large-scale corpus to learn annotation, which requires manual participation in feature extraction and makes it difficult to avoid generating human errors.In recent years, with the continuous upgrading of computer hardware, deep learning has been widely developed and achieved good results in entity recognition tasks. The BERT-BiLSTM-CRF structural model has been widely used in various named entity recognition methods because of its high recognition rate, where BERT [14] is able to obtain contextualized word vectors to improve the performance of subsequent entity recognition and has a strong robustness. However, since BERT is based on transformer structure and training model, its number of parameters is huge (e.g., 110M for BERT-base and 330M for BERT-large), which is a big test for the deployed mobile computing power and is not conducive to its promotion. And, in this case, the number of entities named for the kill chain neighborhood is limited, and the phenomenon of multiple meanings of the word is small.In summary, this entity recognition model uses CBOW in Word2Vec as the word vector transformation layer and uses a bidirectional long short-term memory network model (BiLSTM) as the word vector encoding layer, as well as a conditional random field model layer (CRF) as the final output layer, and the overall model is shown in Figure5.Figure 5
BiLSTM-CRF entity recognition model.Bidirectional long short-term memory (BiLSTM). The long short-term memory network (LSTM) [15] can better solve the gradient explosion and gradient disappearance problems that occur because the sequence is too long, and can relate the dependencies over a long period of time. However, LSTM can only consider the information of the previous text, but in many cases, the output of the current moment is not only related to the previous text but also to the later text. Therefore, the BiLSTM, which is based on the improvement of LSTM, is proposed. The BiLSTM [16] model is a forward LSTM (dealing with the previous information) followed by a reverse LSTM to deal with the later information, capturing the contextual dependencies; i.e., the BiLSTM is composed of two unidirectional LSTMs concatenated together, and its structural model is shown in Figure 6.Figure 6
BiLSTM structure.x1,x2,...,xN is the word vector transformed by the CBOW model; Wff=1,2,…,N represents the forward LSTM hidden layer output vector, which is jointly determined by the word vector of the current input xf and the forward LSTM output of the previous moment Wf−1. Similarly, Wbb=1,2,…,N represents the output vector of the inverse LSTM hidden layer, which is jointly determined by the word vector of the current input xb and the forward LSTM output of the previous moment Wb−1. o1,o2,…,oN is the output of the BiLSTM model, which is jointly determined by Wf and Wb, and the mathematical expression is as follows:(2)oN=wfWf+wbWb+bt,where wf represents the weight matrix of the forward LSTM output, wb represents the weight matrix of the reverse LSTM output, and bt is the bias.Conditional random fields (CRF). BiLSTM can predict the probability of each word corresponding to the predicted label, and then the label with the highest probability can be obtained by Softmax, but this will ignore the correlation between the labels, resulting in the label with the highest probability not conforming to the semantic logic, such as producing sentences with obvious errors like noun + verb + verb. Therefore, it is necessary to add some conditions that can constrain the sentences after the output layer of BiLSTM to ensure the validity of the final prediction results. The conditional random field (CRF) [17] model is a serialized labeling algorithm that can automatically learn some useful constraints to reduce erroneous prediction sequences when training data, and the structure of CRF is shown in Figure 7.Figure 7
CRF structure.The core principle is as follows:(3)ScoreX,y=∑iEi,yi+∑iTyi,yi+1,where E is the given emission matrix, T is the CRF parameter matrix that needs to be inverted and optimized by the loss function in the calculation, and ScoreX,y is the matching score for the given input and output. X is the input word sequence, and y is the predicted label sequence. Then, the following objective is maximized:(4)Pygold|X=expsX,ygold∑y∈YsexpsX,y.


## 2.4.3. Intent Recognition Module Based on BiGRU Model

Classifying questions from user input natural language and recognizing the user's intention constitute one of the indispensable tasks in Chinese natural language processing. The natural language question sentences input by users of military kill chain Q&A system are usually short texts, so the problem to be solved in this project is the classification problem of short texts.The bidirectional gated recurrent neural network (BiGRU) is selected for this intention recognition module. The gated recurrent neural network (GRU) [18] is an improvement of LSTM (with minimum unit consisting of input gate, forget gate, and output gate), and the reset gate of GRU merges the incoming gate and forget gate in LSTM, which makes the model structure simpler, reduces the number of parameters, and saves the training time. BiGRU, on the other hand, solves the problem that the input behind GRU is more important than the front when it advances from left to right, takes into account the future context information, and captures the complete context. Its structural model is shown in Figure 8.Figure 8
BiGRU structure.GRU detailed workflow is as follows:Step 1.
The reset gate controls which of the old cell states and inputs are discarded and which are retained, and its mathematical expression is as follows:(5)rt=σWrxt+Urht−1,where σ represents the Sigmoid activation function, xt represents the input information, and ht−1 represents the output of the hidden layer at the previous moment.Step 2.
Decide which new information is saved to the state of the metacell:(1)
The operation of forgetting the previous information and adding new information is done by the update gate and the mathematical expression is as follows:(6)zt=σWzxt+Uzht−1.(2)
Create a new candidate valueh˜t=tanhWhxt+rtUhht−1 from the tanh layer.Step 3.
The combination of Step 1 and Step 2 is used to update the old cell stateht−1 to the new cell state ht; the mathematical expression is as follows:(7)ht=1−ztht−1+zth˜t.


## 2.4.4. Answer Generation Module

Currently, there is no publicly available dataset for the types of interrogative sentences about the kill chain, and it is difficult to collect valid interrogative sentences about the military kill chain on military websites. Therefore, an interrogative template about military kill chains was designed for this project, and some of the interrogatives are displayed in Table7.Table 7
Question template.
Question templateExamplesWhat are the parameters of a particular weapon?What are the parameters of the E-3A Sentry?What platforms can a weapon be mounted on?What platforms can the AGM-158B JASSM-ER be equipped with?Which kill chains can a weapon apply to?What kill chains can the F-14B Tomcat be applied to?Composition of each link of a linkWhat is the equipment composition of each link in link 1?Since the knowledge graph triad has two representations: <entity-relationship-entity> and <entity-attribute-attribute value>, after the BiLSTM-CRF named entity model, we can get the entities in the triad, and after the BiGRU intent recognition model, we get the relationships/attributes in the triad. The obtained two parameters are input into the Neo4j graph database to get the required entity/attribute values and construct the corresponding answers to be returned.


## 3. Results and Analysis

### 3.1. Experimental Environment and Settings

All training and testing in this work were performed on the same hardware and software platform. The environment is as follows: Windows (64 bit) operating system, Intel Core i7-9700 CPU, and 2080 Ti GPU. Considering the memory size of GPU and the experiment time, we set different training parameters according to different deep learning models, and the detailed training parameters are shown in Tables6, 8, and 9.Table 8
BiLSTM-CRF model training parameters.
Training parameterParameter valueBatch size32Number of LSTM hidden layer nodes128Learning rate0.00005Dropout0.5Maximum input length30Sentence annotation methodBIOESCompilation optimizerAdamTable 9
BiGRU model training parameters.
Training parameterParameter valueLearning rate0.0001Batch size32Word vector dimension250OptimizerAdamLoss functionCross-entropy loss functionThe number of kill chains collected this time is 2767, with a total of 30124 entities (including the four components of control equipment, sensor equipment, strike equipment (weapon and platform), and evaluator equipment), and the distribution of kill chain parts is shown in Table2. In this experiment, the model parameters and the average accuracy of the returned answer (input 200 times) are used as experimental indicators.


### 3.2. Application Scenarios

In order to better interact with users, this project has carried out cross platform design (used at both ends of IOS and Android) based on the current mainstream Uni-app development framework in the market. The page design is shown in Figure9.Figure 9
Query function.


### 3.3. Experimentation on the Effectiveness of the Q&A System Module

#### 3.3.1. Experiments on the Effectiveness of the CBOW Word Encoding Model

In order to verify the effectiveness of CBOW word encoding model in Q&A system, this paper conducted experiments on CBOW model, Skip-gram model, and BERT model under the same test environment, and the experimental results are shown in Table10.Table 10
Comparison of three word encoding models.
Word encoding modelModel parameters (M)Accuracy (%)One-hot, BiLSTM-CRF [10], BiGRU [11]24.469.8CBOW [12], BiLSTM-CRF, BiGRU27.985.5Skip-gram [13], BiLSTM-CRF, BiGRU28.177.2BERT [14], BiLSTM-CRF, BiGRU13190.1


#### 3.3.2. Experiments on the Effectiveness of the BiLSTM-CRF Named Entity Model

In order to verify the effectiveness of the BiLSTM-CRF named entity model in the Q&A system, this paper conducted experiments on the BiLSTM-CRF model, BiLSTM model, LSTM-CRF model, and LSTM model under the same test environment, and the experimental results are shown in Table11.Table 11
Comparison of three named entity models.
Named entity modelModel parameters (M)Accuracy (%)CBOW [12], LSTM [15], BiGRU [11]21.363.4CBOW, LSTM-CRF [17], BiGRU21.570.3CBOW, BiLSTM [16], BiGRU27.879.7CBOW, BiLSTM-CRF [10], BiGRU27.985.5


#### 3.3.3. Experiments on the Effectiveness of the BiGRU Intent Recognition Model

In order to verify the effectiveness of the BiGRU intention recognition model in the Q&A system, this paper conducted experiments on the BiGRU model, GRU model, and Text-CNN model [19] under the same test environment, and the experimental results are shown in Table 12.Table 12
Comparison of three Intention recognition model.
Intention recognition modelModel parameters (M)Accuracy (%)CBOW [12], BiLSTM-CRF [10], BiGRU [11]27.985.5CBOW, BiLSTM-CRF, GRU [18]19.776.8CBOW, BiLSTM-CRF, Text-CNN [19]24.881.1


## 3.1. Experimental Environment and Settings

All training and testing in this work were performed on the same hardware and software platform. The environment is as follows: Windows (64 bit) operating system, Intel Core i7-9700 CPU, and 2080 Ti GPU. Considering the memory size of GPU and the experiment time, we set different training parameters according to different deep learning models, and the detailed training parameters are shown in Tables6, 8, and 9.Table 8
BiLSTM-CRF model training parameters.
Training parameterParameter valueBatch size32Number of LSTM hidden layer nodes128Learning rate0.00005Dropout0.5Maximum input length30Sentence annotation methodBIOESCompilation optimizerAdamTable 9
BiGRU model training parameters.
Training parameterParameter valueLearning rate0.0001Batch size32Word vector dimension250OptimizerAdamLoss functionCross-entropy loss functionThe number of kill chains collected this time is 2767, with a total of 30124 entities (including the four components of control equipment, sensor equipment, strike equipment (weapon and platform), and evaluator equipment), and the distribution of kill chain parts is shown in Table2. In this experiment, the model parameters and the average accuracy of the returned answer (input 200 times) are used as experimental indicators.


## 3.2. Application Scenarios

In order to better interact with users, this project has carried out cross platform design (used at both ends of IOS and Android) based on the current mainstream Uni-app development framework in the market. The page design is shown in Figure9.Figure 9
Query function.


## 3.3. Experimentation on the Effectiveness of the Q&A System Module

### 3.3.1. Experiments on the Effectiveness of the CBOW Word Encoding Model

In order to verify the effectiveness of CBOW word encoding model in Q&A system, this paper conducted experiments on CBOW model, Skip-gram model, and BERT model under the same test environment, and the experimental results are shown in Table10.Table 10
Comparison of three word encoding models.
Word encoding modelModel parameters (M)Accuracy (%)One-hot, BiLSTM-CRF [10], BiGRU [11]24.469.8CBOW [12], BiLSTM-CRF, BiGRU27.985.5Skip-gram [13], BiLSTM-CRF, BiGRU28.177.2BERT [14], BiLSTM-CRF, BiGRU13190.1


### 3.3.2. Experiments on the Effectiveness of the BiLSTM-CRF Named Entity Model

In order to verify the effectiveness of the BiLSTM-CRF named entity model in the Q&A system, this paper conducted experiments on the BiLSTM-CRF model, BiLSTM model, LSTM-CRF model, and LSTM model under the same test environment, and the experimental results are shown in Table11.Table 11
Comparison of three named entity models.
Named entity modelModel parameters (M)Accuracy (%)CBOW [12], LSTM [15], BiGRU [11]21.363.4CBOW, LSTM-CRF [17], BiGRU21.570.3CBOW, BiLSTM [16], BiGRU27.879.7CBOW, BiLSTM-CRF [10], BiGRU27.985.5


### 3.3.3. Experiments on the Effectiveness of the BiGRU Intent Recognition Model

In order to verify the effectiveness of the BiGRU intention recognition model in the Q&A system, this paper conducted experiments on the BiGRU model, GRU model, and Text-CNN model [19] under the same test environment, and the experimental results are shown in Table 12.Table 12
Comparison of three Intention recognition model.
Intention recognition modelModel parameters (M)Accuracy (%)CBOW [12], BiLSTM-CRF [10], BiGRU [11]27.985.5CBOW, BiLSTM-CRF, GRU [18]19.776.8CBOW, BiLSTM-CRF, Text-CNN [19]24.881.1


## 3.3.1. Experiments on the Effectiveness of the CBOW Word Encoding Model

In order to verify the effectiveness of CBOW word encoding model in Q&A system, this paper conducted experiments on CBOW model, Skip-gram model, and BERT model under the same test environment, and the experimental results are shown in Table10.Table 10
Comparison of three word encoding models.
Word encoding modelModel parameters (M)Accuracy (%)One-hot, BiLSTM-CRF [10], BiGRU [11]24.469.8CBOW [12], BiLSTM-CRF, BiGRU27.985.5Skip-gram [13], BiLSTM-CRF, BiGRU28.177.2BERT [14], BiLSTM-CRF, BiGRU13190.1


## 3.3.2. Experiments on the Effectiveness of the BiLSTM-CRF Named Entity Model

In order to verify the effectiveness of the BiLSTM-CRF named entity model in the Q&A system, this paper conducted experiments on the BiLSTM-CRF model, BiLSTM model, LSTM-CRF model, and LSTM model under the same test environment, and the experimental results are shown in Table11.Table 11
Comparison of three named entity models.
Named entity modelModel parameters (M)Accuracy (%)CBOW [12], LSTM [15], BiGRU [11]21.363.4CBOW, LSTM-CRF [17], BiGRU21.570.3CBOW, BiLSTM [16], BiGRU27.879.7CBOW, BiLSTM-CRF [10], BiGRU27.985.5


## 3.3.3. Experiments on the Effectiveness of the BiGRU Intent Recognition Model

In order to verify the effectiveness of the BiGRU intention recognition model in the Q&A system, this paper conducted experiments on the BiGRU model, GRU model, and Text-CNN model [19] under the same test environment, and the experimental results are shown in Table 12.Table 12
Comparison of three Intention recognition model.
Intention recognition modelModel parameters (M)Accuracy (%)CBOW [12], BiLSTM-CRF [10], BiGRU [11]27.985.5CBOW, BiLSTM-CRF, GRU [18]19.776.8CBOW, BiLSTM-CRF, Text-CNN [19]24.881.1


## 4. Conclusion

In response to the inefficiency and low accuracy of manual weapon chain analysis, cluttered information of related search sites, many advertisements, and low correct analysis rate, this project uses artificial intelligence methods for military kill chain assessment. For the [20, 21] problem of large amount of kill chain data and complex data relationships obtained by Scrapy and Command simulation system, this project integrates all data based on Neo4j [22, 23] graph database and establishes clear data relationships. The Q&A system of this project has two [24] functions: query and matching, [25, 26] where the query function is to serve researchers and commanders for data mining, intent understanding, intelligence processing of kill chains, etc. We use the BiLSTM-CRF entity recognition model and the BiGRU intent recognition network to identify entities and relationships/attributes of natural language interrogatives input by users, and then use the Neo4j in triad to organize the correct answers and return them. The matching function is designed to address the inefficiency and low accuracy of manual weapon link analysis and the low correct analysis rate by using group clustering and rule constraint methods to evaluate whether a kill chain is composed. In order to make the user have a more convenient interface, we set up the military chain app based on Uni-app that is compatible with both IOS and Android systems, and we also designed the UI of the app, using green as the theme color of the app to match the subject matter of this project.



---

*Source: 7097385-2022-07-14.xml*