# Determination of Evidence Weights Based on Convolutional Neural Network for Classification Problem

Authors: Shijun Xu; Yi Hou; Xinpu Deng; Peibo Chen; Shilin Zhou
Journal: Journal of Sensors
 (2022)

---

## Abstract

The weighted average is an efficient way to address conflicting evidence combination in the Dempster-Shafer evidence theory. However, it is an open issue how to determine the evidence weights reasonably. Although many traditional conflicting evidence combination solutions based on evidence distance or entropy have been put forward, the evidence weights are determined with a single aspect, and no comprehensive consideration of other useful information affects the weights. Thus, it does not ensure that determination of weights is the most reasonable. By introducing deep learning into conflicting evidence combination, this paper proposes a comprehensive method for determining the evidence weights based on a convolutional neural network. Taking the evidence as the network input and the corresponding weight as the output, it utilizes convolutional neural network to fully mine potentially useful information that affects the evidence weights, in order to determine the weights comprehensively. Additionally, we define a weight loss function. The weights are continuously optimized through back propagation and achieve the optimal when the weight loss function value is the minimum. Classification experimental results demonstrate that the proposed method outperforms traditional ones based on evidence distance or entropy and can be flexibly extended to other application fields as a decision-making fusion method.


---

## Body

## 1. Introduction

In practical applications, information from different sources is often uncertain, inconsistent, and vague [1]. How to deal with uncertain information effectively is an open problem. Several mathematical theories have been discussed for solving this problem, such as Bayesian theory [2], fuzzy set theory [3, 4], Dempster-Shafer (D-S) evidence [5, 6], possibility theory [7, 8], D-numbers [9, 10], Z-numbers [11, 12], rough set theory [13, 14], and fractal theory [15, 16]. Especially D-S evidence theory, as a rational and effective method for dealing with uncertain information, has the following three advantages. First, it does not require prior probabilities or satisfy probability additivity compared with Bayesian theory. Secondly, information from different experts and data sources can be fused by Dempster’s combination rule [17] to obtain more reasonable results. Thirdly, it can describe uncertainty more flexibly and conviently than other mathematical theories. Therefore, it has been widely applied to classification [18, 19], risk evaluation [20, 21], fault diagnosis [22, 23], decision-making [24, 25], and so on.However, Zadeh [26] pointed out that evidence combination would produce counterintuitive results, when there is a conflict between evidences. To overcome this defect, scholars have conducted in-depth research and proposed various improved methods. In general, the existing methods can be divided into two categories. One is to modify Dempster’s combination rule; the other is to modify the original evidence.For the first class, some scholars point out that conflicting information is lost in the process of evidence combination. So the key to modifying Dempster’s combination rule is how to allocate the conflict, that is, to which subset the conflict is allocated and in what proportion. Smets et al. suggest that the conflict should be allocated to empty set [27]. Lefevre et al. propose a novel modified approach which proportionally assign the conflict information to the focal element sets [28]. Smardndache et al. propose the conflicting proportional distribution rule named PCR3 [29]. The method of modifying Dempster’s combination rule can solve the conflict problem to a certain extent, whereas the drawback of this method is that the good performance is destroyed, like commutativity and associativity. So this paper focuses on the second method.For the second class, the initial evidences are corrected with the weights to obtain the weighted average evidences. Then, the weighted average evidences are fused by using Dempster’s combination rule to get reasonable results. Therefore, the weighted average is an efficient way to address conflicting evidence combination. Nevertheless, this is a challenging problem how to determinate the evidence weights reasonably. For this reason, scholars have proposed some novel methods. Deng defined a new uncertainty measure Deng entropy to construct the weight coefficient bodies of evidence [30]. Tang et al. propose a weighted belief entropy which measure the uncertainty by using the information of the mass function and the scale of the FOD, in order to obtain the weight of evidence [31]. Qin et al. use a novel belief entropy which is an improved version of Dubois–Prade entropy and Nguyen entropy, to allocate the weights of evidence [32]. Yan et al. use an improved belief entropy based on Deng entropy to determine the weights of evidence [33]. Liu et al. propose a novel weighted evidence combination based on MaxDiff distance [34]. Han et al. introduced the concept of evidence support based on the Jousselme distance function and took a weighted average of all the evidences [35]. Liu et al. design an improved weighted evidence combination method by combining probability distance and conflict coefficient [36]. Xiao generalizes the traditional Jousselme distance to the complex evidence distance to measure the conflicts of complex the basic probability assignment (BPA) functions [37], and used it as a weighted factor to revise the original evidence [38]. The above methods only use entropy or distance information to determine the weights from the perspective of uncertainty and evidence conflict. The disadvantage is that determination of the weights is rather one-sided, and no comprehensive consideration of other useful information that affects the weights, such as importance, reliability, relativity, and unknown information hidden within or between evidence. Consequently, these methods based on evidence distance or entropy do not ensure that determination of weights is the most reasonable.In view of the powerful adaptive learning and information mining capabilities of convolutional neural networks (CNN), we introduce deep learning into conflicting evidence combination and propose a comprehensive method for determining the evidence weights based on a CNN to solve this problem. Taking the evidence as the network input and the corresponding weight as the output, we define a weight loss function. Through back propagation, update the network parameters and fully mine potentially useful information that affects the weights. So the evidence weights are determined comprehensively and optimized continuously, and finally achieve the optimal when the weight loss function value is the minimum. Compared with traditional algorithms based on evidence distance or entropy, the proposed method makes the determination of weights more reasonable and can achieve higher accuracy rate in classification application.In summary, the primary contributions in this study are summarized as follows:(i)
Different from traditional methods, this paper proposes a comprehensive method for determining the evidence weights based on CNN(ii)
The evidence weights are not determined for a certain aspect related to the evidence and can reflect the relationship among evidence comprehensively(iii)
Compared with the traditional algorithms based on evidence distance or entropy, the proposed method can achieve higher accuracy rate in classification applicationThe rest part of this paper is organized below. Section2 introduces some relevant basic theoretical knowledge about the D-S evidence theory. Section 3 proposes a comprehensive method for determining the evidence weights based on a CNN and introduces overview of proposed method, CNN architecture, and the weight loss function. Section 4 presents the classification application of the proposed method, and analyzes and discusses its results. Conclusions are in Section 5.


## 2. D-S Evidence Theory

D-S evidence theory is a reasoning system theory first put forward by Dempster in 1967 and further developed by Shafer in 1976. Compared with Bayesian probability theory, it can more flexibly and effectively deal with uncertain information without prior probabilities. Thus, it is an extension of Bayesian probability theory. In the frame of D-S evidence theory, Dempster’s combination rule can be used to combinate evidences collected from different sources, which satisfies the commutative and associative laws. Some basic concepts about evidence theory is introduced as follows.Definition 1 (Frame of discernment).
Assuming thatΘ is a set of mutually exclusive and exhaustive elements Fii=1,2,3,4,⋯,N, and it can be defined as [31].
(1)Θ=F1,F2,F3,F4,⋯,FN,where Θ is called the frame of discernment (FOD), and Fi is named single-element proposition or subset. We define 2Θ as a power set which contains 2N elements and can be described as
(2)2Θ=∅,F1,⋯,FN,F1,F2,F1,F3,⋯,Θ,where ∅ is an empty set in Eq.(2).Definition 2 (The basic probability assignment function).
The BPA functionm is also called mass function and is defined as a mapping of the power set 2Θ to [0,1] [33].
(3)m:2Θ⟶0,1,which satisfies
(4)m∅=00≤mA≤1,∑A⊆ΘmA=1,where mass function mA represents the degree of support to A, and A is called focal element or proposition. The mass function m∅ is equal to 0 in classical D-S evidence theory.Definition 3 (Dempster’ s combination rule).
In D-S evidence theory, two BPAs can be combined with Dempster’ s rule of combination, defined as followers [17]:
(5)mA=m1⊕m2A=11−K∑B∩C=Am1Bm2C,in which
(6)K=∑B∩C=∅m1Bm2C,where ⊕ represents Dempster’s combination rule. K is called conflict coefficient, and it has values between 0 and 1. The bigger K is, the more conflict between two evidences is.Definition 4 (Weighted average evidence).
Suppose thatm1,m2,⋯,mn is the evidence collected from different data sources, and w1,w2,⋯,wn is the corresponding weights of evidence. Then, original evidence can be modified by the weight to obtain the weighted average evidence:
(7)m′=∑i=1nwi×mi,in which
(8)∑i=1nwi=1.The weightwi is equal to 1, indicating that the corresponding evidence mi is a piece of information that can be fully reliable. Relatively small weight indicates that it plays little role in evidence combination, and the weight is equal to 0, which means that it can be discarded directly.


## 3. Determination of Evidence Weights Based on Convolutional Neural Network

### 3.1. Overview of Proposed Method

To determine the evidence weights more comprehensively and reasonably, we propose a novel method for determining the evidence weights based on a CNN. First, we take initial BPAs, namely, evidences, as the input of CNN, and the corresponding weights are defined as output. Then, we define a weight loss function and the weights are continuously optimized through back propagation. Finally, the optimal evidence weights can be obtained when the weight loss function value is the minimum.


### 3.2. CNN Architecture

In this section, a CNN is introduced, which consists of a convolution layer, two fully connected layer, and a softmax function output layer, as shown in Figure1. The initial BPA mi which is a 1×N vector is defined as the input to the network. N represents the number of elements within FOD, namely, the number of categories in the dataset. Before a convolution operation, these input evidence vectors are combined into k×N matrix. After that, we choose k×k convolution kernels to mine the potential information contained within and between the evidences. Finally, the outputs of two fully connected layer are fed into a softmax function output layer, in order to produce probability weights corresponding to the input evidences. The size of the convolution kernel and softmax function output lay depends on the amount of input evidences.Figure 1
Architecture of the proposed method (S=Stride, P=Padding).


### 3.3. The Weight Loss Function

During the conflicting evidence combination, the input evidencemi is modified by the output weight wi:
(9)m′=∑i=1kwi×mi,where wi ranges from 0 to 1 and m′ is the weighted average evidence. After the input evidences is corrected, the final combination results y^ can be got by using Dempster’s combination rule to combine the weighted averaged evidence m′ for k−1 times:
(10)y^A=m′⊕m′⊕⋯⊕m′k−1A=∑A1,⋯,Ak⊆ΘA1∩⋯∩Ak=Am′A1⋯m′Ak1−∑A1,⋯,Ak⊆ΘA1∩⋯∩Ak=∅m′A1⋯m′Ak,where ⊕ represents Dempster’s combination rule which satisfies the polarizability. It means that the total belief degree of a single element increases and the total belief degree of multiple elements decreases when multiple identical evidences are fused. In addition, provided the basic probability value of a single element of the evidence is the largest, the basic probability value of this single element is still the largest when two identical evidences are combinated by Dempster’s combination rule.The output weights corresponding to the input evidences should be determined according to the following rules: (a) The higher the relativity between the input evidence vector and the true category one-hot vector is, the larger its corresponding output weight will be. (b) The higher the conflict between the input evidence vector and the true category one-hot vector is, the smaller its corresponding output weight will be, that is, the evidence will contribute little to the combination results.Based on the above rules, we define a weight loss functionLw as the sum of the cross entropy between all pairs of yi and y^i to obtain the optimal evidence weights:
(11)Lw=−1n∑iyilny^i.In the above formula,n represents the number of training samples. yi∈R1×N is the true category one-hot vector. y^i is the final combination result. In the process of seeking the minimized loss function value Lw, when Lw does not meet the condition, network parameters are constantly updated and output weights are continuously optimized through feedback. Since Dempster’s combination rule satisfies the polarizability, the weight of normal evidence will be larger and larger, and the weight of conflicting evidence will be smaller and smaller. When Lw satisfies the condition, namely, reaches the minimum value, the output weights achieve the optimal. The flow of the evidence weights optimization is showed in Figure 2.Figure 2
Flow of the evidence weights optimization.


## 3.1. Overview of Proposed Method

To determine the evidence weights more comprehensively and reasonably, we propose a novel method for determining the evidence weights based on a CNN. First, we take initial BPAs, namely, evidences, as the input of CNN, and the corresponding weights are defined as output. Then, we define a weight loss function and the weights are continuously optimized through back propagation. Finally, the optimal evidence weights can be obtained when the weight loss function value is the minimum.


## 3.2. CNN Architecture

In this section, a CNN is introduced, which consists of a convolution layer, two fully connected layer, and a softmax function output layer, as shown in Figure1. The initial BPA mi which is a 1×N vector is defined as the input to the network. N represents the number of elements within FOD, namely, the number of categories in the dataset. Before a convolution operation, these input evidence vectors are combined into k×N matrix. After that, we choose k×k convolution kernels to mine the potential information contained within and between the evidences. Finally, the outputs of two fully connected layer are fed into a softmax function output layer, in order to produce probability weights corresponding to the input evidences. The size of the convolution kernel and softmax function output lay depends on the amount of input evidences.Figure 1
Architecture of the proposed method (S=Stride, P=Padding).


## 3.3. The Weight Loss Function

During the conflicting evidence combination, the input evidencemi is modified by the output weight wi:
(9)m′=∑i=1kwi×mi,where wi ranges from 0 to 1 and m′ is the weighted average evidence. After the input evidences is corrected, the final combination results y^ can be got by using Dempster’s combination rule to combine the weighted averaged evidence m′ for k−1 times:
(10)y^A=m′⊕m′⊕⋯⊕m′k−1A=∑A1,⋯,Ak⊆ΘA1∩⋯∩Ak=Am′A1⋯m′Ak1−∑A1,⋯,Ak⊆ΘA1∩⋯∩Ak=∅m′A1⋯m′Ak,where ⊕ represents Dempster’s combination rule which satisfies the polarizability. It means that the total belief degree of a single element increases and the total belief degree of multiple elements decreases when multiple identical evidences are fused. In addition, provided the basic probability value of a single element of the evidence is the largest, the basic probability value of this single element is still the largest when two identical evidences are combinated by Dempster’s combination rule.The output weights corresponding to the input evidences should be determined according to the following rules: (a) The higher the relativity between the input evidence vector and the true category one-hot vector is, the larger its corresponding output weight will be. (b) The higher the conflict between the input evidence vector and the true category one-hot vector is, the smaller its corresponding output weight will be, that is, the evidence will contribute little to the combination results.Based on the above rules, we define a weight loss functionLw as the sum of the cross entropy between all pairs of yi and y^i to obtain the optimal evidence weights:
(11)Lw=−1n∑iyilny^i.In the above formula,n represents the number of training samples. yi∈R1×N is the true category one-hot vector. y^i is the final combination result. In the process of seeking the minimized loss function value Lw, when Lw does not meet the condition, network parameters are constantly updated and output weights are continuously optimized through feedback. Since Dempster’s combination rule satisfies the polarizability, the weight of normal evidence will be larger and larger, and the weight of conflicting evidence will be smaller and smaller. When Lw satisfies the condition, namely, reaches the minimum value, the output weights achieve the optimal. The flow of the evidence weights optimization is showed in Figure 2.Figure 2
Flow of the evidence weights optimization.


## 4. Experimental Results and Discussion

### 4.1. Experimental Setup

This section gives the introduction of UCR datasets, and the datasets generation, parameter settings, and experiment procedure are presented.


#### 4.1.1. UCR

UCR datasets are obtained from UCR time series data mining archive (http://www.cs.ucr.edu/eamonn/timeseriesdata/) which is a publicly available and real-world time series dataset and has always been used for classification. To verify the classification ability of the proposed method, we conduct the experiments on 6 UCR datasets, including ElectricDevices (ED), UWaveGestureLibraryY (UY), UWaveGestureLibraryZ (UZ), CricketX, CricketY, and CricketZ. ElectricDevices consists of 7711 test samples and contains 7 categories. Both UWaveGestureLibraryY and UWaveGestureLibraryZ consist of 3582 test samples, and contain 8 categories. Each of CricketX, CricketY, and CricketZ consists of 390 test samples and contains 12 categories.


#### 4.1.2. Datasets Generation

UCR datasets are the original data containing attribute information and cannot be directly used as the input of the model which determines the evidence weights. Therefore, to obtain the initial BPAs from these datasets, we utilize the BPA generation method that is the probability output of neural network [39]. Each class in UCR test set can be regarded as one element within the frame of discernment Θ=F1,F2,F3,⋯,FN. These elements are exclusive and independent. We take the probability output of a UCR test set on a single neural network as BPAs. Since the MultiLayer Perceptron (MLP), Fully Convolutional Network (FCN), and Residual Network (ResNet) are defined as the standard baseline in the time series classification [40], these three single neural network are used to generate BPAs in this paper. A test sample will generate three pieces of evidence on three single neural networks. For the same test set, the prediction accuracy rates obtained on three single neural networks which differ in structure and performance will certainly not be all the same, so the BPAs generated by three neural networks will be different. In other words, there is a conflict between the BPAs. Hence, it is reasonable that we use the probability output of a test set on the MLP, FCN, and ResNet as a dataset for conflicting evidence combination research. However, the BPAs generated by the neural network only contains singleton element proposition. Therefore, the artificial simulation method is adopted to obtain the BPAs which contain multi-element propositions. The artificial synthesis dataset is formed by manually collecting classical numerical examples from papers with conflicting evidence fusion. It consists of 21 training samples and 42 test samples. Each sample in this dataset contains 5 pieces of evidences, and 3 categories which are denoted by F1,F2,F3. Assuming that the five pieces of evidence in the first sample are defined as m11·, m12·, m13·, m14·, and m15·, the BPAs are shown as follows.
(12)m11F1=0.41,m11F2=0.29,m11F3=0.30m12F2=0.90,m12F1,F3=0.10m13F1=0.58,m13F2=0.07,m13F1,F3=0.35m14F1=0.55,m14F2=0.10,m14F1,F3=0.35m15F1=0.60,m15F2=0.10,m15F1,F3=0.30.


#### 4.1.3. Parameter Settings

The proposed method is trained in TensorFlow using back propagation with Adam to update the network. The learning rate of Adam is 0.001,β1= 0.9, β2= 0.999, and ε=1e-8 [40]. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8, and the number of training sets is 1542, 716, 716, 78, 78, and 78, respectively. Therefore, the batch sizes of 13 and 19 are opted. Since the proposed model is not complicated, we set the number of epochs to 2000.


#### 4.1.4. Experiment Procedure

This paper uses two different types of datasets as database to introduce the application of the proposed method in classification. A detailed description of the procedure is depicted as follows:(1)
The first type of dataset containing multi-element propositions is derived from the artificial synthesis dataset. Each sample in this dataset contains 5 pieces of evidences. Correspondingly, the size of the convolution kernel and the softmax function output lay is 5. The second type of dataset contains only single-element propositions. It is derived from the predicted results of 6 UCR test sets on the MLP, FCN, and Resnet. One sample in these test sets will produce 3 pieces of evidences. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8. In order to verify the robustness of model, each dataset is divided randomly for 5 times, and the same random state is repeated for 5 times to record the average value(2)
Get the optimal evidence weights when the proposed cross entropy loss function value satisfies the minimum(3)
Original evidences are corrected with the weights to obtain the weighted average evidences(4)
Then, the weighted average evidences are fused by using Dempster’s rule to get the final combination evidences(5)
The predicted category is determined by the final combination evidence. As to the first class, the category corresponding to the maximum value in the singleton element is the predicted result. As to the second class, the category corresponding to the maximum probability value is the predicted resultTo further verify the feasibility of the proposed method, we have compared with four well-known traditional methods, namely, classical D-S theory method [17], two belief entropy-based methods [30, 33], and a distance-based method [35]. Accuracy and processing time for classification are adopted as the evaluation index.


### 4.2. Results and Discussion

#### 4.2.1. Classification Results

The classification accuracy on the two types of datasets are shown in Tables1 and 2, respectively. According to experimental results, we can draw the following conclusions: (a) For the first type of dataset, Yan et al.’s method, Han et al.’s method, and our proposed method similarly achieve the highest classification accuracy of 97.6%, while Dempster’s has a classification accuracy of 64.2% and Deng’s method has a classification accuracy of 95.2%. It indicates that the proposed method retains competitive performance. (b) The classification accuracy of the proposed method is always higher compared with a single best-performing neural network classification algorithm, and the average improvement on the second type of datasets is 2.47%, 4.20%, 1.59%, 0.74%, 3.21%, and 1.73%, respectively. It illustrates that the proposed method which combines the predicted results of three single neural networks can obtain better classification results than a single network. In addition, proposed method is to get the final decision by fusing multiple prediction results, which is regarded as the decision level fusion, so it can be flexibly extended to other application fields. (c) For the same dataset, the classification accuracy of the proposed method under five random states has little fluctuation. It proves the robustness of our model. (d) Compared with classic traditional method, two belief entropy-based traditional methods and a distance-based traditional method on the second type of datasets, the classification accuracy of the proposed method is the highest, and the total average improvement is 4.06%, 10.11%, 10.11%, and 16.19%, respectively. These indicate that the determination of weights based on CNN is more reasonable, which proves the validity of proposed method.Table 1
The classification performance comparison on the first type of dataset.
DatasetDempster [17]Deng[30]Yan and Deng [33]Han et al. [35]Proposed methodArtificial synthesis64.2%95.2%97.6%97.6%97.6%Table 2
The classification performance comparison on the second type of datasets.
DatasetRandom stateMLPFCNResNetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodED159.1%69.2%74.1%71.4%61.6%61.6%37.9%76.5%259.2%68.9%74.2%71.6%61.8%61.8%37.3%76.4%359.6%69.6%74.0%71.9%62.0%62.0%37.7%77.1%459.8%69.3%74.2%71.6%62.3%62.3%37.2%76.7%559.3%69.4%74.3%71.4%62.2%62.2%37.1%76.4%UY168.1%60.0%64.1%71.9%62.8%62.8%70.4%72.4%269.2%59.9%64.6%72.0%62.6%62.6%71.4%73.5%368.0%59.6%64.4%71.3%62.5%62.5%71.1%72.3%469.1%60.3%65.0%72.5%63.3%63.3%71.4%73.7%569.2%59.9%64.4%72.5%63.1%63.1%71.2%72.7%UZ168.3%69.6%75.8%76.6%70.1%70.1%64.5%77.4%269.5%69.4%75.8%76.9%70.5%70.5%64.4%77.5%368.8%69.3%75.8%76.6%70.2%70.2%64.0%77.7%469.1%69.9%75.5%76.4%70.5%70.5%64.2%76.5%568.4%69.4%75.2%76.3%70.4%70.4%64.2%76.9%CricketX156.0%79.1%78.2%72.1%74.0%74.0%63.4%80.4%254.8%79.8%78.5%73.0%72.7%72.7%63.4%80.4%358.9%80.7%78.8%73.7%74.3%74.3%62.1%80.8%457.0%81.0%79.4%74.3%73.7%73.7%62.1%82.1%558.9%80.7%80.1%73.0%74.6%74.6%63.1%81.4%CricketY158.6%78.2%78.5%76.9%68.9%68.9%70.1%82.6%260.2%79.4%80.1%77.8%69.8%69.8%69.5%83.4%360.8%79.1%79.4%76.6%70.1%70.1%69.8%83.5%460.2%78.5%79.8%76.6%68.5%68.5%69.2%81.6%557.6%78.2%79.1%75.0%68.5%68.5%70.8%81.9%CricketZ161.2%79.4%80.1%78.2%73.3%73.3%69.5%82.5%260.2%80.4%79.4%77.2%73.0%73.0%70.1%81.4%360.2%81.0%80.4%79.4%74.3%74.3%69.8%83.3%460.5%79.1%80.4%78.2%73.7%73.7%70.5%82.6%561.8%80.1%81.7%79.4%75.3%75.3%70.8%82.5%


#### 4.2.2. Processing Time Results

The experiment of processing time is implemented on an ordinary personal computer with a Intel Core i7-9750H CPU at 2.60 GHz and 8 GB RAM. According to the time complexity calculation presented in [41], the time complexity of four traditional algorithms is Ox×2n, Ox×22n, Ox×22n, and Ox2×2n, where x and n represent the number of evidences and elements in the FOD, respectively. The time complexity of Deng and Yan’s algorithms is higher. For all datasets, we take the average processing time of 5 random states for comparison, and the results are illustrated in Table 3. The proposed method spends a lot of time in the training phase, but after training, the processing time of our approach is comparable to that of the other four methods.Table 3
The processing time of different approaches.
DatasetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodTraining time (s)Testing time (s)ED0.271.081.080.67252.330.05UY0.140.590.590.35117.430.02UZ0.140.590.590.35117.680.02CricketX0.020.090.090.0616.760.01CricketY0.020.090.090.0616.590.01CricketZ0.020.090.090.0617.170.01Artificial synthesis0.010.010.010.01235.530.04


## 4.1. Experimental Setup

This section gives the introduction of UCR datasets, and the datasets generation, parameter settings, and experiment procedure are presented.


### 4.1.1. UCR

UCR datasets are obtained from UCR time series data mining archive (http://www.cs.ucr.edu/eamonn/timeseriesdata/) which is a publicly available and real-world time series dataset and has always been used for classification. To verify the classification ability of the proposed method, we conduct the experiments on 6 UCR datasets, including ElectricDevices (ED), UWaveGestureLibraryY (UY), UWaveGestureLibraryZ (UZ), CricketX, CricketY, and CricketZ. ElectricDevices consists of 7711 test samples and contains 7 categories. Both UWaveGestureLibraryY and UWaveGestureLibraryZ consist of 3582 test samples, and contain 8 categories. Each of CricketX, CricketY, and CricketZ consists of 390 test samples and contains 12 categories.


### 4.1.2. Datasets Generation

UCR datasets are the original data containing attribute information and cannot be directly used as the input of the model which determines the evidence weights. Therefore, to obtain the initial BPAs from these datasets, we utilize the BPA generation method that is the probability output of neural network [39]. Each class in UCR test set can be regarded as one element within the frame of discernment Θ=F1,F2,F3,⋯,FN. These elements are exclusive and independent. We take the probability output of a UCR test set on a single neural network as BPAs. Since the MultiLayer Perceptron (MLP), Fully Convolutional Network (FCN), and Residual Network (ResNet) are defined as the standard baseline in the time series classification [40], these three single neural network are used to generate BPAs in this paper. A test sample will generate three pieces of evidence on three single neural networks. For the same test set, the prediction accuracy rates obtained on three single neural networks which differ in structure and performance will certainly not be all the same, so the BPAs generated by three neural networks will be different. In other words, there is a conflict between the BPAs. Hence, it is reasonable that we use the probability output of a test set on the MLP, FCN, and ResNet as a dataset for conflicting evidence combination research. However, the BPAs generated by the neural network only contains singleton element proposition. Therefore, the artificial simulation method is adopted to obtain the BPAs which contain multi-element propositions. The artificial synthesis dataset is formed by manually collecting classical numerical examples from papers with conflicting evidence fusion. It consists of 21 training samples and 42 test samples. Each sample in this dataset contains 5 pieces of evidences, and 3 categories which are denoted by F1,F2,F3. Assuming that the five pieces of evidence in the first sample are defined as m11·, m12·, m13·, m14·, and m15·, the BPAs are shown as follows.
(12)m11F1=0.41,m11F2=0.29,m11F3=0.30m12F2=0.90,m12F1,F3=0.10m13F1=0.58,m13F2=0.07,m13F1,F3=0.35m14F1=0.55,m14F2=0.10,m14F1,F3=0.35m15F1=0.60,m15F2=0.10,m15F1,F3=0.30.


### 4.1.3. Parameter Settings

The proposed method is trained in TensorFlow using back propagation with Adam to update the network. The learning rate of Adam is 0.001,β1= 0.9, β2= 0.999, and ε=1e-8 [40]. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8, and the number of training sets is 1542, 716, 716, 78, 78, and 78, respectively. Therefore, the batch sizes of 13 and 19 are opted. Since the proposed model is not complicated, we set the number of epochs to 2000.


### 4.1.4. Experiment Procedure

This paper uses two different types of datasets as database to introduce the application of the proposed method in classification. A detailed description of the procedure is depicted as follows:(1)
The first type of dataset containing multi-element propositions is derived from the artificial synthesis dataset. Each sample in this dataset contains 5 pieces of evidences. Correspondingly, the size of the convolution kernel and the softmax function output lay is 5. The second type of dataset contains only single-element propositions. It is derived from the predicted results of 6 UCR test sets on the MLP, FCN, and Resnet. One sample in these test sets will produce 3 pieces of evidences. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8. In order to verify the robustness of model, each dataset is divided randomly for 5 times, and the same random state is repeated for 5 times to record the average value(2)
Get the optimal evidence weights when the proposed cross entropy loss function value satisfies the minimum(3)
Original evidences are corrected with the weights to obtain the weighted average evidences(4)
Then, the weighted average evidences are fused by using Dempster’s rule to get the final combination evidences(5)
The predicted category is determined by the final combination evidence. As to the first class, the category corresponding to the maximum value in the singleton element is the predicted result. As to the second class, the category corresponding to the maximum probability value is the predicted resultTo further verify the feasibility of the proposed method, we have compared with four well-known traditional methods, namely, classical D-S theory method [17], two belief entropy-based methods [30, 33], and a distance-based method [35]. Accuracy and processing time for classification are adopted as the evaluation index.


## 4.1.1. UCR

UCR datasets are obtained from UCR time series data mining archive (http://www.cs.ucr.edu/eamonn/timeseriesdata/) which is a publicly available and real-world time series dataset and has always been used for classification. To verify the classification ability of the proposed method, we conduct the experiments on 6 UCR datasets, including ElectricDevices (ED), UWaveGestureLibraryY (UY), UWaveGestureLibraryZ (UZ), CricketX, CricketY, and CricketZ. ElectricDevices consists of 7711 test samples and contains 7 categories. Both UWaveGestureLibraryY and UWaveGestureLibraryZ consist of 3582 test samples, and contain 8 categories. Each of CricketX, CricketY, and CricketZ consists of 390 test samples and contains 12 categories.


## 4.1.2. Datasets Generation

UCR datasets are the original data containing attribute information and cannot be directly used as the input of the model which determines the evidence weights. Therefore, to obtain the initial BPAs from these datasets, we utilize the BPA generation method that is the probability output of neural network [39]. Each class in UCR test set can be regarded as one element within the frame of discernment Θ=F1,F2,F3,⋯,FN. These elements are exclusive and independent. We take the probability output of a UCR test set on a single neural network as BPAs. Since the MultiLayer Perceptron (MLP), Fully Convolutional Network (FCN), and Residual Network (ResNet) are defined as the standard baseline in the time series classification [40], these three single neural network are used to generate BPAs in this paper. A test sample will generate three pieces of evidence on three single neural networks. For the same test set, the prediction accuracy rates obtained on three single neural networks which differ in structure and performance will certainly not be all the same, so the BPAs generated by three neural networks will be different. In other words, there is a conflict between the BPAs. Hence, it is reasonable that we use the probability output of a test set on the MLP, FCN, and ResNet as a dataset for conflicting evidence combination research. However, the BPAs generated by the neural network only contains singleton element proposition. Therefore, the artificial simulation method is adopted to obtain the BPAs which contain multi-element propositions. The artificial synthesis dataset is formed by manually collecting classical numerical examples from papers with conflicting evidence fusion. It consists of 21 training samples and 42 test samples. Each sample in this dataset contains 5 pieces of evidences, and 3 categories which are denoted by F1,F2,F3. Assuming that the five pieces of evidence in the first sample are defined as m11·, m12·, m13·, m14·, and m15·, the BPAs are shown as follows.
(12)m11F1=0.41,m11F2=0.29,m11F3=0.30m12F2=0.90,m12F1,F3=0.10m13F1=0.58,m13F2=0.07,m13F1,F3=0.35m14F1=0.55,m14F2=0.10,m14F1,F3=0.35m15F1=0.60,m15F2=0.10,m15F1,F3=0.30.


## 4.1.3. Parameter Settings

The proposed method is trained in TensorFlow using back propagation with Adam to update the network. The learning rate of Adam is 0.001,β1= 0.9, β2= 0.999, and ε=1e-8 [40]. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8, and the number of training sets is 1542, 716, 716, 78, 78, and 78, respectively. Therefore, the batch sizes of 13 and 19 are opted. Since the proposed model is not complicated, we set the number of epochs to 2000.


## 4.1.4. Experiment Procedure

This paper uses two different types of datasets as database to introduce the application of the proposed method in classification. A detailed description of the procedure is depicted as follows:(1)
The first type of dataset containing multi-element propositions is derived from the artificial synthesis dataset. Each sample in this dataset contains 5 pieces of evidences. Correspondingly, the size of the convolution kernel and the softmax function output lay is 5. The second type of dataset contains only single-element propositions. It is derived from the predicted results of 6 UCR test sets on the MLP, FCN, and Resnet. One sample in these test sets will produce 3 pieces of evidences. The second type of dataset is randomly divided into the training set and test set according to the ratio of 2 : 8. In order to verify the robustness of model, each dataset is divided randomly for 5 times, and the same random state is repeated for 5 times to record the average value(2)
Get the optimal evidence weights when the proposed cross entropy loss function value satisfies the minimum(3)
Original evidences are corrected with the weights to obtain the weighted average evidences(4)
Then, the weighted average evidences are fused by using Dempster’s rule to get the final combination evidences(5)
The predicted category is determined by the final combination evidence. As to the first class, the category corresponding to the maximum value in the singleton element is the predicted result. As to the second class, the category corresponding to the maximum probability value is the predicted resultTo further verify the feasibility of the proposed method, we have compared with four well-known traditional methods, namely, classical D-S theory method [17], two belief entropy-based methods [30, 33], and a distance-based method [35]. Accuracy and processing time for classification are adopted as the evaluation index.


## 4.2. Results and Discussion

### 4.2.1. Classification Results

The classification accuracy on the two types of datasets are shown in Tables1 and 2, respectively. According to experimental results, we can draw the following conclusions: (a) For the first type of dataset, Yan et al.’s method, Han et al.’s method, and our proposed method similarly achieve the highest classification accuracy of 97.6%, while Dempster’s has a classification accuracy of 64.2% and Deng’s method has a classification accuracy of 95.2%. It indicates that the proposed method retains competitive performance. (b) The classification accuracy of the proposed method is always higher compared with a single best-performing neural network classification algorithm, and the average improvement on the second type of datasets is 2.47%, 4.20%, 1.59%, 0.74%, 3.21%, and 1.73%, respectively. It illustrates that the proposed method which combines the predicted results of three single neural networks can obtain better classification results than a single network. In addition, proposed method is to get the final decision by fusing multiple prediction results, which is regarded as the decision level fusion, so it can be flexibly extended to other application fields. (c) For the same dataset, the classification accuracy of the proposed method under five random states has little fluctuation. It proves the robustness of our model. (d) Compared with classic traditional method, two belief entropy-based traditional methods and a distance-based traditional method on the second type of datasets, the classification accuracy of the proposed method is the highest, and the total average improvement is 4.06%, 10.11%, 10.11%, and 16.19%, respectively. These indicate that the determination of weights based on CNN is more reasonable, which proves the validity of proposed method.Table 1
The classification performance comparison on the first type of dataset.
DatasetDempster [17]Deng[30]Yan and Deng [33]Han et al. [35]Proposed methodArtificial synthesis64.2%95.2%97.6%97.6%97.6%Table 2
The classification performance comparison on the second type of datasets.
DatasetRandom stateMLPFCNResNetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodED159.1%69.2%74.1%71.4%61.6%61.6%37.9%76.5%259.2%68.9%74.2%71.6%61.8%61.8%37.3%76.4%359.6%69.6%74.0%71.9%62.0%62.0%37.7%77.1%459.8%69.3%74.2%71.6%62.3%62.3%37.2%76.7%559.3%69.4%74.3%71.4%62.2%62.2%37.1%76.4%UY168.1%60.0%64.1%71.9%62.8%62.8%70.4%72.4%269.2%59.9%64.6%72.0%62.6%62.6%71.4%73.5%368.0%59.6%64.4%71.3%62.5%62.5%71.1%72.3%469.1%60.3%65.0%72.5%63.3%63.3%71.4%73.7%569.2%59.9%64.4%72.5%63.1%63.1%71.2%72.7%UZ168.3%69.6%75.8%76.6%70.1%70.1%64.5%77.4%269.5%69.4%75.8%76.9%70.5%70.5%64.4%77.5%368.8%69.3%75.8%76.6%70.2%70.2%64.0%77.7%469.1%69.9%75.5%76.4%70.5%70.5%64.2%76.5%568.4%69.4%75.2%76.3%70.4%70.4%64.2%76.9%CricketX156.0%79.1%78.2%72.1%74.0%74.0%63.4%80.4%254.8%79.8%78.5%73.0%72.7%72.7%63.4%80.4%358.9%80.7%78.8%73.7%74.3%74.3%62.1%80.8%457.0%81.0%79.4%74.3%73.7%73.7%62.1%82.1%558.9%80.7%80.1%73.0%74.6%74.6%63.1%81.4%CricketY158.6%78.2%78.5%76.9%68.9%68.9%70.1%82.6%260.2%79.4%80.1%77.8%69.8%69.8%69.5%83.4%360.8%79.1%79.4%76.6%70.1%70.1%69.8%83.5%460.2%78.5%79.8%76.6%68.5%68.5%69.2%81.6%557.6%78.2%79.1%75.0%68.5%68.5%70.8%81.9%CricketZ161.2%79.4%80.1%78.2%73.3%73.3%69.5%82.5%260.2%80.4%79.4%77.2%73.0%73.0%70.1%81.4%360.2%81.0%80.4%79.4%74.3%74.3%69.8%83.3%460.5%79.1%80.4%78.2%73.7%73.7%70.5%82.6%561.8%80.1%81.7%79.4%75.3%75.3%70.8%82.5%


### 4.2.2. Processing Time Results

The experiment of processing time is implemented on an ordinary personal computer with a Intel Core i7-9750H CPU at 2.60 GHz and 8 GB RAM. According to the time complexity calculation presented in [41], the time complexity of four traditional algorithms is Ox×2n, Ox×22n, Ox×22n, and Ox2×2n, where x and n represent the number of evidences and elements in the FOD, respectively. The time complexity of Deng and Yan’s algorithms is higher. For all datasets, we take the average processing time of 5 random states for comparison, and the results are illustrated in Table 3. The proposed method spends a lot of time in the training phase, but after training, the processing time of our approach is comparable to that of the other four methods.Table 3
The processing time of different approaches.
DatasetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodTraining time (s)Testing time (s)ED0.271.081.080.67252.330.05UY0.140.590.590.35117.430.02UZ0.140.590.590.35117.680.02CricketX0.020.090.090.0616.760.01CricketY0.020.090.090.0616.590.01CricketZ0.020.090.090.0617.170.01Artificial synthesis0.010.010.010.01235.530.04


## 4.2.1. Classification Results

The classification accuracy on the two types of datasets are shown in Tables1 and 2, respectively. According to experimental results, we can draw the following conclusions: (a) For the first type of dataset, Yan et al.’s method, Han et al.’s method, and our proposed method similarly achieve the highest classification accuracy of 97.6%, while Dempster’s has a classification accuracy of 64.2% and Deng’s method has a classification accuracy of 95.2%. It indicates that the proposed method retains competitive performance. (b) The classification accuracy of the proposed method is always higher compared with a single best-performing neural network classification algorithm, and the average improvement on the second type of datasets is 2.47%, 4.20%, 1.59%, 0.74%, 3.21%, and 1.73%, respectively. It illustrates that the proposed method which combines the predicted results of three single neural networks can obtain better classification results than a single network. In addition, proposed method is to get the final decision by fusing multiple prediction results, which is regarded as the decision level fusion, so it can be flexibly extended to other application fields. (c) For the same dataset, the classification accuracy of the proposed method under five random states has little fluctuation. It proves the robustness of our model. (d) Compared with classic traditional method, two belief entropy-based traditional methods and a distance-based traditional method on the second type of datasets, the classification accuracy of the proposed method is the highest, and the total average improvement is 4.06%, 10.11%, 10.11%, and 16.19%, respectively. These indicate that the determination of weights based on CNN is more reasonable, which proves the validity of proposed method.Table 1
The classification performance comparison on the first type of dataset.
DatasetDempster [17]Deng[30]Yan and Deng [33]Han et al. [35]Proposed methodArtificial synthesis64.2%95.2%97.6%97.6%97.6%Table 2
The classification performance comparison on the second type of datasets.
DatasetRandom stateMLPFCNResNetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodED159.1%69.2%74.1%71.4%61.6%61.6%37.9%76.5%259.2%68.9%74.2%71.6%61.8%61.8%37.3%76.4%359.6%69.6%74.0%71.9%62.0%62.0%37.7%77.1%459.8%69.3%74.2%71.6%62.3%62.3%37.2%76.7%559.3%69.4%74.3%71.4%62.2%62.2%37.1%76.4%UY168.1%60.0%64.1%71.9%62.8%62.8%70.4%72.4%269.2%59.9%64.6%72.0%62.6%62.6%71.4%73.5%368.0%59.6%64.4%71.3%62.5%62.5%71.1%72.3%469.1%60.3%65.0%72.5%63.3%63.3%71.4%73.7%569.2%59.9%64.4%72.5%63.1%63.1%71.2%72.7%UZ168.3%69.6%75.8%76.6%70.1%70.1%64.5%77.4%269.5%69.4%75.8%76.9%70.5%70.5%64.4%77.5%368.8%69.3%75.8%76.6%70.2%70.2%64.0%77.7%469.1%69.9%75.5%76.4%70.5%70.5%64.2%76.5%568.4%69.4%75.2%76.3%70.4%70.4%64.2%76.9%CricketX156.0%79.1%78.2%72.1%74.0%74.0%63.4%80.4%254.8%79.8%78.5%73.0%72.7%72.7%63.4%80.4%358.9%80.7%78.8%73.7%74.3%74.3%62.1%80.8%457.0%81.0%79.4%74.3%73.7%73.7%62.1%82.1%558.9%80.7%80.1%73.0%74.6%74.6%63.1%81.4%CricketY158.6%78.2%78.5%76.9%68.9%68.9%70.1%82.6%260.2%79.4%80.1%77.8%69.8%69.8%69.5%83.4%360.8%79.1%79.4%76.6%70.1%70.1%69.8%83.5%460.2%78.5%79.8%76.6%68.5%68.5%69.2%81.6%557.6%78.2%79.1%75.0%68.5%68.5%70.8%81.9%CricketZ161.2%79.4%80.1%78.2%73.3%73.3%69.5%82.5%260.2%80.4%79.4%77.2%73.0%73.0%70.1%81.4%360.2%81.0%80.4%79.4%74.3%74.3%69.8%83.3%460.5%79.1%80.4%78.2%73.7%73.7%70.5%82.6%561.8%80.1%81.7%79.4%75.3%75.3%70.8%82.5%


## 4.2.2. Processing Time Results

The experiment of processing time is implemented on an ordinary personal computer with a Intel Core i7-9750H CPU at 2.60 GHz and 8 GB RAM. According to the time complexity calculation presented in [41], the time complexity of four traditional algorithms is Ox×2n, Ox×22n, Ox×22n, and Ox2×2n, where x and n represent the number of evidences and elements in the FOD, respectively. The time complexity of Deng and Yan’s algorithms is higher. For all datasets, we take the average processing time of 5 random states for comparison, and the results are illustrated in Table 3. The proposed method spends a lot of time in the training phase, but after training, the processing time of our approach is comparable to that of the other four methods.Table 3
The processing time of different approaches.
DatasetDempster [17]Deng [30]Yan and Deng [33]Han et al. [35]Proposed methodTraining time (s)Testing time (s)ED0.271.081.080.67252.330.05UY0.140.590.590.35117.430.02UZ0.140.590.590.35117.680.02CricketX0.020.090.090.0616.760.01CricketY0.020.090.090.0616.590.01CricketZ0.020.090.090.0617.170.01Artificial synthesis0.010.010.010.01235.530.04


## 5. Conclusion

In this paper, by introducing deep learning into conflicting evidence combination, we propose a comprehensive method for determining the evidence weights based on a CNN. Taking the evidence as the network input and the corresponding weight as the output, it utilizes CNN to fully mine potentially useful information that affects the evidence weights, in order to determine the weights comprehensively. Besides, the weights are continuously optimized through back propagation and achieve the optimal when the weight loss function value satisfies the minimum. The classification experimental results show that the proposed method makes the determination of weights more reasonable and obtains a higher classification accuracy compared with traditional ones based on evidence distance or entropy and can be flexibly extended to other application fields as a decision-making fusion method.



---

*Source: 1616401-2022-09-29.xml*