# Evaluation for Sortie Generation Capacity of the Carrier Aircraft Based on the Variable Structure RBF Neural Network with the Fast Learning Rate

Authors: Tiantian Luan; Mingxiao Sun; Guoqing Xia; Daidai Chen
Journal: Complexity
 (2018)

---

## Abstract

The neural network has the advantages of self-learning, self-adaptation, and fault tolerance. It can establish a qualitative and quantitative evaluation model which is closer to human thought patterns. However, the structure and the convergence rate of the radial basis function (RBF) neural network need to be improved. This paper proposes a new variable structure radial basis function (VS-RBF) with a fast learning rate, in order to solve the problem of structural optimization design and parameter learning algorithm for the radial basis function neural network. The number of neurons in the hidden layer is adjusted by calculating the output information of neurons in the hidden layer and the multi-information between neurons in the hidden layer and output layer. This method effectively solves the problem that the RBF neural network structure is too large or too small. The convergence rate of the RBF neural network is improved by using the robust regression algorithm and the fast learning rate algorithm. At the same time, the convergence analysis of the VS-RBF neural network is given to ensure the stability of the RBF neural network. Compared with other self-organizing RBF neural networks (self-organizing RBF (SORBF) and rough RBF neural networks (RS-RBF)), VS-RBF has a more compact structure, faster dynamic response speed, and better generalization ability. The simulations of approximating a typical nonlinear function, identifying UCI datasets, and evaluating sortie generation capacity of an carrier aircraft show the effectiveness of VS-RBF.


---

## Body

## 1. Introduction

A carrier aircraft is an important part in the modern naval warfare. The research on the warfare capacity of the carrier aircraft has become a hot issue with the increasing attention of the security in the territorial sea. The comparison of sortie generation capacity of the carrier aircraft in different operational schemes is helpful to determine the final plan. Therefore, the evaluation for sortie generation capacity of the carrier aircraft has important theoretical significance and application value [1].The evaluation for sortie generation capacity of the carrier aircraft is complex, due to the mutual influence and complex nonlinear of factors. The research of evaluation for the sortie generation capacity of the carrier aircraft has been studied recently. Xia et al. [2, 3] applied the principal component reduction method and the nonlinear fuzzy matter-element method to evaluate sortie generation capacity of the carrier aircraft. Both methods did not consider the mutual influence of factors. Gilchrist [4] proposed an evaluation method of the suitability of LCOM for modeling. This report studied the base-level munition production process in LCOM. However, the common evaluation methods ignore the correlation between the influencing factors. There is a certain deviation between the evaluation results and the actual situation.The neural network has the advantages of self-learning, self-adaptation, and fault tolerance. It can establish a qualitative and quantitative evaluation model which is closer to human thought patterns. The trained neural network can connect expert evaluation ideas with the neural network. Thus, this neural network can not only simulate the expert evaluation but also avoid the human errors in the evaluation process and the subjective influence of the human to calculate weight. The evaluation method based on the RBF neural network has advantages of fast calculation speed, high efficiency of problem solving, and strong self-learning ability. However, there are two problems in the application of the radial basis function (RBF) neural network.The first problem is the structural design problem of the RBF neural network. In recent years, many optimization methods of the RBF neural network have been put forward.(1)
The pruning algorithm [5] was regarded as an effective way to optimize the network structure and improve the generalization ability of the network, but the parameters set in the pruning method required experience and skills
(2)
The growing algorithm [6] increased the number of neurons and connections until the generalization ability met the requirements, but it was difficult to determine when to stop growing
(3)
The pruning and growing algorithm: Kokkinos and Margaritis [7] present a Hierarchical Markovian Radial Basis Function Neural Network (HiMarkovRBFNN) model that enabled recursive operations. The hierarchical structure of this network was composed of recursively nested RBF neural networks with arbitrary levels of hierarchy. All hidden neurons in the hierarchy levels were composed of truly RBF neural networks with two weight matrices. The hidden RBF response units were recursive. However, this method was affected by the initial value, and sometimes, the final RBF neural network was unstable. However, the algorithm ignored the adjustment of structural parameters, which led to the slow convergence speed of the neural network learning algorithm. Therefore, the RBF neural network structure optimization design method is still an open problem, and especially the convergence of the dynamic structure adjustment process has not been solved wellThe second problem is the training method for the weights of the RBF neural network and the learning rate. At present, RBF neural network weights are usually trained by the linear least square algorithm, but the least square estimation is affected by outliers [8, 9]. Kadalbajoo et al. [10] presented an RBF-based implicit explicit numerical method to solve the partial integro-differential equation which described the nature of the option price under the jump diffusion model. But the sum of squared errors increased rapidly with the increase of squared error of each training sample. At the same time, there is a problem to set the learning rate of the RBF neural network [11–14]. In the use of the RBF neural network, the learning rate was often subjective to set as a fixed value [15–20]. It remained unchanged throughout the learning process. If the learning rate was set too high, the convergence speed of the network might be very fast, and it might cause network instability. If the learning rate was too small, it would cause that the network convergence speed was slow and consume a large amount of computing time. Therefore, it is very difficult to choose a suitable learning rate for the traditional RBF neural network.In order to solve the problems above, this paper proposes a variable structure RBF neural network (VS-RBF) with a fast learning rate. The number of neurons in the hidden layer is adjusted by calculating the output information (OI) of neurons in the hidden layer and the multi-information (MI) between neurons in the hidden layer and output layer. The robust regression method is used to replace the linear least square algorithm to reduce the influence of outliers on weight training. Then, the fast learning rate method is used to adjust the learning rate of the RBF neural network, which can guarantee the stable learning of the network and the convergence speed. In this paper, the proposed VS-RBF neural network can be used to grow or prune the neurons in the hidden layer according to the actual system.The rest of this paper is organized as follows. Section2 describes the VS-RBF neural network with a fast learning rate. Section 3 proves convergence of VS-RBF. In Section 4, VS-RBF is compared with SORBF and RS-RBF in approximating a typical nonlinear function and identifying UCI datasets. In Section 5, VS-RBF is used to evaluate sortie generation capacity of the carrier aircraft. Finally, conclusions are presented in Section 6.


## 2. Sortie Generation Capacity of Carrier Aircrafts

At present, the use of carrier aircrafts for domestic experience is very little. In theory, the index system of sortie generation capacity is established according to the foreign research results. The index of the sortie generation capability system established in this paper is based on the main factors pointed out in the 1997 “Nimitz” carrier aircraft exercise report. The factors are taken from the real environment and have high reference value. During the four-day exercise period, the equipment and environment did not affect the sorties. So the selection of indexes did not include the corresponding equipment and environment indexes [2, 3].In this paper, when the index system of sortie generation capability is established, on one hand, we hope that the factors will be more comprehensive. Thus the credibility of the evaluation results will be increased. On the other hand, considering that if all the possible factors are added to the index system, the modeling difficulty will be greatly increased, and the evaluation system will also be proofread. So we should select the evaluation index according to the following principles:(1)
Considering the hierarchy and correlation between the evaluation indexes, if the high-level indexes can be obtained, the underlying indexes are not considered repeatedly. For example, one of the lowest levels is the carrier deck design, but the impact of the carrier deck design on the sortie generation capability is derived from the redeparture preparation time, ejection interval time, and recovery interval time. So the corresponding high-level indexes will not be included in the evaluation system
(2)
Grasping the main indexes makes the evaluation system easy to understand and operate. Environmental factors have little impact on most operations, and they are ignored
(3)
The main purpose of this paper is to carry out quantitative evaluation. The indexes which are difficult to quantify, such as the quality of the environment and the ability of personnel, are not used to establish the evaluation system in this paperIn summary, the index system for sortie generation capacity of carrier aircrafts is established with related research results [2]. A three-level index system with complexity, hierarchy, contradiction, and relevance is established by the recursive hierarchy method. The index system for sortie generation capacity of carrier aircrafts is shown in Figure 1.Figure 1
Index system for sortie generation of carrier aircrafts.These indexes are defined as follow:(1)
Emergency sortie generation rate (ESGR): the maximum number of ready aircrafts taking off in a few minutes
(2)
Surge sortie generation rate (SSGR): the average number of aircrafts per day in the surge operation (4 days)
(3)
Last sortie generation rate (LSGR): the average number of aircrafts per day in the continuous operation (30 days)
(4)
Performing tasks proportion (PTP): the time proportion that the aircrafts can carry out one task at least under a certain flight plan and logistics condition
(5)
Missing tasks proportion waiting for parts (MTPWP): the proportion of aircrafts missing the tasks due to waiting for parts
(6)
Missing tasks proportion waiting for repair (MTPWR): the proportion of aircrafts missing the tasks due to waiting for repair
(7)
Scheduled completion proportion (SCP): the proportion of the completed number in the planned number of aircrafts
(8)
Pilot utilization rate (PUR): the average utilization rate of the pilots per day
(9)
Plan implementation probability per aircraft (PIPA): the plan implementation probability per aircraft under the certain constraints in a given period of time
(10)
Sortie generation rate per aircraft (SGRA): the sortie generation rate per aircraft under the certain constraints
(11)
Preparation time for next sortie (PTNS): the preparation time for next sortie under the condition of a certain resource allocation
(12)
Ejection interval (EI): the average time for ejecting a single aircraft per catapult
(13)
Take-off outage proportion (TOOP): the proportion of the cancelled number in the ready number of aircrafts
(14)
Recovery interval (RI): the average time for recovering a single aircraft
(15)
Overshoot proportion (OP): the proportion of the number of aircrafts failed to recover in the number of aircrafts ready to recoverIn the three-level recursive hierarchical graph of Figure1, there are interactions between the underlying indexes:
(1)
In practice, the surge sortie generation rate and last sortie generation rate are contradictory, and they cannot reach the optimal value at the same time
(2)
The preparation time for next sortie and the ejection interval constitute one wave duration. If the preparation time is sufficient, the time spent on ejection and recovery will be reduced. And the sortie generation capability will be reduced. If the time spent on ejection and recovery is sufficient, the preparation time may not be able to meet the carrier aircraft support operations, resulting in the reduction of the available carrier aircraft and the reduction of sortie generation capabilityTherefore, there are correlations and contradictions among the indexes of sortie generation capability. The evaluation of sortie generation capability shows complex nonlinear relationship. Using the nonlinear mapping ability of the neural network to evaluate the complex nonlinear sortie generation capability can avoid the subjectivity of traditional evaluation methods and the complexity of the evaluation process.Some references used different methods to evaluate sortie generation capacity of carrier aircrafts. Xie et al. [21] investigated the complicated relation between the sortie generation and aviation maintenance of the Nimitz class carrier aircraft. It was observed that the state transition diagram was both effective and efficient for system analysis. Liu et al. [22] proposed that sortie generation was one of the critical indexes which were used to characterize carrier and air wing capabilities. In order to research the index system of sortie generation capacity and their effects on the embarked air wings, attention was drawn to the analysis on the basic concept of sortie generation rate (SGR) and a range of constraints for launch and recovery, followed by a review of the definition of the SGR index system that commonly applies in the assessment of various types of carrier aircrafts commissioned in the foreign navies (e.g., the USN, the RN, the French Navy, and the Russian Navy). Analyses were also performed on the SGR index system used for these typical carriers. The conclusion was that the establishment of an effective index system and setting for the relevant indexes must be accomplished by actual operations and combat exercises. Zhou et al. [23] considered that operational capability of the flight deck was the critical factor to affect the sortie generation of a carrier-based aircraft, including launch operation, recovery and repot operations, and serving. The definition of an optimized flight deck operation plan was given. A method to calculate the number of sortie generation in the optimized flight deck operation plan was proposed. Some factors including aircraft number, launch time, repotted time, recovery time, and how they affect the sortie generation were analysed. The results showed that promoting the capability of the flight deck was the key factor to increase the number of sorties. Wang and Yan [24] summarized three evaluation methods of SGR of embarked aircrafts according to the relevant exercise data and papers released by the US Navy. And the characteristics of them were analysed. The advantages of statistical analysis were that the accuracy and credibility of the data were high. The disadvantage was that it can only evaluate the carrier aircraft in service. The cost was high, and the cycle was long. The empirical formula method was based on the recovery data of different carrier aircrafts at different times. It could quickly predict the capacity of the carrier aircraft under typical combat tasks. But there must be a large number of the actual operational data of the carrier aircraft. And the errors of evaluation results were large. The experimental method is based on the operation process of the carrier aircraft. And the computer simulation method was used. It was characterized by wide applicability. The evaluation was of high accuracy, less cost, and short cycle. Zhang et al. [25] established a system with models based on parameters to analyse effectiveness about major factors of the swarming aircraft. The AHP method was used to calculate the weight of each parameter. The fuzzy synthetic evaluation method was applied to access the operational performance. The result demonstrated that the method was feasible to deliver a scientific evaluation and presented a new perspective to judge the efficiency of swarming aircrafts. References [26–28] used the AHP method to evaluate antiship combat capability of the carrier aircraft, the threat the of carrier-borne aircraft, and effectiveness of the carrier-based aircraft in air defence.


## 3. VS-RBF Neural Network with the Fast Learning Rate

The robust regression method is used to replace the linear least square algorithm to reduce the influence of outliers on weight training. Then, the fast learning rate method is used to adjust the learning rate of RBF neural network, which can guarantee the stable learning of the network and the convergence speed. Finally, the proposed VS-RBF has better nonlinear approximation ability, faster training speed, and a more compact network structure compared with SORBF and RS-RBF.


### 3.1. Structure of the RBF Neural Network

The structure of the RBF neural network is similar to that of the multilayer forward network. There are three layers: input layer, hidden layer, and output layer. The topology of the RBF neural network is shown in Figure2.Figure 2
Topology structure of the RBF neural network.Therefore, the outputy can be described as (1) for multi-input and single-output RBF neural networks,
(1)y=∑i=1mwiϕiX=∑i=1mwie−X−Ci2/2δi2,where X=x1,x2,⋯,xnT∈Rn is the input vector; n is the node number in the input layer; W=w1,w2,⋯,wmT∈Rm is the weight of the output layer; m is the neuron number in the hidden layer; Φ=ϕ1,ϕ2,⋯,ϕmT∈Rm is radial basis function in the hidden layer, which is the Gauss function, namely, ϕiX=e−X−Ci2/2δi2; δi is the expansion constant of radial basis function; ∗ is the Euclid norm; Ci is the data center of ith hidden nodes; and y is the output of the RBF network. i=1,2,…,m. In Figure 2, the output information (OI) is ϕiX=e−X−Ci2/2δi2, and the multi-information (MI) is the intensity of information between Ci and y.In the human cerebral cortex, the local accommodation and overlapping receptive fields are the characteristics of human brain response.Φ has a strong response in a part of the Ci surrounding area, which reflects the characteristics of the cerebral cortex response. In this paper, the OI intensity of neurons in the hidden layer and the MI intensity between neurons are used to adjust the neurons in the hidden layer. Thus, the topology structure of the neural network is modified.There are many methods to change the RBF structure. Zheng et al. [29] proposed a meshfree or meshless local RBF collocation method to calculate the band structures of two-dimensional antiplane transverse elastic waves in phononic crystals. Three new techniques were developed for calculating the normal derivative of the field quantity required by the treatment of the boundary conditions, which improved the stability of the local RBF collocation method significantly. The pruning was at the end of the training rather than in the learning process. Therefore, it would be limited in application. Sarimveis et al. [30] presented a new method for extracting the valuable process information from input/output data. The proposed methodology produced dynamical RBF neural network models based on a specially designed genetic algorithm (GA), which was used to autoconfigure the structure of the networks and obtain the model parameters. But this algorithm was a global optimization algorithm, which took longer time in the training process. Han et al. [31] presented a flexible structure RBF neural network (FS-RBFNN). The FS-RBFNN could vary its structure dynamically in order to maintain the prediction accuracy. The hidden neurons in the RBF neural network could be added or removed online based on the neuron activity and mutual information, to achieve the appropriate network complexity and maintain overall computational efficiency. However, the algorithm ignored the implicit relationship between neurons, which might cause an overfitting phenomenon. Wu et al. [32] adopted the cloud RBF neural network as the function approximation structure of approximate dynamic programming, and it had the advantage of the fuzziness and randomness of the cloud model. But the cloud method was a global search algorithm, which would reduce the overall learning speed. Fu and Wang [33] proposed a novel separability-correlation measure (SCM) document to rank the importance of attributes. According to the attribute ranking results, different attribute subsets were used as inputs to a classifier, such as an RBF neural network. Those attributes that increase the validation error were deemed irrelevant and were deleted. The complexity of the classifier could thus be reduced, and its classification performance improved. But the initial values were set with the global sample data, which were hard to obtain in practical applications. Peng et al. [34] proposed a novel hybrid forward algorithm (HFA) for the construction of RBF networks with tunable nodes. The set neural main objective was to efficiently and effectively produce a parsimonious RBF neural network that generalizes well. It was achieved through simultaneous network structure determination and parameter optimization on the continuous parameter space. However, the parameters of this method were too complex. Han et al. [35] proposed a new growing and pruning algorithm for RBF neural network structure design, which was named as self-organizing RBF (SORBF). The growing and pruning algorithm was used to design the structure of the RBF neural network automatically. But the overall training time of SORBF was too long. Ding et al. [36] combined rough set theory with neural network (RS-RBF). The model overcame the shortcoming that when neural network inputs too many dimensions, the structure of the network was too big. In order to solve the problems above, this paper proposes a variable structure RBF neural network (VS-RBF) with a fast learning rate. The number of neurons in the hidden layer is adjusted by calculating the output information (OI) of neurons in the hidden layer and the multi-information (MI) between neurons in the hidden layer and output layer. The convergence of the final network in the structural adjustment process is proved. In this paper, the proposed VS-RBF neural network can be used to grow or prune the neurons in the hidden layer according to the actual system.


### 3.2. VS-RBF Neural Network

VS-RBF is used to adjust its structure based on information intensity. Firstly, the activity of the neurons in the hidden layer is determined by OI intensity of neurons in the hidden layer. And the neurons with strong activity are decomposed. Secondly, the connection strength between the neurons in the hidden layer and in output layers is analyzed by calculating MI intensity between the neurons. Then the network structure is modified according to MI intensity. Finally, the parameters of the neural network are adjusted. Therefore, the structure of the RBF neural network can be divided into two parts: the decomposition of the neurons in the hidden layer and the interconnection adjustment between the neurons in the hidden layer and in the output layer.


#### 3.2.1. Decomposition of Neurons in the Hidden Layer

The activity of the neuron denotes the information capacity of the neuron. When the neuron provides higher activities, it means that the neuron includes more information. In order to increase the accuracy of RBF, the information should be evenly distributed in neurons. Thus, if the neuron provides higher activities, it should be decomposed to higher number of neurons.The activity of neurons in the hidden layer is calculated by(2)AiX=ϕi∑j=1mϕjX−Ci+α,where i,j=1,2,…,m, AiX is the activity of the ith neuron in the hidden layer, m is the number of neurons in the hidden layer, ϕi is the output of the ith neuron in the hidden layer, ϕj is the output of the jth neuron in the hidden layer, and α is a small real number. The activity of neurons in the hidden layer is inversely proportional to the Euclid distance between X and Ci. The closer the distance is, the higher the activity value is. The activity of neurons AiX is obtained from the output of the hidden layer, which denotes the need of decomposition.When activityAiX of the neuron in the hidden layer is strong, the neuron is decomposed, and the neuron is decomposed to l new neurons. The main idea behind this decomposition is to evenly distribute information in neurons in order to increase the accuracy of VS-RBF. When the activity of the ith neuron in the hidden layer is greater than the threshold of activity A0=max100ed,1/m (ed is the expected error), the connection between the ith neuron and the output neuron is broken down as shown in Figure 3.Figure 3
Decomposition of the neuron in the hidden layer.In Figure3, if the activity between the ith neuron and the output neuron y is greater than the threshold of activity A0, the connection between the ith neuron and the output neuron y is broken down. After decomposition, the ith neuron is decomposed to l new neurons, and each of l new neurons connects to the output neuron y. l=integerAi/∑i=1mAiX/m.After the connection between theith neuron and the output neuron is broken down, there will be l new neurons connecting to the output neuron. The initial center and variance of the new neurons are Cij and δij, which can be obtained from
(3)Cij=λiCi+μiX,(4)δij=λiδi,where 0.9≤λi≤1.1, 0≤μi≤0.2, and Ci and δi are the center and variance of the ith neuron in the hidden layer, respectively, and Cij and δij are the center and variance of the jth neuron decomposed from the ith neuron. j=1,2,…,l. l is the number of the new neurons. l is the integer part of Ai/AV. AV=∑i=1mAiX/m.The weightwij between the new neuron and the output neuron is
(5)wij=βjwiϕiX−eϕijX,where βj is the decomposition parameter of the jth neuron; ∑j=1lβj=1, βj=1/l; ϕijX is the output of the new neuron; and e is the output error of the neural network before decomposition.


#### 3.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

The MI functionIa;y of neuron a in the hidden layer and neuron y in the output layer is obtained from (6). Ia;y depends on the intensity of information between a and y,
(6)Ia;y=∑a,yfa;ylog2fa;yfafy,where fa;y is the joint distribution density function of a and y. fa;y=fafy∣a, where fa and fy are the probability densities of a and y, respectively. According to the Shannon entropy theory, fa;y, fa, and fy are not calculated in fact, and the entropies are calculated instead.Assuming thata and y are interconnected neurons, Ia;y depends on the intensity of information between a and y.According to the Shannon entropy theory, the connection strengthIa;y between a and y can be calculated by
(7)Ia;y=ENa−ENay=ENy−ENya,where ENa is the entropy of a and ENa∣y is the entropy of a in the condition of y. When a and y are independent, Ia;y=0. Otherwise, Ia;y>0. Therefore, Ia;y≥0. The range of Ia;y is shown in
(8)Ia;y≤minENa,ENy.Then, based on (8), the normalized MI is obtained from
(9)Isa;y=Ia;y/minENa,ENy,where Isa;y∈0,1. The calculation of Is is able to determine the interaction strength between a and y.In the RBF neural network, whenIs is large, the interaction strength between a and y is strong and the connection of a and y exists. When Is approaches 0, it indicates that the interaction strength between a and y is weak. Then the connection of a and y can be deleted in the structure adjustment. It can reduce the redundancy of the neural network.In Figure4, when Is≤I0 (I0=0.01ed), the connection between a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The new neuron b contains neurons a and b. Thus, a new layer is not added to the network, and there is no weight between neurons a and b. The parameters of neuron b are adjusted from (10)–(12). Cb′ is the same as Ca. δb′ is the same as δa:
(10)Cb′=Ca,(11)δb′=δa,(12)wb′=wb+wa⋅ϕaXϕbX,where Cb′, δb′, and wb′ are the center, variance, and weight between b and y in the new output layer, respectively.Figure 4
Disconnection of the neuron in the hidden layer and the neuron in output layers.VS-RBF can not only increase the number of neurons in the hidden layer but also remove redundant neurons. The optimal number of RBF neurons is obtained by adjusting the output information of neurons in the hidden layer and the multi-information between neurons in the hidden layer and output layer. When the error of output meets the demand, the optimal number of RBF neurons is obtained.


### 3.3. Robust Regression Training Method for Weights in the Output Layer

At present, RBF neural network weights are usually trained by the linear least square algorithm, but the least square estimation is affected by outliers. In order to reduce the influence of outliers, the robust regression method is applied to train weights in the output layer.The output of the RBF network is the weighted sum of the output of the hidden layer. The aim of the training is to make the sum of squared errorEW between the output of the whole network and the actual output least. The sum of squared error EW is obtained from
(13)EW=min∑t=1Nyt−ΦtTW2=min∑t=1Nσt2,where yt is the value of actual output; ΦtT is the output sample vector of the hidden layer; Wt is the weight vector; N is the number of samples; and t=1,2,…,N.In order to reduce the influence of outliers, we hope to find a functionθσt, which increases with the increase of σt and the growth rate is slower than σt2. Then the sum of squared error EW can be expressed as
(14)EW=min∑t=1Nθσt.θσt is an Andrews function, which can be expressed as (15). And the derivative of θσt can be expressed as
(15)θσt=K2−K2cosσtK,σt≥πK,2K2,σt<πK,(16)θ′σt=KsinσtK,σt≥πK,0,σt<πK,where θ′σt is the derivative of θσt and K is a threshold constant.Therefore, robust regression can be written in recursive form, which is expressed in(17)Wt=Wt−1−η∂θσt∂W,where W is the estimated weight and η is the training coefficient, which can be determined according to the actual situation. Then Wt is transferred to
(18)Wt=Wt−1−η∂θσt∂σt⋅∂σt∂W=Wt−1−ηθ′σt⋅−ΦtT=Wt−1+ηθ′yt−ΦtTWtΦtT.It can reduce the influence of outliers on neural network training by using the robust regression method.


### 3.4. Fast Learning Rate

In the use of the RBF neural network, the learning rate was often subjective to set as a fixed value. It remained unchanged throughout the learning process. If the learning rate was set too high, the convergence speed of the network might be very fast and it might cause network instability. If the learning rate was too small, it would cause that the network convergence speed was slow and consume a large amount of computing time. Therefore, it is very difficult to choose a suitable learning rate for the traditional RBF neural network. In order to solve this problem, this paper proposes a new fast learning rate. The fast learning rate is suitable for each step of the iteration, which can ensure the stability of the network. At the same time, the convergence speed of the network and the efficiency of the network can be improved.Set(19)Φ=ϕ11ϕ12⋯ϕ1mϕ21ϕ22⋯ϕ2m⋮⋮⋱⋮ϕN1ϕN2⋯ϕNm.N is the number of samples. m is the number of nodes in the hidden layer. y^ is the output calculated by the network. Then the cost function Et of the tth train can be defined in
(20)Et=12∑j=1Nyjt−y^jt2=12∑j=1Nej2t.The output error iset=e1t,e2t,⋯,eNtT. ΔWt=Wt−Wt−1 is the change of the weight in the tth train. According to (18), ΔWt is obtained in
(21)ΔWt=ηtθ′yt−ΦtTWtΦtT.The increase of error can be expressed as(22)Δet=et−et−1=yt−y^t−yt−1+y^t−1.Δyt=yt−yt−1 is the change of the actual output. Δy^t=y^t−y^t−1 is the change of the network output. In general, the absolute value of the change of the actual output is far less than the absolute value of the change of the network output. Δyt≪Δy^t. The change of the actual output is negligible compared with the change of the network output, because the actual output is often constrained by many conditions, and the network output will not be limited. This assumption has practical significance. Thus, (22) can be approximated as
(23)Δet=et−et−1≈−Δy^t.Considering (21), the change of error Δet is expressed in
(24)Δet≈−Δy^t=−ΦΔWt=−ηtΦθ′y−ΦTWtΦT.Then,et is expressed in (25) considering (24):
(25)et=et−1+Δet≈et−1−ηtΦθ′y−ΦTWtΦT.The cost functionEt of the tth train can be obtained from (25). Et is expressed in
(26)Et=12eTtet≈12et−1−ηtΦθ′y−ΦTWtΦTT⋅et−1−ηtΦθ′y−ΦTWtΦT.This cost function can be considered as a function of the learning rateηt. The optimal value of learning rate ηt can be obtained by minimizing Et. The first order condition of (26) is expressed in
(27)∂Et∂ηtηt=η∗t=−12Φθ′y−ΦTWtΦTT⋅et−1−ηtΦθ′y−ΦTWtΦT−12et−1−ηtΦθ′y−ΦTWtΦTT⋅Φθ′y−ΦTWtΦT=0.The second order condition of (26) is expressed in
(28)∂2Et∂η2tηt=η∗t=12Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT+12Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT=Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT>0.The second order condition (28) holds as Φ is positive definite. And the fast learning rate η∗t can be obtained by (27). η∗t is expressed in
(29)η∗t=12Φθ′y−ΦTWtΦTTet−1Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT+12eTt−1Φθ′y−ΦTWtΦTΦθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT.


### 3.5. Learning Algorithm of VS-RBF with the Fast Learning Rate

VS-RBF with the fast learning rate can not only increase the number of neurons in the hidden layer but also remove redundant neurons. At the same time, the use of the robust regression algorithm can avoid the influence of the abnormal sample on the neural network. And the fast learning rate can ensure that the learning rate is appropriate in the iterative process. In this paper, the steps of the learning algorithm are shown in Figure5.
Step 1.
For a given RBF neural network, the number of neurons in the hidden layer is a small natural number. Considering that VS-RBF can adjust the number of neurons in the hidden layer, the number of initial neurons in the hidden layer is defined as the neurons in the input layer or a little more. In this paper, we consider that the number of initial neurons in the hidden layer is equal to the neurons in the input layer. The neural network centerC and variance δ are selected by using the least square method. Φ is calculated. The learning accuracy ed and the maximum number of iterations maxT are set. The number of neurons in the input layer is the number of input. The number of neurons in the output layer is the number of output. A0=max100ed,1/m, I0=0.01ed
Step 2.
The initial weights between the hidden layer and the output layerW0 are set. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer is calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedFigure 5
Learning algorithm of VS-RBF with the fast learning rate.VS-RBF with fast learning rate can realize the self-organization of the structure and determine whether to grow or prune the neurons in the hidden layer by calculating the information intensity. A new RBF neural network structure design method is proposed. It not only can adjust the weights of neural network online but also can grow or prune the neurons in the hidden layer. From the biological point of view, this neural network structure is more similar to the mechanism of human brain neuron information processing.


### 3.6. Comparison of VS-RBF, SORBF, and RS-RBF

The advantages, main novelties, and disadvantages of SORBF [35] and RS-RBF [36] are as follows:
(1)
SORBF: neither the number of nodes in the hidden layer nor the parameters need to be predefined and fixed. They are adjusted automatically in the learning process by comparingvx−Pk=minx=1,2,…,Nvx−Pk and βrx. This type of SORBF-based approach offers a promisingly inexpensive approach to real-time measurement of variables that have typically proved difficult to measure reliably using hardware. However, this algorithm ignores the adjustment of structural parameters, which leads to the slow convergence speed of the neural network learning algorithm
(2)
RS-RBF: by processing multiple nodes of the network at one time, multiple hidden nodes can be cut off, and the core nodes in the hidden nodes can be found by calculating the output of the network asDn=outputn−OUTPUT/OUTPUT. Adaptive principle is introduced to make the segmentation change with pruning. The memory function is to remember the most important nodes for each pruning, and the most important nodes will be not deleted in the subsequent pruning even if the output of these nodes is small. However, this method was affected by the initial value, and sometimes, the final RBF neural network was unstable


## 3.1. Structure of the RBF Neural Network

The structure of the RBF neural network is similar to that of the multilayer forward network. There are three layers: input layer, hidden layer, and output layer. The topology of the RBF neural network is shown in Figure2.Figure 2
Topology structure of the RBF neural network.Therefore, the outputy can be described as (1) for multi-input and single-output RBF neural networks,
(1)y=∑i=1mwiϕiX=∑i=1mwie−X−Ci2/2δi2,where X=x1,x2,⋯,xnT∈Rn is the input vector; n is the node number in the input layer; W=w1,w2,⋯,wmT∈Rm is the weight of the output layer; m is the neuron number in the hidden layer; Φ=ϕ1,ϕ2,⋯,ϕmT∈Rm is radial basis function in the hidden layer, which is the Gauss function, namely, ϕiX=e−X−Ci2/2δi2; δi is the expansion constant of radial basis function; ∗ is the Euclid norm; Ci is the data center of ith hidden nodes; and y is the output of the RBF network. i=1,2,…,m. In Figure 2, the output information (OI) is ϕiX=e−X−Ci2/2δi2, and the multi-information (MI) is the intensity of information between Ci and y.In the human cerebral cortex, the local accommodation and overlapping receptive fields are the characteristics of human brain response.Φ has a strong response in a part of the Ci surrounding area, which reflects the characteristics of the cerebral cortex response. In this paper, the OI intensity of neurons in the hidden layer and the MI intensity between neurons are used to adjust the neurons in the hidden layer. Thus, the topology structure of the neural network is modified.There are many methods to change the RBF structure. Zheng et al. [29] proposed a meshfree or meshless local RBF collocation method to calculate the band structures of two-dimensional antiplane transverse elastic waves in phononic crystals. Three new techniques were developed for calculating the normal derivative of the field quantity required by the treatment of the boundary conditions, which improved the stability of the local RBF collocation method significantly. The pruning was at the end of the training rather than in the learning process. Therefore, it would be limited in application. Sarimveis et al. [30] presented a new method for extracting the valuable process information from input/output data. The proposed methodology produced dynamical RBF neural network models based on a specially designed genetic algorithm (GA), which was used to autoconfigure the structure of the networks and obtain the model parameters. But this algorithm was a global optimization algorithm, which took longer time in the training process. Han et al. [31] presented a flexible structure RBF neural network (FS-RBFNN). The FS-RBFNN could vary its structure dynamically in order to maintain the prediction accuracy. The hidden neurons in the RBF neural network could be added or removed online based on the neuron activity and mutual information, to achieve the appropriate network complexity and maintain overall computational efficiency. However, the algorithm ignored the implicit relationship between neurons, which might cause an overfitting phenomenon. Wu et al. [32] adopted the cloud RBF neural network as the function approximation structure of approximate dynamic programming, and it had the advantage of the fuzziness and randomness of the cloud model. But the cloud method was a global search algorithm, which would reduce the overall learning speed. Fu and Wang [33] proposed a novel separability-correlation measure (SCM) document to rank the importance of attributes. According to the attribute ranking results, different attribute subsets were used as inputs to a classifier, such as an RBF neural network. Those attributes that increase the validation error were deemed irrelevant and were deleted. The complexity of the classifier could thus be reduced, and its classification performance improved. But the initial values were set with the global sample data, which were hard to obtain in practical applications. Peng et al. [34] proposed a novel hybrid forward algorithm (HFA) for the construction of RBF networks with tunable nodes. The set neural main objective was to efficiently and effectively produce a parsimonious RBF neural network that generalizes well. It was achieved through simultaneous network structure determination and parameter optimization on the continuous parameter space. However, the parameters of this method were too complex. Han et al. [35] proposed a new growing and pruning algorithm for RBF neural network structure design, which was named as self-organizing RBF (SORBF). The growing and pruning algorithm was used to design the structure of the RBF neural network automatically. But the overall training time of SORBF was too long. Ding et al. [36] combined rough set theory with neural network (RS-RBF). The model overcame the shortcoming that when neural network inputs too many dimensions, the structure of the network was too big. In order to solve the problems above, this paper proposes a variable structure RBF neural network (VS-RBF) with a fast learning rate. The number of neurons in the hidden layer is adjusted by calculating the output information (OI) of neurons in the hidden layer and the multi-information (MI) between neurons in the hidden layer and output layer. The convergence of the final network in the structural adjustment process is proved. In this paper, the proposed VS-RBF neural network can be used to grow or prune the neurons in the hidden layer according to the actual system.


## 3.2. VS-RBF Neural Network

VS-RBF is used to adjust its structure based on information intensity. Firstly, the activity of the neurons in the hidden layer is determined by OI intensity of neurons in the hidden layer. And the neurons with strong activity are decomposed. Secondly, the connection strength between the neurons in the hidden layer and in output layers is analyzed by calculating MI intensity between the neurons. Then the network structure is modified according to MI intensity. Finally, the parameters of the neural network are adjusted. Therefore, the structure of the RBF neural network can be divided into two parts: the decomposition of the neurons in the hidden layer and the interconnection adjustment between the neurons in the hidden layer and in the output layer.


### 3.2.1. Decomposition of Neurons in the Hidden Layer

The activity of the neuron denotes the information capacity of the neuron. When the neuron provides higher activities, it means that the neuron includes more information. In order to increase the accuracy of RBF, the information should be evenly distributed in neurons. Thus, if the neuron provides higher activities, it should be decomposed to higher number of neurons.The activity of neurons in the hidden layer is calculated by(2)AiX=ϕi∑j=1mϕjX−Ci+α,where i,j=1,2,…,m, AiX is the activity of the ith neuron in the hidden layer, m is the number of neurons in the hidden layer, ϕi is the output of the ith neuron in the hidden layer, ϕj is the output of the jth neuron in the hidden layer, and α is a small real number. The activity of neurons in the hidden layer is inversely proportional to the Euclid distance between X and Ci. The closer the distance is, the higher the activity value is. The activity of neurons AiX is obtained from the output of the hidden layer, which denotes the need of decomposition.When activityAiX of the neuron in the hidden layer is strong, the neuron is decomposed, and the neuron is decomposed to l new neurons. The main idea behind this decomposition is to evenly distribute information in neurons in order to increase the accuracy of VS-RBF. When the activity of the ith neuron in the hidden layer is greater than the threshold of activity A0=max100ed,1/m (ed is the expected error), the connection between the ith neuron and the output neuron is broken down as shown in Figure 3.Figure 3
Decomposition of the neuron in the hidden layer.In Figure3, if the activity between the ith neuron and the output neuron y is greater than the threshold of activity A0, the connection between the ith neuron and the output neuron y is broken down. After decomposition, the ith neuron is decomposed to l new neurons, and each of l new neurons connects to the output neuron y. l=integerAi/∑i=1mAiX/m.After the connection between theith neuron and the output neuron is broken down, there will be l new neurons connecting to the output neuron. The initial center and variance of the new neurons are Cij and δij, which can be obtained from
(3)Cij=λiCi+μiX,(4)δij=λiδi,where 0.9≤λi≤1.1, 0≤μi≤0.2, and Ci and δi are the center and variance of the ith neuron in the hidden layer, respectively, and Cij and δij are the center and variance of the jth neuron decomposed from the ith neuron. j=1,2,…,l. l is the number of the new neurons. l is the integer part of Ai/AV. AV=∑i=1mAiX/m.The weightwij between the new neuron and the output neuron is
(5)wij=βjwiϕiX−eϕijX,where βj is the decomposition parameter of the jth neuron; ∑j=1lβj=1, βj=1/l; ϕijX is the output of the new neuron; and e is the output error of the neural network before decomposition.


### 3.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

The MI functionIa;y of neuron a in the hidden layer and neuron y in the output layer is obtained from (6). Ia;y depends on the intensity of information between a and y,
(6)Ia;y=∑a,yfa;ylog2fa;yfafy,where fa;y is the joint distribution density function of a and y. fa;y=fafy∣a, where fa and fy are the probability densities of a and y, respectively. According to the Shannon entropy theory, fa;y, fa, and fy are not calculated in fact, and the entropies are calculated instead.Assuming thata and y are interconnected neurons, Ia;y depends on the intensity of information between a and y.According to the Shannon entropy theory, the connection strengthIa;y between a and y can be calculated by
(7)Ia;y=ENa−ENay=ENy−ENya,where ENa is the entropy of a and ENa∣y is the entropy of a in the condition of y. When a and y are independent, Ia;y=0. Otherwise, Ia;y>0. Therefore, Ia;y≥0. The range of Ia;y is shown in
(8)Ia;y≤minENa,ENy.Then, based on (8), the normalized MI is obtained from
(9)Isa;y=Ia;y/minENa,ENy,where Isa;y∈0,1. The calculation of Is is able to determine the interaction strength between a and y.In the RBF neural network, whenIs is large, the interaction strength between a and y is strong and the connection of a and y exists. When Is approaches 0, it indicates that the interaction strength between a and y is weak. Then the connection of a and y can be deleted in the structure adjustment. It can reduce the redundancy of the neural network.In Figure4, when Is≤I0 (I0=0.01ed), the connection between a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The new neuron b contains neurons a and b. Thus, a new layer is not added to the network, and there is no weight between neurons a and b. The parameters of neuron b are adjusted from (10)–(12). Cb′ is the same as Ca. δb′ is the same as δa:
(10)Cb′=Ca,(11)δb′=δa,(12)wb′=wb+wa⋅ϕaXϕbX,where Cb′, δb′, and wb′ are the center, variance, and weight between b and y in the new output layer, respectively.Figure 4
Disconnection of the neuron in the hidden layer and the neuron in output layers.VS-RBF can not only increase the number of neurons in the hidden layer but also remove redundant neurons. The optimal number of RBF neurons is obtained by adjusting the output information of neurons in the hidden layer and the multi-information between neurons in the hidden layer and output layer. When the error of output meets the demand, the optimal number of RBF neurons is obtained.


## 3.2.1. Decomposition of Neurons in the Hidden Layer

The activity of the neuron denotes the information capacity of the neuron. When the neuron provides higher activities, it means that the neuron includes more information. In order to increase the accuracy of RBF, the information should be evenly distributed in neurons. Thus, if the neuron provides higher activities, it should be decomposed to higher number of neurons.The activity of neurons in the hidden layer is calculated by(2)AiX=ϕi∑j=1mϕjX−Ci+α,where i,j=1,2,…,m, AiX is the activity of the ith neuron in the hidden layer, m is the number of neurons in the hidden layer, ϕi is the output of the ith neuron in the hidden layer, ϕj is the output of the jth neuron in the hidden layer, and α is a small real number. The activity of neurons in the hidden layer is inversely proportional to the Euclid distance between X and Ci. The closer the distance is, the higher the activity value is. The activity of neurons AiX is obtained from the output of the hidden layer, which denotes the need of decomposition.When activityAiX of the neuron in the hidden layer is strong, the neuron is decomposed, and the neuron is decomposed to l new neurons. The main idea behind this decomposition is to evenly distribute information in neurons in order to increase the accuracy of VS-RBF. When the activity of the ith neuron in the hidden layer is greater than the threshold of activity A0=max100ed,1/m (ed is the expected error), the connection between the ith neuron and the output neuron is broken down as shown in Figure 3.Figure 3
Decomposition of the neuron in the hidden layer.In Figure3, if the activity between the ith neuron and the output neuron y is greater than the threshold of activity A0, the connection between the ith neuron and the output neuron y is broken down. After decomposition, the ith neuron is decomposed to l new neurons, and each of l new neurons connects to the output neuron y. l=integerAi/∑i=1mAiX/m.After the connection between theith neuron and the output neuron is broken down, there will be l new neurons connecting to the output neuron. The initial center and variance of the new neurons are Cij and δij, which can be obtained from
(3)Cij=λiCi+μiX,(4)δij=λiδi,where 0.9≤λi≤1.1, 0≤μi≤0.2, and Ci and δi are the center and variance of the ith neuron in the hidden layer, respectively, and Cij and δij are the center and variance of the jth neuron decomposed from the ith neuron. j=1,2,…,l. l is the number of the new neurons. l is the integer part of Ai/AV. AV=∑i=1mAiX/m.The weightwij between the new neuron and the output neuron is
(5)wij=βjwiϕiX−eϕijX,where βj is the decomposition parameter of the jth neuron; ∑j=1lβj=1, βj=1/l; ϕijX is the output of the new neuron; and e is the output error of the neural network before decomposition.


## 3.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

The MI functionIa;y of neuron a in the hidden layer and neuron y in the output layer is obtained from (6). Ia;y depends on the intensity of information between a and y,
(6)Ia;y=∑a,yfa;ylog2fa;yfafy,where fa;y is the joint distribution density function of a and y. fa;y=fafy∣a, where fa and fy are the probability densities of a and y, respectively. According to the Shannon entropy theory, fa;y, fa, and fy are not calculated in fact, and the entropies are calculated instead.Assuming thata and y are interconnected neurons, Ia;y depends on the intensity of information between a and y.According to the Shannon entropy theory, the connection strengthIa;y between a and y can be calculated by
(7)Ia;y=ENa−ENay=ENy−ENya,where ENa is the entropy of a and ENa∣y is the entropy of a in the condition of y. When a and y are independent, Ia;y=0. Otherwise, Ia;y>0. Therefore, Ia;y≥0. The range of Ia;y is shown in
(8)Ia;y≤minENa,ENy.Then, based on (8), the normalized MI is obtained from
(9)Isa;y=Ia;y/minENa,ENy,where Isa;y∈0,1. The calculation of Is is able to determine the interaction strength between a and y.In the RBF neural network, whenIs is large, the interaction strength between a and y is strong and the connection of a and y exists. When Is approaches 0, it indicates that the interaction strength between a and y is weak. Then the connection of a and y can be deleted in the structure adjustment. It can reduce the redundancy of the neural network.In Figure4, when Is≤I0 (I0=0.01ed), the connection between a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The new neuron b contains neurons a and b. Thus, a new layer is not added to the network, and there is no weight between neurons a and b. The parameters of neuron b are adjusted from (10)–(12). Cb′ is the same as Ca. δb′ is the same as δa:
(10)Cb′=Ca,(11)δb′=δa,(12)wb′=wb+wa⋅ϕaXϕbX,where Cb′, δb′, and wb′ are the center, variance, and weight between b and y in the new output layer, respectively.Figure 4
Disconnection of the neuron in the hidden layer and the neuron in output layers.VS-RBF can not only increase the number of neurons in the hidden layer but also remove redundant neurons. The optimal number of RBF neurons is obtained by adjusting the output information of neurons in the hidden layer and the multi-information between neurons in the hidden layer and output layer. When the error of output meets the demand, the optimal number of RBF neurons is obtained.


## 3.3. Robust Regression Training Method for Weights in the Output Layer

At present, RBF neural network weights are usually trained by the linear least square algorithm, but the least square estimation is affected by outliers. In order to reduce the influence of outliers, the robust regression method is applied to train weights in the output layer.The output of the RBF network is the weighted sum of the output of the hidden layer. The aim of the training is to make the sum of squared errorEW between the output of the whole network and the actual output least. The sum of squared error EW is obtained from
(13)EW=min∑t=1Nyt−ΦtTW2=min∑t=1Nσt2,where yt is the value of actual output; ΦtT is the output sample vector of the hidden layer; Wt is the weight vector; N is the number of samples; and t=1,2,…,N.In order to reduce the influence of outliers, we hope to find a functionθσt, which increases with the increase of σt and the growth rate is slower than σt2. Then the sum of squared error EW can be expressed as
(14)EW=min∑t=1Nθσt.θσt is an Andrews function, which can be expressed as (15). And the derivative of θσt can be expressed as
(15)θσt=K2−K2cosσtK,σt≥πK,2K2,σt<πK,(16)θ′σt=KsinσtK,σt≥πK,0,σt<πK,where θ′σt is the derivative of θσt and K is a threshold constant.Therefore, robust regression can be written in recursive form, which is expressed in(17)Wt=Wt−1−η∂θσt∂W,where W is the estimated weight and η is the training coefficient, which can be determined according to the actual situation. Then Wt is transferred to
(18)Wt=Wt−1−η∂θσt∂σt⋅∂σt∂W=Wt−1−ηθ′σt⋅−ΦtT=Wt−1+ηθ′yt−ΦtTWtΦtT.It can reduce the influence of outliers on neural network training by using the robust regression method.


## 3.4. Fast Learning Rate

In the use of the RBF neural network, the learning rate was often subjective to set as a fixed value. It remained unchanged throughout the learning process. If the learning rate was set too high, the convergence speed of the network might be very fast and it might cause network instability. If the learning rate was too small, it would cause that the network convergence speed was slow and consume a large amount of computing time. Therefore, it is very difficult to choose a suitable learning rate for the traditional RBF neural network. In order to solve this problem, this paper proposes a new fast learning rate. The fast learning rate is suitable for each step of the iteration, which can ensure the stability of the network. At the same time, the convergence speed of the network and the efficiency of the network can be improved.Set(19)Φ=ϕ11ϕ12⋯ϕ1mϕ21ϕ22⋯ϕ2m⋮⋮⋱⋮ϕN1ϕN2⋯ϕNm.N is the number of samples. m is the number of nodes in the hidden layer. y^ is the output calculated by the network. Then the cost function Et of the tth train can be defined in
(20)Et=12∑j=1Nyjt−y^jt2=12∑j=1Nej2t.The output error iset=e1t,e2t,⋯,eNtT. ΔWt=Wt−Wt−1 is the change of the weight in the tth train. According to (18), ΔWt is obtained in
(21)ΔWt=ηtθ′yt−ΦtTWtΦtT.The increase of error can be expressed as(22)Δet=et−et−1=yt−y^t−yt−1+y^t−1.Δyt=yt−yt−1 is the change of the actual output. Δy^t=y^t−y^t−1 is the change of the network output. In general, the absolute value of the change of the actual output is far less than the absolute value of the change of the network output. Δyt≪Δy^t. The change of the actual output is negligible compared with the change of the network output, because the actual output is often constrained by many conditions, and the network output will not be limited. This assumption has practical significance. Thus, (22) can be approximated as
(23)Δet=et−et−1≈−Δy^t.Considering (21), the change of error Δet is expressed in
(24)Δet≈−Δy^t=−ΦΔWt=−ηtΦθ′y−ΦTWtΦT.Then,et is expressed in (25) considering (24):
(25)et=et−1+Δet≈et−1−ηtΦθ′y−ΦTWtΦT.The cost functionEt of the tth train can be obtained from (25). Et is expressed in
(26)Et=12eTtet≈12et−1−ηtΦθ′y−ΦTWtΦTT⋅et−1−ηtΦθ′y−ΦTWtΦT.This cost function can be considered as a function of the learning rateηt. The optimal value of learning rate ηt can be obtained by minimizing Et. The first order condition of (26) is expressed in
(27)∂Et∂ηtηt=η∗t=−12Φθ′y−ΦTWtΦTT⋅et−1−ηtΦθ′y−ΦTWtΦT−12et−1−ηtΦθ′y−ΦTWtΦTT⋅Φθ′y−ΦTWtΦT=0.The second order condition of (26) is expressed in
(28)∂2Et∂η2tηt=η∗t=12Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT+12Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT=Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT>0.The second order condition (28) holds as Φ is positive definite. And the fast learning rate η∗t can be obtained by (27). η∗t is expressed in
(29)η∗t=12Φθ′y−ΦTWtΦTTet−1Φθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT+12eTt−1Φθ′y−ΦTWtΦTΦθ′y−ΦTWtΦTTΦθ′y−ΦTWtΦT.


## 3.5. Learning Algorithm of VS-RBF with the Fast Learning Rate

VS-RBF with the fast learning rate can not only increase the number of neurons in the hidden layer but also remove redundant neurons. At the same time, the use of the robust regression algorithm can avoid the influence of the abnormal sample on the neural network. And the fast learning rate can ensure that the learning rate is appropriate in the iterative process. In this paper, the steps of the learning algorithm are shown in Figure5.
Step 1.
For a given RBF neural network, the number of neurons in the hidden layer is a small natural number. Considering that VS-RBF can adjust the number of neurons in the hidden layer, the number of initial neurons in the hidden layer is defined as the neurons in the input layer or a little more. In this paper, we consider that the number of initial neurons in the hidden layer is equal to the neurons in the input layer. The neural network centerC and variance δ are selected by using the least square method. Φ is calculated. The learning accuracy ed and the maximum number of iterations maxT are set. The number of neurons in the input layer is the number of input. The number of neurons in the output layer is the number of output. A0=max100ed,1/m, I0=0.01ed
Step 2.
The initial weights between the hidden layer and the output layerW0 are set. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer is calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedFigure 5
Learning algorithm of VS-RBF with the fast learning rate.VS-RBF with fast learning rate can realize the self-organization of the structure and determine whether to grow or prune the neurons in the hidden layer by calculating the information intensity. A new RBF neural network structure design method is proposed. It not only can adjust the weights of neural network online but also can grow or prune the neurons in the hidden layer. From the biological point of view, this neural network structure is more similar to the mechanism of human brain neuron information processing.


## 3.6. Comparison of VS-RBF, SORBF, and RS-RBF

The advantages, main novelties, and disadvantages of SORBF [35] and RS-RBF [36] are as follows:
(1)
SORBF: neither the number of nodes in the hidden layer nor the parameters need to be predefined and fixed. They are adjusted automatically in the learning process by comparingvx−Pk=minx=1,2,…,Nvx−Pk and βrx. This type of SORBF-based approach offers a promisingly inexpensive approach to real-time measurement of variables that have typically proved difficult to measure reliably using hardware. However, this algorithm ignores the adjustment of structural parameters, which leads to the slow convergence speed of the neural network learning algorithm
(2)
RS-RBF: by processing multiple nodes of the network at one time, multiple hidden nodes can be cut off, and the core nodes in the hidden nodes can be found by calculating the output of the network asDn=outputn−OUTPUT/OUTPUT. Adaptive principle is introduced to make the segmentation change with pruning. The memory function is to remember the most important nodes for each pruning, and the most important nodes will be not deleted in the subsequent pruning even if the output of these nodes is small. However, this method was affected by the initial value, and sometimes, the final RBF neural network was unstable


## 4. Proof of Convergence

The convergence of VS-RBF with the fast learning rate affects the performance of the final network.


### 4.1. Fixed Neural Network Structure

When the neural network structure is fixed, it can be concluded that VS-RBF with the fast learning rate can guarantee the convergence of the final neural network referring to [31].Theorem 1.
According to the changed neural network structure of VS-RBF, the error after decomposition is 0, and the error after interconnection adjustment is equal to the error of the fixed neural network structure.


### 4.2. Changed Neural Network Structure

At the momentt, VS-RBF with the fast learning rate has m neurons in the hidden layer. Current error is emt.


#### 4.2.1. Decomposition of the Neuron in the Hidden Layer

When the neurona in the hidden layer is decomposed, the number of new neurons is l after decomposition. Then the number of neurons in the hidden layer becomes m+l−1 after decomposition. The error after decomposition em+l−1′t is expressed in
(30)em+l−1′t=∑i=1m+l−1wiϕiXt−ydt=∑i=1mwiϕiXt+∑j=1lwa,jϕjXt−waϕaXt−ydt,where ydt is expected output at the moment t and Xt is a sample at the moment t. It can be obtained from (3) and (4).In (30), two terms in the middle can be expressed in Eq. (31). ∑j=1lβj=1,
(31)∑j=1lwa,jϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emtϕjXtϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emt−waϕaXt=waϕaXt−emt−waϕaXt=−emt,where Xt=x1,x2,⋯,xnT∈Rn is the input vector.According to (30) and (31), the error of the neural network after decomposition em+l−1′t is expressed in
(32)em+l−1′t=∑i=1mwiϕiXt−ydt−emt=emt−emt=0.After adjusting the structure, the error of network output at the timet is zero. The convergence of average error E is speeded up (E=1/2N∑i=1Ne2i, N is the number of samples). It reflects the decomposition of neurons can improve neural network learning efficiency.


#### 4.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

At the momentt, the connections between the neuron a in the hidden layer and neuron y in the output layer need to be disconnected. And the nearest neuron with a is b. Then error of network output em−1′t will become (33) after disconnection,
(33)em−1′t=∑i=1mwiϕiXt−ydt−waϕaXt.According to (10)–(12), (33) can be rewritten as
(34)em−1′t=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb′ϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb+waϕaXtϕbXtϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wbϕbXt+waϕaXt=∑i=1mwiϕiXt−ydt=emt.Therefore, the disconnection of the neurons in the hidden layer and the neurons in output layers cannot affect the error of network output. Thus, the optimization process of the RBF structure does not affect the convergence of the neural network.To sum up, VS-RBF with the fast learning rate can guarantee the convergence of the final network. At the same time, the algorithm is simple. The neural network can not only realize the structure and parameter adjustment but also consider the weight changes in the structure of the optimization process.


## 4.1. Fixed Neural Network Structure

When the neural network structure is fixed, it can be concluded that VS-RBF with the fast learning rate can guarantee the convergence of the final neural network referring to [31].Theorem 1.
According to the changed neural network structure of VS-RBF, the error after decomposition is 0, and the error after interconnection adjustment is equal to the error of the fixed neural network structure.


## 4.2. Changed Neural Network Structure

At the momentt, VS-RBF with the fast learning rate has m neurons in the hidden layer. Current error is emt.


### 4.2.1. Decomposition of the Neuron in the Hidden Layer

When the neurona in the hidden layer is decomposed, the number of new neurons is l after decomposition. Then the number of neurons in the hidden layer becomes m+l−1 after decomposition. The error after decomposition em+l−1′t is expressed in
(30)em+l−1′t=∑i=1m+l−1wiϕiXt−ydt=∑i=1mwiϕiXt+∑j=1lwa,jϕjXt−waϕaXt−ydt,where ydt is expected output at the moment t and Xt is a sample at the moment t. It can be obtained from (3) and (4).In (30), two terms in the middle can be expressed in Eq. (31). ∑j=1lβj=1,
(31)∑j=1lwa,jϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emtϕjXtϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emt−waϕaXt=waϕaXt−emt−waϕaXt=−emt,where Xt=x1,x2,⋯,xnT∈Rn is the input vector.According to (30) and (31), the error of the neural network after decomposition em+l−1′t is expressed in
(32)em+l−1′t=∑i=1mwiϕiXt−ydt−emt=emt−emt=0.After adjusting the structure, the error of network output at the timet is zero. The convergence of average error E is speeded up (E=1/2N∑i=1Ne2i, N is the number of samples). It reflects the decomposition of neurons can improve neural network learning efficiency.


### 4.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

At the momentt, the connections between the neuron a in the hidden layer and neuron y in the output layer need to be disconnected. And the nearest neuron with a is b. Then error of network output em−1′t will become (33) after disconnection,
(33)em−1′t=∑i=1mwiϕiXt−ydt−waϕaXt.According to (10)–(12), (33) can be rewritten as
(34)em−1′t=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb′ϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb+waϕaXtϕbXtϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wbϕbXt+waϕaXt=∑i=1mwiϕiXt−ydt=emt.Therefore, the disconnection of the neurons in the hidden layer and the neurons in output layers cannot affect the error of network output. Thus, the optimization process of the RBF structure does not affect the convergence of the neural network.To sum up, VS-RBF with the fast learning rate can guarantee the convergence of the final network. At the same time, the algorithm is simple. The neural network can not only realize the structure and parameter adjustment but also consider the weight changes in the structure of the optimization process.


## 4.2.1. Decomposition of the Neuron in the Hidden Layer

When the neurona in the hidden layer is decomposed, the number of new neurons is l after decomposition. Then the number of neurons in the hidden layer becomes m+l−1 after decomposition. The error after decomposition em+l−1′t is expressed in
(30)em+l−1′t=∑i=1m+l−1wiϕiXt−ydt=∑i=1mwiϕiXt+∑j=1lwa,jϕjXt−waϕaXt−ydt,where ydt is expected output at the moment t and Xt is a sample at the moment t. It can be obtained from (3) and (4).In (30), two terms in the middle can be expressed in Eq. (31). ∑j=1lβj=1,
(31)∑j=1lwa,jϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emtϕjXtϕjXt−waϕaXt=∑j=1lβjwaϕaXt−emt−waϕaXt=waϕaXt−emt−waϕaXt=−emt,where Xt=x1,x2,⋯,xnT∈Rn is the input vector.According to (30) and (31), the error of the neural network after decomposition em+l−1′t is expressed in
(32)em+l−1′t=∑i=1mwiϕiXt−ydt−emt=emt−emt=0.After adjusting the structure, the error of network output at the timet is zero. The convergence of average error E is speeded up (E=1/2N∑i=1Ne2i, N is the number of samples). It reflects the decomposition of neurons can improve neural network learning efficiency.


## 4.2.2. Interconnection Adjustment between the Neurons in the Hidden Layer and in the Output Layer

At the momentt, the connections between the neuron a in the hidden layer and neuron y in the output layer need to be disconnected. And the nearest neuron with a is b. Then error of network output em−1′t will become (33) after disconnection,
(33)em−1′t=∑i=1mwiϕiXt−ydt−waϕaXt.According to (10)–(12), (33) can be rewritten as
(34)em−1′t=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb′ϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wb+waϕaXtϕbXtϕbXt=∑i=1,i≠bmwiϕiXt−ydt−waϕaXt+wbϕbXt+waϕaXt=∑i=1mwiϕiXt−ydt=emt.Therefore, the disconnection of the neurons in the hidden layer and the neurons in output layers cannot affect the error of network output. Thus, the optimization process of the RBF structure does not affect the convergence of the neural network.To sum up, VS-RBF with the fast learning rate can guarantee the convergence of the final network. At the same time, the algorithm is simple. The neural network can not only realize the structure and parameter adjustment but also consider the weight changes in the structure of the optimization process.


## 5. Experimental Validation

VS-RBF with the fast learning rate can adjust the number of neurons in the hidden layer on the basis of the complexity of the object and improve the performance of the RBF neural network. Compared with other self-organizing RBF neural networks (SORBF and RS-RBF), VS-RBF has a more compact structure, faster dynamic response speed, and better generalization ability in approximating a typical nonlinear function, identifying UCI datasets, and evaluating sortie generation capacity of the carrier aircraft.


### 5.1. Nonlinear Function Approximation

The nonlinear function SIF is selected as(35)y=2x12sin4x1+x22+sin3x2+x1sin4x2,where x1∈−2,2 and x2∈−2,2. The nonlinear function SIF is commonly used to test the performance of the neural network [37].Data points are randomly sampled adding white Gaussian noise with a standard deviation of 0.01 to produce training and validation datasets, each containing 300 samples. Thus, 600 groups of samples are selected. 300 groups are used for training and the other 300 groups are used to test. The simulation environment is a computer with Intel Core i3–4160 CPU, 4.00 GB RAM and 64 bit operation system. The software is Matlab R2016b.The main steps on the identification of VS-RBF are as follows:Step 1.
The structure of the neural network is 2-3-1. The initial number of neurons in the hidden layer is set as 3. The neural network centerC is selected as [−2, 0, −2; −2, 0, −2]. The variance δ is 3. The initial function width is 1. The learning accuracy ed is 0.01. The maximum number of iterations maxT is set as 10,000
Step 2.
The initial weights between the hidden layer and the output layerW0 are set as 1. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer are calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedUnder this condition, the comparison of performances of VS-RBF, SORBF [35], and RS-RBF [36] is shown in Table 1. The remaining neurons in the hidden layer in the SIF approximation are shown in Figure 6. The approximation effect of SIF is shown in Figure 7. The error surface is shown in Figure 8.Table 1
Comparisons of performances in approximating SIF.
Algorithm
Learning accuracy
Actual error
Node number of the final hidden layer
Training time (s)
VS-RBF
0.01
0.0148
14
213.41
SORBF
0.01
0.0164
16
221.37
RS-RBF
0.01
0.0206
21
459.12Figure 6
Number of remaining neurons in SIF training.Figure 7
Effect of VS-RBF in SIF approximation.Figure 8
SIF approximation error of VS-RBF.Figure6 shows the changes of the number of remaining neurons in the training process. We can find that the structure adjustment of VS-RBF is stable, and the structure is most compact. A neuron of VS-RBF can be decomposed into several at one time. VS-RBF structure adjustment is quicker. The information processing ability of the RBF neural network is improved.Figure7 shows that VS-RBF can well approximate the nonlinear function SIF after training. And the output value of VS-RBF coincides with the actual value.Figure8 shows the error surface of VS-RBF in approximation. The test error is less than 0.015. Table 1 gives the comparison of VS-RBF, SORBF, and RS-RBF. Under the same initial conditions, the training times of SORBF and RS-RBF are more than that of VS-RBF. The structures of SORBF and RS-RBF after training are more complex than that of VS-RBF.In addition, when using the trained neural network for function approximation, the test errors of SORBF and RS-RBF are larger than that of VS-RBF. The VS-RBF neural network has faster training speed, more compact network structure, and stronger nonlinear function approximation ability.


### 5.2. UCI Datasets

In order to show the effectiveness of VS-RBF, the identification effect is justified on UCI datasets. Istanbul Stock Exchange Dataset is selected in UCI datasets. Data is collected fromhttp://imkb.gov.tr and http://finance.yahoo.com. Data is organized with regard to working days in Istanbul Stock Exchange. The selected datasets include returns of Istanbul Stock Exchange (ISE) with seven other international indexes: Standard & Poor’s 500 return index (SP), stock market return index of Germany (DAX), stock market return index of UK (FTSE), stock market return index of Japan (NIK), stock market return index of Brazil (BVSP), MSCI European Index (EU), and MSCI Emerging Markets Index (EM) from June 5, 2009, to February 22, 2011. There are 536 groups in this dataset. The first 436 groups are used to train the network, and the last 100 groups are used to test the network. The input number is 7, and the output (ISE) number is 1.The main steps on the identification of VS-RBF on UCI datasets are as follows:Step 1.
The structure of the neural network is 7-10-1. The initial number of neurons in the hidden layer is set as 10. The neural network centerC is selected as [−3, −3, −3, −3, −3, −3, −3, −3, −3, −3; −2, −2, −2, −2, −2, −2, −2, −2, −2, −2; −1, −1, −1, −1, −1, −1, −1, −1, −1, −1; 0, 0, 0, 0, 0, 0, 0, 0, 0, 0; 1, 1, 1, 1, 1, 1, 1, 1, 1, 1; 2, 2, 2, 2, 2, 2, 2, 2, 2, 2; 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,]. The variance δ is 2. The initial function width is 2. The learning accuracy ed is 0.01. The maximum number of iterations maxT is set as 10,000
Step 2.
The initial weights between the hidden layer and the output layerW0 are set as 1. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer is calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedUnder this condition, the comparison of performances of VS-RBF, SORBF [35], and RS-RBF [36] is shown in Table 2. The identification effect in the test is shown in Figure 9. The identification error is shown in Figure 10.Table 2
Comparisons of performances in UCI datasets.
Algorithm
Learning accuracy
Actual error
Node number of the final hidden layer
Training time (s)
VS-RBF
0.01
0.0131
20
230.56
SORBF
0.01
0.0208
24
292.30
RS-RBF
0.01
0.0263
33
441.52Figure 9
Justification of UCI datasets.Figure 10
Identification error of UCI datasets.In Table2, the remaining node number in the hidden layer of VS-RBF is the least. And the actual error and training time of VS-RBF are less than those of SORBF and RSRBF.Figure9 shows that VS-RBF can well identify the test data. And the output value of VS-RBF coincides with the actual value.Figure10 shows the identification error of VS-RBF. The test error is less than 0.014.Thus, when using the trained neural network for identification, the test errors of SORBF and RS-RBF are larger than that of VS-RBF. The VS-RBF neural network has faster training speed, more compact network structure, and stronger identification ability in Istanbul Stock Exchange Dataset.


## 5.1. Nonlinear Function Approximation

The nonlinear function SIF is selected as(35)y=2x12sin4x1+x22+sin3x2+x1sin4x2,where x1∈−2,2 and x2∈−2,2. The nonlinear function SIF is commonly used to test the performance of the neural network [37].Data points are randomly sampled adding white Gaussian noise with a standard deviation of 0.01 to produce training and validation datasets, each containing 300 samples. Thus, 600 groups of samples are selected. 300 groups are used for training and the other 300 groups are used to test. The simulation environment is a computer with Intel Core i3–4160 CPU, 4.00 GB RAM and 64 bit operation system. The software is Matlab R2016b.The main steps on the identification of VS-RBF are as follows:Step 1.
The structure of the neural network is 2-3-1. The initial number of neurons in the hidden layer is set as 3. The neural network centerC is selected as [−2, 0, −2; −2, 0, −2]. The variance δ is 3. The initial function width is 1. The learning accuracy ed is 0.01. The maximum number of iterations maxT is set as 10,000
Step 2.
The initial weights between the hidden layer and the output layerW0 are set as 1. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer are calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedUnder this condition, the comparison of performances of VS-RBF, SORBF [35], and RS-RBF [36] is shown in Table 1. The remaining neurons in the hidden layer in the SIF approximation are shown in Figure 6. The approximation effect of SIF is shown in Figure 7. The error surface is shown in Figure 8.Table 1
Comparisons of performances in approximating SIF.
Algorithm
Learning accuracy
Actual error
Node number of the final hidden layer
Training time (s)
VS-RBF
0.01
0.0148
14
213.41
SORBF
0.01
0.0164
16
221.37
RS-RBF
0.01
0.0206
21
459.12Figure 6
Number of remaining neurons in SIF training.Figure 7
Effect of VS-RBF in SIF approximation.Figure 8
SIF approximation error of VS-RBF.Figure6 shows the changes of the number of remaining neurons in the training process. We can find that the structure adjustment of VS-RBF is stable, and the structure is most compact. A neuron of VS-RBF can be decomposed into several at one time. VS-RBF structure adjustment is quicker. The information processing ability of the RBF neural network is improved.Figure7 shows that VS-RBF can well approximate the nonlinear function SIF after training. And the output value of VS-RBF coincides with the actual value.Figure8 shows the error surface of VS-RBF in approximation. The test error is less than 0.015. Table 1 gives the comparison of VS-RBF, SORBF, and RS-RBF. Under the same initial conditions, the training times of SORBF and RS-RBF are more than that of VS-RBF. The structures of SORBF and RS-RBF after training are more complex than that of VS-RBF.In addition, when using the trained neural network for function approximation, the test errors of SORBF and RS-RBF are larger than that of VS-RBF. The VS-RBF neural network has faster training speed, more compact network structure, and stronger nonlinear function approximation ability.


## 5.2. UCI Datasets

In order to show the effectiveness of VS-RBF, the identification effect is justified on UCI datasets. Istanbul Stock Exchange Dataset is selected in UCI datasets. Data is collected fromhttp://imkb.gov.tr and http://finance.yahoo.com. Data is organized with regard to working days in Istanbul Stock Exchange. The selected datasets include returns of Istanbul Stock Exchange (ISE) with seven other international indexes: Standard & Poor’s 500 return index (SP), stock market return index of Germany (DAX), stock market return index of UK (FTSE), stock market return index of Japan (NIK), stock market return index of Brazil (BVSP), MSCI European Index (EU), and MSCI Emerging Markets Index (EM) from June 5, 2009, to February 22, 2011. There are 536 groups in this dataset. The first 436 groups are used to train the network, and the last 100 groups are used to test the network. The input number is 7, and the output (ISE) number is 1.The main steps on the identification of VS-RBF on UCI datasets are as follows:Step 1.
The structure of the neural network is 7-10-1. The initial number of neurons in the hidden layer is set as 10. The neural network centerC is selected as [−3, −3, −3, −3, −3, −3, −3, −3, −3, −3; −2, −2, −2, −2, −2, −2, −2, −2, −2, −2; −1, −1, −1, −1, −1, −1, −1, −1, −1, −1; 0, 0, 0, 0, 0, 0, 0, 0, 0, 0; 1, 1, 1, 1, 1, 1, 1, 1, 1, 1; 2, 2, 2, 2, 2, 2, 2, 2, 2, 2; 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,]. The variance δ is 2. The initial function width is 2. The learning accuracy ed is 0.01. The maximum number of iterations maxT is set as 10,000
Step 2.
The initial weights between the hidden layer and the output layerW0 are set as 1. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer is calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedUnder this condition, the comparison of performances of VS-RBF, SORBF [35], and RS-RBF [36] is shown in Table 2. The identification effect in the test is shown in Figure 9. The identification error is shown in Figure 10.Table 2
Comparisons of performances in UCI datasets.
Algorithm
Learning accuracy
Actual error
Node number of the final hidden layer
Training time (s)
VS-RBF
0.01
0.0131
20
230.56
SORBF
0.01
0.0208
24
292.30
RS-RBF
0.01
0.0263
33
441.52Figure 9
Justification of UCI datasets.Figure 10
Identification error of UCI datasets.In Table2, the remaining node number in the hidden layer of VS-RBF is the least. And the actual error and training time of VS-RBF are less than those of SORBF and RSRBF.Figure9 shows that VS-RBF can well identify the test data. And the output value of VS-RBF coincides with the actual value.Figure10 shows the identification error of VS-RBF. The test error is less than 0.014.Thus, when using the trained neural network for identification, the test errors of SORBF and RS-RBF are larger than that of VS-RBF. The VS-RBF neural network has faster training speed, more compact network structure, and stronger identification ability in Istanbul Stock Exchange Dataset.


## 6. Evaluation for Sortie Generation Capacity of the Carrier Aircraft

The inputs of the neural network are the evaluation indexes of sortie generation capacity of the carrier aircraft. The output of the neural network is the evaluation value of sortie generation capacity of the carrier aircraft by the expert scoring method. This paper selects the surge operation of the “Nimitz” carrier in 1997 as the object [1].500 groups of data are selected excluding abnormal data. 400 groups are used in training, and 100 groups are used in testing. The inputs of the neural network arex1∼x15, which are corresponding to the 15 indexes of the evaluation index system, respectively.The number of output is 1, which value is0,1. The data of each group in the 500 groups of samples included 15 normalized lowest level indexes and one evaluation value in the range of 0,1 for sortie generation capacity of the carrier aircraft evaluated by experts. Due to limitation of space, 10 groups of samples are given in Tables 3–6.When the evaluation for sortie generation capacity of the carrier aircraft is closer to 1, it is indicated that the allocation of the sortie generation scheme under the current indexes is more reasonable, and the sortie generation capacity of the carrier aircraft is higher. Otherwise, when the evaluation for sortie generation capacity of the carrier aircraft is closer to 0, it indicates that the allocation of the sortie generation scheme under the current indexes is more unreasonable. And the sortie generation capacity of the carrier aircraft is lower, which means that the scheme should be advanced.Table 3
Sortie generation rate capacity.
Group
x1 (sortie)
x2 (sortie/day)
x3 (sortie/day)
Evaluation
1
30
250
200
0.8622
2
31
240
180
0.2943
3
29
235
210
0.8112
4
33
260
220
0.5285
5
32
210
170
0.1656
6
29
245
194
0.6020
7
27
267
230
0.9630
8
32
211
183
0.6541
9
25
261
201
0.6892
10
32
232
196
0.5482Table 4
Aircraft availability capacity.
Group
x4 (%)
x5 (%)
x6 (%)
Evaluation
1
80
11
9
0.8622
2
85
20
5
0.2943
3
90
4
6
0.8112
4
75
11
14
0.5285
5
82
10
18
0.1656
6
91
3
5
0.6020
7
78
23
9
0.9630
8
84
9
7
0.6541
9
85
1
4
0.6892
10
72
2
16
0.5482Table 5
Task completion capacity.
Group
x7 (%)
x8 (sortie/day)
x9 (%)
x10 (sortie/day)
Evaluation
1
85
2.5
90
6
0.8622
2
74
2.2
80
7
0.2943
3
81
2.0
84
5
0.8112
4
90
1.8
75
6
0.5285
5
61
1.5
68
5
0.1656
6
86
1.9
86
8
0.6020
7
78
2.1
88
5
0.9630
8
65
2.3
94
6
0.6541
9
79
2.4
81
7
0.6892
10
83
1.7
82
5
0.5482Table 6
Support, ejection, and recovery capabilities.
Group
x11 (minute)
x12 (minute)
x13 (%)
x14 (minute)
x15 (%)
Evaluation
1
30
1
1
1.5
3.3
0.8622
2
32
2
0.5
1.8
5
0.2943
3
28
1.4
1.2
1.4
1
0.8112
4
25
1.6
1.6
1.9
7
0.5285
5
33
2.5
2
2.2
6
0.1656
6
45
1.1
3
1.1
2
0.6020
7
27
0.6
0.6
1.2
10
0.9630
8
26
0.7
0.8
1.6
3
0.6541
9
36
0.5
1.6
1.7
4.5
0.6892
10
29
1.2
0.4
2.1
6
0.5482The main steps on the identification of VS-RBF are as follows:Step 1.
The number of neurons in the hidden layer is set to 15 initially. The structure of neural network is 15-15-1. The initial number of neurons in the hidden layer is set as 15. The neural network centerC is selected as 0.1∗[−7, −7, −7, −7, −7, −7, −7, −7, −7, −7, −7, −7, −7, −7, −7; −6, −6, −6, −6, −6, −6, −6, −6, −6, −6, −6, −6, −6, −6, −6; −5, −5, −5, −5, −5, −5, −5, −5, −5, −5, −5, −5, −5, −5, −5; −4, −4, −4, −4, −4, −4, −4, −4, −4, −4, −4, −4, −4, −4, −4; −3, −3, −3, −3, −3, −3, −3, −3, −3, −3, −3, −3, −3, −3, −3; −2, −2, −2, −2, −2, −2, −2, −2, −2, −2, −2, −2, −2, −2, −2; −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1; 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0; 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1; 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2; 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3; 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4; 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5; 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6; 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7]. The variance δ is 0.5. The initial function width is 1. The learning accuracy ed is 0.001. The maximum number of iterations maxT is set as 10,000
Step 2.
The initial weights between the hidden layer and the output layerW0 are set as 0.5. The network output y^=ΦW is calculated. And the root mean square error e between the actual output and the network output is calculated
Step 3.
Ife<ed or the number of iterations reaches maxT, go to Step 7. Otherwise, the fast learning rate η∗t is calculated by (29). The weights are updated by Wt=Wt−1+η∗tθ′yt−ΦtTWtΦtT
Step 4.
The activityAiXi=1,2,⋯,m of the neuron in the hidden layer is calculated. If AiX is greater than the threshold A0, the neuron i is decomposed, and the network structure is adjusted. The initial parameters of the new neuron are set according to (3) and (4)
Step 5.
The connection strengthIs between neuron a in the hidden layer and neuron y in the output layer is calculated. When Is≤I0, the connection of a and y is disconnected. In the hidden layer, the nearest neuron b with a is found. The parameters of neuron b are adjusted as (10)–(12)
Step 6.
The network output valuey^=ΦW is updated. The root mean square error between the actual output and the network output e is updated. Return to Step 3
Step 7.
The root mean square errore and the network output y^ are calculatedThe trained VS-RBF neural network makes the final evaluation results closer to the evaluation results of the expert, which indicates that VS-RBF can not only replace experts to evaluate the sortie generation capacity of the carrier aircraft but also guarantee the high efficiency of evaluation. What is more, the trained VS-RBF can avoid human errors and the subjective effect on the evaluation process. At the same time, the trained VS-RBF neural network is used to establish the complex nonlinear relationship between the 15 indexes and the evaluation result. By using the nonlinear model of the indexes and the evaluation result, the influence of each index on the evaluation result can be further determined so as to provide the reference and suggestion for improving the sortie generation capacity of the carrier aircraft.The linear transformation method is used to convert the original sample to the sample in0,1, which is divided into two cases:
(1)
When the index value is bigger, the sortie generation capacity of the carrier aircraft is better. Linear transformation is expressed in(36)Y=X−minmax−min
(2)
When the index value is smaller, the sortie generation capacity of the carrier aircraft is better. Linear transformation is expressed in(37)Y=max−Xmax−min,where Y is the normalized sample value, X is the original sample value, and max and min are the maximum value and the minimum value of the current system, respectivelyThe initial parameters of VS-RBF, SORBF, and RS-RBF are the same. The initial weights are arbitrary values. The initial center is arbitrary value from 0 to 1. The initial function width is given to 1. The learning accuracy of the neural network is 0.001. The maximum number of iterations is 10,000.The average performances of VS-RBF, SORBF, and RS-RBF after 50 trains are shown in Table7. The mean square errors of VS-RBF, SORBF, and RS-RBF in the training process are shown in Figure 11. The changes of the number of neurons in the hidden layer of VS-RBF, SORBF, and RS-RBF are shown in Figure 12. The comparisons of the network output values and the actual output values of VS-RBF, SORBF, and RS-RBF in the testing process are shown in Figure 13. The errors between the network output values and the actual output values for VS-RBF, SORBF, and RS-RBF are shown in Figure 14.Table 7
Comparisons of average training performance.
Algorithm
Learning accuracy
Actual error
Node number of the final hidden layer
Training time (s)
VS-RBF
0.001
0.0024
31
195.38
SORBF
0.001
0.0037
42
202.42
RS-RBF
0.001
0.0041
66
418.45Figure 11
Error variations during training.Figure 12
Number of remaining neurons during training.Figure 13
Output value of evaluation of the sortie generation capacity of the carrier aircraft.Figure 14
Evaluation error of the sortie generation capacity of the carrier aircraft.The simulation shows that VS-RBF can accurately evaluate the sortie generation capacity of the carrier aircraft. Figures13 and 14 show that evaluation value of VS-RBF agrees with the actual evaluation value. The error is less than 0.01. The evaluation error of VS-RBF is less than those of SORBF and RS-RBF. It proves the effectiveness of the evaluation of the sortie generation capacity of the carrier aircraft with VS-RBF.Figure12 and Table 7 show that the average training time of VS-RBF is shorter than those of the other two neural network algorithms. And the final structure of the network of VS-RBF is most compact, which illustrates the effectiveness of VS-RBF in structural adjustment of neural networks. Table 7 shows that the evaluation error of VS-RBF is smallest, which shows that VS-RBF has good generalization ability.


### 6.1. Analysis for Sortie Generation Capacity of Carrier Aircrafts

The nonlinear VS-RBF model of the indexes and evaluation result not only can be used for evaluating a given sortie generation scheme but also can further determine the effect of various indexes on the evaluation result. The scheme with basic configuration of surge operation of “Nimitz” carrier in 1997 is selected as a reference scheme, which is shown in Table8.Table 8
The reference scheme.
No.
Index
Unit
Basic configuration
Normalized result
1
ESGR
Sortie
30
0.76
2
SSGR
Sortie/day
250
0.87
3
LSGR
Sortie/day
200
0.79
4
PTP
%
80
0.91
5
MTPWP
%
11
0.64
6
MTPWR
%
9
0.58
7
SCP
%
85
0.87
8
PUR
Sortie/day
2.5
0.94
9
PIPA
%
90
0.95
10
SGRA
Sortie/day
6
0.86
11
PTNS
Minute
30
0.83
12
EI
Minute
1
0.94
13
TOOP
%
1
0.87
14
RI
Minute
1.5
0.74
15
OP
%
3.3
0.90On this basis, the indexxii=1,2,⋯,15 is, respectively, adjusted as 0, 0.25, 0.5, 0.75, 1, and sortie generation capacity of the carrier aircraft is evaluated.The 15 indexes are adjusted, respectively, and the influence curve and slope of each index on the sortie generation capacity are obtained, which are shown in Figure15 and Table 9.Figure 15
The influence curves of 15 indexes on the sortie generation capacity.
(a)
Sortie generation rate capacity
(b)
Availability capacity
(c)
Tasks completion capacity
(d)
Support, ejection, and recovery capacitiesTable 9
The slopes of 15 indexes.
No.
Index
Slope
Absolute value of slope
1
ESGR
0.2120
0.2120
2
SSGR
0.6320
0.6320
3
LSGR
0.7120
0.7120
4
PTP
0.9400
0.9400
5
MTPWP
−0.6000
0.6000
6
MTPWR
−0.6680
0.6680
7
SCP
0.9280
0.9280
8
PUR
0.4880
0.4880
9
PIPA
0.8920
0.8920
10
SGRA
0.8910
0.8910
11
PTNS
−0.7000
0.7000
12
EI
−0.8320
0.8320
13
TOOP
−0.7880
0.7880
14
RI
−0.9000
0.9000
15
OP
−0.7520
0.7520The simulations show that the VS-RBF model can be used to quickly get the influence curve of each index on the sortie generation capacity, which can be used to directly determine the factors which have a great influence on the sortie generation capacity.Table9 shows the fitting slope and absolute value of slope for each index. The greater the absolute value of the slope, the greater the influence of the index on the sortie generation capacity. Otherwise, the smaller the absolute value of the slope, the smaller the influence of the index on the sortie generation capacity is.Figure15 shows that when PTP, SCP, PIPA, SGRA, or RI changes, the change of sortie generation capacity is great. So these indexes should be allocated at first in planning the scheme and guaranteed to remain the good state in the exercise or war, which will keep the sortie generation capacity high.At the same time, Figure15 also shows that when ESGR, SSGR, MTPWP, MTPWR, or PUR changes, the change of sortie generation capacity is small. The requirements of these indexes can be lightly reduced, which will ensure that the limited resources focus on those indexes with great influence on the sortie generation capacity and achieve the reasonable allocation of resources with low cost.


## 6.1. Analysis for Sortie Generation Capacity of Carrier Aircrafts

The nonlinear VS-RBF model of the indexes and evaluation result not only can be used for evaluating a given sortie generation scheme but also can further determine the effect of various indexes on the evaluation result. The scheme with basic configuration of surge operation of “Nimitz” carrier in 1997 is selected as a reference scheme, which is shown in Table8.Table 8
The reference scheme.
No.
Index
Unit
Basic configuration
Normalized result
1
ESGR
Sortie
30
0.76
2
SSGR
Sortie/day
250
0.87
3
LSGR
Sortie/day
200
0.79
4
PTP
%
80
0.91
5
MTPWP
%
11
0.64
6
MTPWR
%
9
0.58
7
SCP
%
85
0.87
8
PUR
Sortie/day
2.5
0.94
9
PIPA
%
90
0.95
10
SGRA
Sortie/day
6
0.86
11
PTNS
Minute
30
0.83
12
EI
Minute
1
0.94
13
TOOP
%
1
0.87
14
RI
Minute
1.5
0.74
15
OP
%
3.3
0.90On this basis, the indexxii=1,2,⋯,15 is, respectively, adjusted as 0, 0.25, 0.5, 0.75, 1, and sortie generation capacity of the carrier aircraft is evaluated.The 15 indexes are adjusted, respectively, and the influence curve and slope of each index on the sortie generation capacity are obtained, which are shown in Figure15 and Table 9.Figure 15
The influence curves of 15 indexes on the sortie generation capacity.
(a)
Sortie generation rate capacity
(b)
Availability capacity
(c)
Tasks completion capacity
(d)
Support, ejection, and recovery capacitiesTable 9
The slopes of 15 indexes.
No.
Index
Slope
Absolute value of slope
1
ESGR
0.2120
0.2120
2
SSGR
0.6320
0.6320
3
LSGR
0.7120
0.7120
4
PTP
0.9400
0.9400
5
MTPWP
−0.6000
0.6000
6
MTPWR
−0.6680
0.6680
7
SCP
0.9280
0.9280
8
PUR
0.4880
0.4880
9
PIPA
0.8920
0.8920
10
SGRA
0.8910
0.8910
11
PTNS
−0.7000
0.7000
12
EI
−0.8320
0.8320
13
TOOP
−0.7880
0.7880
14
RI
−0.9000
0.9000
15
OP
−0.7520
0.7520The simulations show that the VS-RBF model can be used to quickly get the influence curve of each index on the sortie generation capacity, which can be used to directly determine the factors which have a great influence on the sortie generation capacity.Table9 shows the fitting slope and absolute value of slope for each index. The greater the absolute value of the slope, the greater the influence of the index on the sortie generation capacity. Otherwise, the smaller the absolute value of the slope, the smaller the influence of the index on the sortie generation capacity is.Figure15 shows that when PTP, SCP, PIPA, SGRA, or RI changes, the change of sortie generation capacity is great. So these indexes should be allocated at first in planning the scheme and guaranteed to remain the good state in the exercise or war, which will keep the sortie generation capacity high.At the same time, Figure15 also shows that when ESGR, SSGR, MTPWP, MTPWR, or PUR changes, the change of sortie generation capacity is small. The requirements of these indexes can be lightly reduced, which will ensure that the limited resources focus on those indexes with great influence on the sortie generation capacity and achieve the reasonable allocation of resources with low cost.


## 7. Conclusions

This paper proposes a VS-RBF network with the fast learning rate aimed at structuring optimization design and parameter learning algorithm of RBF. At the same time, the convergence analysis of VS-RBF is given to ensure the accuracy of the RBF neural network. By comparing with other self-organizing RBF neural networks, the following conclusions are obtained:(1)
VS-RBF can automatically adjust the structure of the RBF neural network according to the complexity of the object. It obtains a compact RBF neural network and strong dynamic response capability
(2)
The learning algorithm of parameters adapted to the structure adjustment is obtained. The fast learning rate algorithm and the robust regression algorithm improve the convergence speed of the RBF neural network
(3)
The convergence analysis of the VS-RBF neural network is given. And VS-RBF has good convergence and stability
(4)
Compared with several other self-organizing RBF neural networks, the proposed VS-RBF has the advantages of a compact structure, strong approximation ability, and self-organization ability. The evaluation of sortie generation capacity of the carrier aircraft with VS-RBF provides technical support for the evaluation of complex systemsTo sum up, the VS-RBF proposed in this paper can effectively solve the problem of structure design and parameter learning algorithm of the RBF neural network. And the approximation of the typical nonlinear function and the evaluation of the sortie generation capacity of the carrier aircraft are realized.



---

*Source: 6950124-2018-10-22.xml*