# Solution of Fractional Order System of Bagley-Torvik Equation Using Evolutionary Computational Intelligence

Authors: Muhammad Asif Zahoor Raja; Junaid Ali Khan; Ijaz Mansoor Qureshi
Journal: Mathematical Problems in Engineering
 (2011)

---

## Abstract

A stochastic technique has been developed for the solution of fractional order system represented by Bagley-Torvik equation. The mathematical model of the equation was developed with the help of feed-forward artificial neural networks. The training of the networks was made with evolutionary computational intelligence based on genetic algorithm hybrid with pattern search technique. Designed scheme was successfully applied to different forms of the equation. Results are compared with standard approximate analytic, stochastic numerical solvers and exact solutions.


---

## Body

## 1. Introduction

The Bagley-Torvik equation is originally formulated in the studies on behavior of real material by use of fractional calculus [1, 2].  It has raised its importance since then in many engineering and applied sciences applications.  In particular, the equation with 1/2-order derivative or 3/2-order derivative can model the frequency-dependent damping materials quite satisfactorily. It can also describe motion of real physical systems, the modeling of the motion of a rigid plate immersed in a Newtonian fluid and a gas in a fluid, respectively [3, 4]. Fractional dynamic systems have found many applications in various problems such as viscoelasticity, heat conduction, electrode-electrolyte polarization, electromagnetic waves, diffusion wave, control theory, and signal processing [1–10].The generic form of Bagley-Torvik equation can be written as(1.1)Ad2y(t)dt2+Bd3/2y(t)dt3/2+C[y(t)]n=f(t),0<t≤T
with initial conditions given as(1.2)dkdtky(0)=ck,k=0,1
whereas boundary condition at t=t0, for 0<t0≤T,  is written as(1.3)dkdtky(t0)=bk,k=0,1,
where n  is the nonlinear operator of the equations, y(t) is the solution of the equation, A,B, and C are constant coefficients, T is the constant representing the span of inputs within the close interval [0,T], and ck, bk are the constants.The general response expression (1.1) contains parameters that can be varied to obtain various responses. In the case of n=1, A=M, the mass of thin rigid plate, C=K, the stiffness of the spring, B=2Sμρ, where S is area of plate immersed in Newtonian fluid, μ is viscosity, and ρ is the fluid density, then (1.1) represents the motion of a large thin plate in a Newtonian fluid [3]. Similarly, linearly damped fractional oscillator with the damping term has a fractional derivative of order ν=1.5, and it can be represented by Bagley-Torvik [3, 11].The problem to develop the numerical solvers to find the solution of Bagley-Torvik fractional differential equation has attracted much attention and has been studied by many authors. In this regard an approximate analytical solution of the equation was derived using Adomian decomposition method [12, 13], He’s variational iteration method [14], Taylor collocation method [15]. Diethelm transformed the equation into first-order coupled fractional differential equation and solved the problem with Adams predictor and corrector approach [16]. Podlubny used successive approximation method to solve the equation and recently applied the Matrix Approach to Discretization of Fractional Derivatives for the same problem [3, 17]. However, there has been no advancement to apply the stochastic numerical solvers to find the solution of the equation.In this paper, investigation and analysis are carried out for successful modeling of the equation using feed-forward artificial neural networks (ANNs). The linear combination of these networks is defined by an unsupervised error. This error is minimized with the help of appropriate unknown parameter, that is, weights. The training of weights is conducted with stochastic optimization techniques based on genetic algorithm hybridized with pattern search technique. It is well known that these techniques are reliable, successful, and efficient to avoid the possibility to get stuck in local minima or diverge to unstable situations and also maintain the diversity throughout [18, 19].


## 2. Basic Definition

In this section, some definitions and relations are given, which will be used in the proceeding sections. The definitions of fractional integral and derivative have been provided in the fractional calculus literature in a variety of ways, including Riemann-Liouville, Caputo, Erdélyi-Kober, Hadamard, Grünwald-Letnikov, and Riesz type. Equivalence of these definitions on some function has also been established [20–22]. However, in this paper, Riemann-Liouville definition for fractional derivative is used with lower terminal at 0.Definition 2.1 (Riemann-Liouville fractional order derivatives).
The Riemann-Liouville derivative ofRLDν of fractional order ν is normally provided in the literature as [3, 22]
(2.1)DνRLf(t)=1Γ(m-ν)dmdtm∫0tf(τ)(t-τ)1+ν-mdτ(m-1<ν≤m),
where ν∈ℛ, m∈N, f is the continuous function, and Γ(x) is the gamma function defined by
(2.2)Γ(x)=∫0∞e-ttx-1dt,(R(x)>0).Definition 2.2 (Mittag-Leffler function).
The Mittag-Leffler function (MLF) is one of the main functions that has widespread use in the field of fractional calculus. Specially, its importance is realized in providing the analytic solution for differential equation of fractional order.
The definition of classical MLF function in two parametersα and β  is given as [3, 23]
(2.3)Eα,β(t)=∑k=0∞tkΓ(αk+β)(α>0,β>0).
It reduces to standard MLF function of one parameter by taking β=1.


## 3. Mathematical Modeling

In this section mathematical modeling of Bagley-Torvik equations with feed-forward artificial neural network is presented.The solutiony of the fractional differential equation along with its ν arbitrary order derivative dνy/dtν can be approximated by the following continuous mapping as in neural network methodology [24–26]:(3.1)ŷ(t)=∑i=1mαig(wit+βi),dνŷdtν=∑i=1mαidνdtνg(wit+βi),
where αi,wi, and βi  are bounded real-valued adaptive parameters, m is the number of neurons, and f is the activation function taken as exponential function instead of normally taken log-sigmoid due to nonavailability of its fractional derivative. Fractional differential equation neural networks (FDE-NNs) for expression (1.1) can be formulated as(3.2)ŷ(t)=∑i=1mαiewit+βi,(3.3)d2dt2ŷ(t)=∑i=1mαiwi2ewit+βi,(3.4)d3/2dx3/2ŷ(t)=∑i=1mαieβit-3/2E1,-1/2(wit).The mathematical model for (1.1) can be the linear combinations of the networks represented above. The FDE-NN architecture formulated for Bagley-Torvik equation can be seen in Figure 1. It is clear that the solution y can be approximated with ŷ subject to finding appropriate unknown weights.Figure 1
FDE-NN architecture of Bagley-Torvik equation.


## 4. Evolutionary Computational Intelligence

Evolutionary computational intelligence uses natural evolution as an optimization mechanism for solving various problems. The main objective of the scheme is to find a good solution to a problem in a large search space of candidate solutions. In this section, the methodology for learning of the unknown weights of networks is given. Our intent is to use evolutionary computation based on Genetic Algorithm (GA) hybrid with Pattern Search technique (PS) to solve such problem. The main advantages of the GA algorithm are robustness in controlling parameters, not to get stuck in local minima, avoid divergence and efficient compared to other mathematical algorithms and heuristic optimization techniques [18, 19].Pattern search algorithm is a kind of aggressive optimization method, belonging to class of direct search method [27, 28]. It can find the optimum values using stochastic searching technology based on scaled and translated integer lattice. Pattern search algorithm has a strong ability to find the local optimistic results [29, 30].So, after the detailed study of the literature about evolutionary computation techniques [31, 32], GAs hybrid with PS algorithm are used by seeing its strengths and applicability. The general flowchart showing the process of evolutionary algorithm is given in Figure 2.Figure 2
Generic flow diagram of evolutionary computation algorithm.Randomly generated initial population consists of finite set of chromosomes or individual, and each chromosome consists of as many numbers of genes as the number of unknown parameters, that is,  weights in the neural networks representing the equation. The fitness of the individual is calculated based on an unsupervised error defined by linear combination of the differential equation neural networks. The fitness function to be minimized is defined as the mean of sum of square errors(4.1)ej=e1j+e2jj=1,2,…,
where j is the generation index and e1j is given as(4.2)e1j=1s∑i=1s(Ad2ŷ(ti)dt2+Bd3/2ŷ(ti)dt3/2+C[ŷ(ti)]n-f(ti))|j,
where s is the total number of steps, each step is randomly taken between (0,T). The greater the value of s the more will be the accuracy,  but increase, in computational complexity of the algorithm.Similarly,e2j, linked with initial and boundary conditions is finally written as (4.3)e2j=12∑k=01[(dkdtkŷ(0)-ck)2+(dkdtkŷ(t0)-bk)2]|j.
It is quite evident that the weights for which fitness function, ej, approaches zero, the solution y(t) of the equation is well approximate by ŷ(t)given in (3.2).Evolutionary algorithm is given in the following steps.Step 1 (initialized population).
Randomly generate bounded real values to form initial population withN number of the chromosomes or individuals. Create M subpopulation each with N/M individuals. Each individual consists of as many number of genes as the number of unknown parameter in neural network. The better search space of algorithm is subjected to enough spread of initial population.Step 2 (initialization).
Following parameter values are initialized for algorithm execution. Set the number of variable equivalent to element in the individual. Set the number of generations and the fitness limit. Set the elite count “2” and value of crossover fraction as 0.80 for reproduction. Set migration in both forward and backward direction. Start generation count, and so forth.Step 3 (fitness evaluation).
Calculate the fitness for each individual using the expressions (4.1) to (4.3).Step 4 (ranking).
Rank each individual of the populations on the basis of minimum fitness values.Step 5 (termination criteria).
Terminate the algorithm if either predefined fitness value, that is, MSE 10−4 is achieved or number of generation is complete. If termination criterion fulfilled, then go to Step 8, else continue.Step 6 (reproduction).
Create the next generation using. Crossover: Call for scattered function, Mutation: Call for Adaptive Feasible function, Selection: Call for Stochastic Uniform function, and Elitism. Repeat the procedure from Step3 to Step 6 until total number of generation are complete.Step 7.
Store the global best individual of this run. Repeat the Steps2 to 6 to have sufficient numbers of global best individual for better statistical analysis.Step 8 (refinement).
Pattern search algorithm is used for further fine-tuning by taking the best individual of GAs as start point of the algorithm. MATLAB Genetic algorithm and direct search tool box are used for pattern search algorithm. Store the refined global best individual for statistical analysis later.


## 5. Simulation and Results

In this section, the simulation results are presented for three different fractional order systems represented by Bagley-Torvik equation. In order to prove the applicability of the designed scheme for such systems, we have considered the equation in example I for which exact solution is available. Moreover, the statistical analysis of results is also carried out to verify and validate reliability of the algorithm. In example II, we have tested the proposed methodology on Bagley-Torvik equation for which PMA technique fails to determine the results due to nonhomogeneous initial conditions. In example III, the strength and weakness of our scheme are analyzed by taking such equation for which the exact solution is not known; however, its numerical solution is available.


### 5.1. Example I

Consider the Bagley-Torvik equation [14, 33](5.1)d2y(t)dt2+d3/2y(t)dt3/2+y(t)=2+4t/π+t2,0<t≤1,
subjected to the initial condition and boundary condition as (5.2)y(0)=ddty(0)=0,y(1)=1,ddty(1)=2.
The exact solution of the equation is given as(5.3)y(t)=t2.Mathematical modeling of the above equation is done with Fractional differential equation neural network (FDE-NN) represented by (3.2) to (3.4) by taking 10 number of neurons resulting in 30 number of unknown parameters or weights. These weights are restricted to real numbers between −10 to 10. The initial population consists of a set of 200 individuals, which is divided into 10 subpopulations each with 20 numbers of individuals. Each individual consists of 30 genes, which is equivalent to number of unknown parameters of FDE-NN. Input of the training set is taken from t∈(0,1). Therefore the fitness function is formulated as (5.4)ej=12∑i=12[d2ŷ(ti)dti2+d3/2ŷ(ti)dti3/2+ŷ(ti)-2-4tiπ-ti2]2+14{[ŷ(0)]2+[ŷ(1)-1]2+[ŷ′(0)]2+[ŷ′(1)-2]2}|jj=1,2,3,…,
where j is the number of generations, ŷ, d2ŷ/dt2, and d3/2ŷ/dt3/2 are networks provided in (3.2), (3.3), and (3.4), respectively. The scheme runs iteratively in order to compute the minimum of fitness function, ej, with a termination criteria as 3000 number of generations executed or value of the fitness function  ej≤10-4   whichever comes earlier.  The best individual found by global search technique is passed to a rapid local search method as a start point for further refinements of the results. The parameter settings used in genetic algorithm (GA) and Pattern Search algorithm (PS) are provided in Table 1. One of the best sets of weights of FDE-NN learned stochastically by PS, GA, and Genetic algorithm hybrid with PS (GA-PS) algorithms is provided in Table 2. Using these weights in (3.2) one can find the solution to this problem for any input time t between 0 and 1. The solutions obtained from the chromosomes given in Table 2 are presented in Table 3.Table 1
Parameters setting for algorithms.
GAPSParametersSettingParametersSettingPopulation CreationConstrain dependantPoll methodGPS Positive basis 2NScaling factionRankPolling orderConsecutiveSelection functionStochastic UniformMesh AcceleratorOffCrossover fraction0.75Mesh Rotae/ScaleOnCrossover  fuctionScatteredMesh expansion factor2.0MutationAdaptive feasibleMesh Contraction factor0.5lElite count2Cache ToleranceEPSInitial Penalty10Initial Penalty10Penalty factor100Penalty factor100Migration fraction0.2Max Iteration3000Migration interval20Max. function evolutions60000Table 2
FDE-NN weights trained by different solvers for example I.
iwiαiβiPSGAGA-PSPSGAGA-PSPSGAGA-PS10.2272490.0143130.0768130.731960−1.16770−1.167700.7693160.0468080.04290520.031563−0.15713−0.157130.7085490.5547050.3047050.871064−0.73877−0.7387730.121262−0.299253.669499−7.73928−0.16469−0.164690.504503−0.12463−8.1246342.5830990.0004650.000465−4.61892−0.74905−0.74905−4.777910.0508330.05083350.5644991.1691051.1691050.941493−0.14210−0.142100.702424−0.00566−0.0056660.659389−1.30293−1.302930.9641770.3347890.2097890.7480120.2482670.24825270.434814−0.40253−0.902530.9880580.2846070.2846070.1998680.6484940.6484948−0.781650.906189−5.093810.711952−0.077493.9225000.9188570.372823−7.627179−0.193010.348033−4.644150.5951460.0768740.0143741.270213−1.37687−9.37687100.1920050.9750220.9437420.2235020.2716790.2716790.8730721.5023321.502332Table 3
Comparison of results for the solution of example I.
Ty(t)ŷ(t)|y(t)-ŷ(t)|PSGAGA-PSPMAVIMGA-PSPMAVIM0.00.002.73e-020.067300.033460.000000.000003.34e-20.000000.000000.10.010.0123920.089250.044379.29e-30.010053.43e-27.04e-45.48e-50.20.040.0496460.128020.073380.038920.040633.33e-21.07e-36.31e-40.30.090.1111430.183750.120440.088730.092663.04e-21.26e-32.66e-30.40.160.1960740.256760.185730.158670.167482.57e-21.32e-37.48e-30.50.250.3033740.347560.269610.248710.266791.96e-21.28e-31.67e-20.60.360.4316200.456840.372620.358830.392771.26e-21.16e-33.22e-20.70.490.5788800.585480.495490.489030.548065.49e-39.67e-45.80e-20.80.640.7425310.734550.639110.639280.735888.80e-47.14e-49.58e-20.90.810.9190150.905300.804570.809580.960075.42e-34.12e-41.50e-11.01.001.1035351.099220.993080.999931.225196.91e-36.83e-52.25e-1The results of other numerical solvers like Podlubny matrix approach (PMA) [3, 17] and reported results of He’s variational iteration method (VIM) [14] for inputs between 0 and 1 with a step of 0.1 are also provided in Table 2. In order to determine the results by PMA technique, the library functions provided by Podlubny at MATLAB central file exchange are used [34]. Following parameter setting is employed for PMA technique as follows: (i)
the value of fractional order derivativeν=1.5;(ii)
set the value for constant coefficients,A=B=C=1;(iii)
the discretization step is taken ash=0.01;(iv)
total number of time steps = 100.It can be seen that the result obtained by our scheme is in good agreement with the state of art numerical solvers.Moreover, the behavior of the derivative of the solution is also analyzed with the same individual as given in Table2. The results for derivative of the solutions are provided for inputs between 0 and 1 with a step of 0.1 by PS, GA, and GA-PS algorithms in Table 4. It is quite evident that the accuracy level lies in the range of 10−2 to 10−3.Table 4
Comparison of results for derivative of the solution of example I.
Tŷ′(t)ŷ′(t)|y′(t)-ŷ′(t)|PSGAGA-PSPSGAGA-PS0.00.000.002670.135260.0176872.67e-31.80e-21.76e-20.10.200.249060.303500.1998584.90e-29.07e-21.41e-40.20.400.494940.472110.3802849.49e-21.20e-21.97e-20.30.600.733660.643010.5613240.133661.34e-23.86e-20.40.800.963180.818050.7450500.163183.14e-25.49e-20.51.001.180470.999090.9333590.180472.07e-26.66e-20.61.121.381251.187981.1280470.181256.03e-27.19e-20.71.401.559601.386591.3308500.159601.80e-26.91e-20.81.601.707511.596851.5434790.107519.07e-45.65e-20.91.801.814241.820731.7676290.014241.20e-23.23e-21.02.001.865492.060332.0049870.134501.34e-24.98e-3Before moving toward the statistical analysis of our designed scheme, it is necessary to mention that entire surrogate model of the algorithm is based on Mittag-Leffler function. Its generic form is given in expression (2.3). However, the difficulty faced in calculation of MLF function is due to its complex nature. This issue is tackled in our simulation by use of MATLAB code provided by Podlubny at MATLAB central file exchanges to determine the value of MLF function for desired inputs [35].Training of the weights for neural networks of the equation is highly stochastic nature. It is necessary to have statistical analysis of the results obtained by FDE-NN algorithms. 125 total independents runs are executed for each stochastic optimizer. These optimizers are PS, GA, and GA hybrid with PS (GA-PS). The values of the absolute error are provided in Table5 for input at 0 and 1. Mean and Standard deviation (STD) were also calculated for the best 100 runs. The value of unsupervised error, ej, of the equation in ascending order is plotted against each independent run of the algorithm in Figure 3.  Moreover, the values of the absolute error for the solution as well as its derivative at 0 and 1 are plotted in Figure 4. It can be seen from Figures 3 and 4 and Table 5 that the best results are obtained by the use of GA-PS algorithm.Table 5
Statistical analysis based on the value of absolute unsupervised error for example I.
T01FDE-NNPSGAGA-PSPSGAGA-PSBest|y(t)-ŷ(t)|2.36e-24.87e-23.11e-32.35e-34.61e-41.06e-4|y´(t)-ŷ′(t)|2.91e-26.13e-68.31e-49.59e-31.48e-32.62e-2Worst|y(t)-ŷ(t)|1.002320.186950.171930.826037.19e-26.87e-2|y´(t)-ŷ′(t)|0.657256.63e-25.82e−21.003570.124910.11201Mean|y(t)-ŷ(t)|0.327890.146350.121110.248343.36e-23.31e-2|y´(t)-ŷ′(t)|0.272332.99e-22.90e-20.409127.54e-26.17e-2STD|y(t)-ŷ(t)|0.246793.25e-24.14e-20.203912.12e-22.00e-2|y´(t)-ŷ′(t)|0.190511.78e-21.70e-20.287783.35e-23.37e-2Figure 3
Comparison of different stochastic numerical solvers.Comparison of FDE-NN for different stochastic numerical solvers, (a) and (b) are for the value of|y(t)-ŷ(t)| at t=0 and t=1 respectively, while (c) and (d) are for the value of |y´(t)-ŷ′(t)| at t=0 and t=1, respectively.
(a)(b)(c)(d)


### 5.2. Example II

Another fractional order system of Bagley-Torvik equation is taken to investigate the strength and weaknesses of the proposed stochastic algorithm. Consider the Bagley-Torvik equation [13, 15](5.5)Ad2y(t)dt2+Bd3/2y(t)dt3/2+Cy(t)=C(t+1),0<t≤1
with initial condition as y(0)=y′(0)=1. It has the exact solution given as(5.6)y(t)=t+1.
This problem can be made simpler by use of following substitution:(5.7)Y(t)=y(t)-1-t.
Equation (5.5) will be (5.8)Ad2Y(t)dt2+Bd3/2Y(t)dt3/2+CY(t)=0,Y(0)=Y′(0)=0.
This problem is solved on the same methodology adopted for previous example. However, the problem-specific fitness function for (5.5) by taking the values of constant coefficient as unity can be formulated as(5.9)ej=15∑i=15[d2ŷ(ti)dti2+d3/2ŷ(ti)dti3/2+ŷ(ti)-(ti+1)]2+12[ŷ(0)+ddtŷ(0)]2|j,j=1,2,3,…,
where j is the generations index, ŷ, d2ŷ/dt2, and d3/2ŷ/dt3/2 are FDE-NN networks provided in (3.2), (3.3), and (3.4), respectively. The one of unknown set of weights of FDE-NN networks trained stochastically by using PS, GA, and GA-SA is given in Table 6. These weights are used in expression (3.2) for finding the solution of the equations for some inputs between 0 and 1.The results are summarized in Table 7.  It can be inferred from the table that the best results are obtained by FDE-NN networks trained by GA-PS algorithm. The average accuracy achieved in the given scheme lies in the range of 10−3 to 10−4.Table 6
The weights trained for neural network modeling for example II.
IwiαiβiPSGAGA-PSPSGAGA-PSPSGAGA-PS1−7.52590.450940.450940.46874−0.6587−0.65870.769316−1.19230.480452−7.7316−0.0968−0.09680.90230−0.5674−0.56740.8710640.677350.844483−5.9519−0.3544−0.3544−6.69400.515340.515340.504503−0.74610.138034−1.58971.758591.758840.43284−0.1467−0.1467−4.777910.337950.5799450.20093−0.8300−4.58000.138290.141950.141710.702424−1.75440.6626460.720300.057500.057500.379760.498430.490620.748012−0.88220.688447−5.45541.105071.107020.371351.511261.509300.199868−0.17090.003858−7.46831.558241.558246.05083−0.7505−0.75050.918857−1.0854−5.94919−7.87360.240570.240570.955880.220790.220791.2702131.829210.8359310−6.9500−0.1263−0.12630.59958−0.2013−0.20130.8730720.473010.63207Table 7
Comparison of results for example II.
Ty(t)ŷ(t)|y(t)-ŷ(t)|PSGAGA-PSPSGAGA-PS0.01.000.6916041.0248621.0160070.308392.30e-21.60e-20.11.100.6237491.1212061.1047330.476252.69e-24.73e-30.21.200.8596971.2208211.1998040.340303.13e-21.95e-40.31.301.1222661.3230411.2993330.177733.45e-26.66e-40.41.401.3377361.4269521.4016296.22e-23.45e-21.62e-30.51.501.5018391.5313301.5049721.83e-32.87e-24.97e-30.61.601.6289071.6345691.6074292.89e-21.36e-27.42e-30.71.701.7344581.7345911.7067053.44e-21.49e-26.70e-30.81.801.8305201.8287381.7999873.05e-22.30e-21.27e-50.91.901.9253081.9136401.8837852.53e-22.69e-21.62e-21.02.002.0240991.9850571.9537622.40e-23.13e-24.62e-2The results computed for the algorithm are also plotted graphically in Figure5. In Figure 5(a), the comparison of the results is given from exact solution on interval (0, 1) with step of 0.1. Moreover, the results are also plotted for larger interval (0, 4) in Figure 5(b) by using same weights, and it can be seen that error is starting to grow for inputs larger than 1 and it is tending to for inputs greater than 2. One can increase the accuracy slightly on large intervals by taking large number of neurons in FDE-NN networks or by training of networks for large input span or increasing the number of steps.  However, it is worth mentioning that in these cases computational complexity will increase exponentially [36, 37].Comparison of the results (a) is for interval (0, 1) and (b) is for interval (0, 4).
(a)(b)


### 5.3. Example III

In this example the fractional order system based on Bagley-Torvik equation is taken for which the exact solution is not available. However, its numerical solutions are obtained by different state of art solvers. The designed scheme is applied for such problem in order to analyze further the applicability, reliability, effectiveness, and efficiency.Consider the fractional order system [3, 17](5.10)d2y(t)dt2+12d3/2y(t)dt3/2+12y(t)=f(t),0<t,
where, the forcing function and the initial conditions are given as(5.11)f(t)={8,(0≤t≤1),0,(t>1),y(0)=y′(0)=0.
The example is also solved on the same manner as the previous one. However, the objective function used in this case can be given as(5.12)ej=110∑i=110[d2ŷ(ti)dti2+12d3/2ŷ(ti)dti3/2+12ŷ(ti)-8]2+12[ŷ(0)+ddtŷ(0)]2|j,j=1,2,3,….
The aim of our algorithm is to tune weights for which the value of fitness function as given in (5.12) is at its minimum. One such set of unknown weights of FDE-NN networks trained stochastically using PS, GA, and GA-SA is given in Figures 6(a), 6(b), and 6(c), respectively. The results obtained based on these weights are summarized in Table 8. It also includes the results of PMA algorithm provided with same set of parameters as taken in example I. It can be inferred from the results of the given scheme that it can perform comparable to the specialized state of art numerical solvers.Table 8
Comparison of results for the solution of the equation in example III.
TPMAŷ(t)PSGAGA-PS0.00.0000000.0164280.0015340.0014370.10.0327340.1010480.0372050.0361020.20.1333000.2763680.1324770.1328430.30.2966540.3278190.2992320.2925620.40.5190240.4423890.5125090.5204110.50.7971070.6829390.7995980.8012900.61.1278220.1236481.1289621.1269230.71.5081990.1602371.5020851.5048720.81.9353172.0098711.9450761.9282390.92.4062762.5364792.3609272.3827381.02.9181752.9783562.9081202.910234The weights trained for FDE-NN networks for example III, (a) is for PS, (b) is for GA, and (c) is for GA-PS algorithms.
(a)(b)(c)


### 5.4. Some Further Discussion

In this section, some necessary further discussion is added to elaborate the results. The advantages and limitation of the proposed method is also presented, so that one should know about its effectiveness and reliability.In the design scheme, the number of stepssis taken as 2, 5, and 10, respectively, for corresponding (5.4), (5.9), and (5.12) in the interval (0, 1) randomly.  It means that the mesh size used in our scheme is 0.5, 0.2, and 0.1 for examples I, II, and III, respectively. On the other hand in PMA technique the results were obtained with number of step equals to 100, using mesh size h=0.01 in the interval (0, 1). It is well known that, for the numerical method, decreasing the mesh size increases the accuracy of algorithm, but at the cost of extensive computational budget. It can be inferred from Tables 3 and 8 that the results of our scheme are close to PMA method with less number of steps. Another advantage of our algorithm is that once weights are obtained by the training of FDE-NN networks, the solution of the equation can be obtained for any continuous input time within the interval (0, 1), whereas PMA approach results are available only on predefined discrete grid of inputs within the interval (0, 1). Therefore, if one desired the result on any new inputs timing, the whole cumbersome iterative procedure has to run.It can be seen from the result given in Table3 that on average the value of absolute error for GA-PS lies in the range 10−2 to 10−3, whereas reported results of VIM [14] have good approximation for the exact solution in closed vicinity of initial guess while the value of absolute error is going to rise with increase of time, for example, at t=1, it is 0.225. However, the results by PMA method are better than those of GA-PS algorithm, but it has limitation as discussed earlier.Moreover, it is a bit tricky to decide the appropriate number for steps in optimization of weights by our scheme. The decision of the input span interval and step is made on compromise between the computation complexity and accuracy of the algorithm. The time taken for the computation is measured in order to solve Bagley-Torvik equation with FDE-NN networks optimized with GA algorithm (200 individuals, 3000 generations) and PS technique (3000 generation). Optimization of weights with the help of fitness function provided in (5.12) the algorithm required executing  MLF function 20000 times for single generation, whereas the single generation using fitness functions in (5.4) and (5.9) required 10000 and 4000 time execution of MLF function, respectively. The average total time taken by the algorithm for its single run and time taken exclusively by MLF function is provided in Table 9.  It can be seen from Table 9 that about 80% of execution time is spent on calculation of MLF function only. The time analysis provided in this paper is carried out using Dell Latitude D630 laptop with Intel(R) Core(TM) 2 Duo CPU T9300, 2.50 GHz, and MATLAB version R2008b.Table 9
Computational complexity of the algorithms.
Span (0,T)No. of stepsFDE-NNCounts for MLF  (Million)Total timeTime (MLF)(0, 1)02GA12.0000462s371s(0, 1)02PS0.060007.66s4.55s(0, 1)05GA30.00001149s920s(0, 1)05PS0.1500015.4s9.34s(0, 1)10GA60.00002297s1852s(0, 1)10PS0.3000031,7s18.6s(0, 1)20GA120.0004593s3695s(0, 1)20PS0.6000060.8s37.5s


## 5.1. Example I

Consider the Bagley-Torvik equation [14, 33](5.1)d2y(t)dt2+d3/2y(t)dt3/2+y(t)=2+4t/π+t2,0<t≤1,
subjected to the initial condition and boundary condition as (5.2)y(0)=ddty(0)=0,y(1)=1,ddty(1)=2.
The exact solution of the equation is given as(5.3)y(t)=t2.Mathematical modeling of the above equation is done with Fractional differential equation neural network (FDE-NN) represented by (3.2) to (3.4) by taking 10 number of neurons resulting in 30 number of unknown parameters or weights. These weights are restricted to real numbers between −10 to 10. The initial population consists of a set of 200 individuals, which is divided into 10 subpopulations each with 20 numbers of individuals. Each individual consists of 30 genes, which is equivalent to number of unknown parameters of FDE-NN. Input of the training set is taken from t∈(0,1). Therefore the fitness function is formulated as (5.4)ej=12∑i=12[d2ŷ(ti)dti2+d3/2ŷ(ti)dti3/2+ŷ(ti)-2-4tiπ-ti2]2+14{[ŷ(0)]2+[ŷ(1)-1]2+[ŷ′(0)]2+[ŷ′(1)-2]2}|jj=1,2,3,…,
where j is the number of generations, ŷ, d2ŷ/dt2, and d3/2ŷ/dt3/2 are networks provided in (3.2), (3.3), and (3.4), respectively. The scheme runs iteratively in order to compute the minimum of fitness function, ej, with a termination criteria as 3000 number of generations executed or value of the fitness function  ej≤10-4   whichever comes earlier.  The best individual found by global search technique is passed to a rapid local search method as a start point for further refinements of the results. The parameter settings used in genetic algorithm (GA) and Pattern Search algorithm (PS) are provided in Table 1. One of the best sets of weights of FDE-NN learned stochastically by PS, GA, and Genetic algorithm hybrid with PS (GA-PS) algorithms is provided in Table 2. Using these weights in (3.2) one can find the solution to this problem for any input time t between 0 and 1. The solutions obtained from the chromosomes given in Table 2 are presented in Table 3.Table 1
Parameters setting for algorithms.
GAPSParametersSettingParametersSettingPopulation CreationConstrain dependantPoll methodGPS Positive basis 2NScaling factionRankPolling orderConsecutiveSelection functionStochastic UniformMesh AcceleratorOffCrossover fraction0.75Mesh Rotae/ScaleOnCrossover  fuctionScatteredMesh expansion factor2.0MutationAdaptive feasibleMesh Contraction factor0.5lElite count2Cache ToleranceEPSInitial Penalty10Initial Penalty10Penalty factor100Penalty factor100Migration fraction0.2Max Iteration3000Migration interval20Max. function evolutions60000Table 2
FDE-NN weights trained by different solvers for example I.
iwiαiβiPSGAGA-PSPSGAGA-PSPSGAGA-PS10.2272490.0143130.0768130.731960−1.16770−1.167700.7693160.0468080.04290520.031563−0.15713−0.157130.7085490.5547050.3047050.871064−0.73877−0.7387730.121262−0.299253.669499−7.73928−0.16469−0.164690.504503−0.12463−8.1246342.5830990.0004650.000465−4.61892−0.74905−0.74905−4.777910.0508330.05083350.5644991.1691051.1691050.941493−0.14210−0.142100.702424−0.00566−0.0056660.659389−1.30293−1.302930.9641770.3347890.2097890.7480120.2482670.24825270.434814−0.40253−0.902530.9880580.2846070.2846070.1998680.6484940.6484948−0.781650.906189−5.093810.711952−0.077493.9225000.9188570.372823−7.627179−0.193010.348033−4.644150.5951460.0768740.0143741.270213−1.37687−9.37687100.1920050.9750220.9437420.2235020.2716790.2716790.8730721.5023321.502332Table 3
Comparison of results for the solution of example I.
Ty(t)ŷ(t)|y(t)-ŷ(t)|PSGAGA-PSPMAVIMGA-PSPMAVIM0.00.002.73e-020.067300.033460.000000.000003.34e-20.000000.000000.10.010.0123920.089250.044379.29e-30.010053.43e-27.04e-45.48e-50.20.040.0496460.128020.073380.038920.040633.33e-21.07e-36.31e-40.30.090.1111430.183750.120440.088730.092663.04e-21.26e-32.66e-30.40.160.1960740.256760.185730.158670.167482.57e-21.32e-37.48e-30.50.250.3033740.347560.269610.248710.266791.96e-21.28e-31.67e-20.60.360.4316200.456840.372620.358830.392771.26e-21.16e-33.22e-20.70.490.5788800.585480.495490.489030.548065.49e-39.67e-45.80e-20.80.640.7425310.734550.639110.639280.735888.80e-47.14e-49.58e-20.90.810.9190150.905300.804570.809580.960075.42e-34.12e-41.50e-11.01.001.1035351.099220.993080.999931.225196.91e-36.83e-52.25e-1The results of other numerical solvers like Podlubny matrix approach (PMA) [3, 17] and reported results of He’s variational iteration method (VIM) [14] for inputs between 0 and 1 with a step of 0.1 are also provided in Table 2. In order to determine the results by PMA technique, the library functions provided by Podlubny at MATLAB central file exchange are used [34]. Following parameter setting is employed for PMA technique as follows: (i)
the value of fractional order derivativeν=1.5;(ii)
set the value for constant coefficients,A=B=C=1;(iii)
the discretization step is taken ash=0.01;(iv)
total number of time steps = 100.It can be seen that the result obtained by our scheme is in good agreement with the state of art numerical solvers.Moreover, the behavior of the derivative of the solution is also analyzed with the same individual as given in Table2. The results for derivative of the solutions are provided for inputs between 0 and 1 with a step of 0.1 by PS, GA, and GA-PS algorithms in Table 4. It is quite evident that the accuracy level lies in the range of 10−2 to 10−3.Table 4
Comparison of results for derivative of the solution of example I.
Tŷ′(t)ŷ′(t)|y′(t)-ŷ′(t)|PSGAGA-PSPSGAGA-PS0.00.000.002670.135260.0176872.67e-31.80e-21.76e-20.10.200.249060.303500.1998584.90e-29.07e-21.41e-40.20.400.494940.472110.3802849.49e-21.20e-21.97e-20.30.600.733660.643010.5613240.133661.34e-23.86e-20.40.800.963180.818050.7450500.163183.14e-25.49e-20.51.001.180470.999090.9333590.180472.07e-26.66e-20.61.121.381251.187981.1280470.181256.03e-27.19e-20.71.401.559601.386591.3308500.159601.80e-26.91e-20.81.601.707511.596851.5434790.107519.07e-45.65e-20.91.801.814241.820731.7676290.014241.20e-23.23e-21.02.001.865492.060332.0049870.134501.34e-24.98e-3Before moving toward the statistical analysis of our designed scheme, it is necessary to mention that entire surrogate model of the algorithm is based on Mittag-Leffler function. Its generic form is given in expression (2.3). However, the difficulty faced in calculation of MLF function is due to its complex nature. This issue is tackled in our simulation by use of MATLAB code provided by Podlubny at MATLAB central file exchanges to determine the value of MLF function for desired inputs [35].Training of the weights for neural networks of the equation is highly stochastic nature. It is necessary to have statistical analysis of the results obtained by FDE-NN algorithms. 125 total independents runs are executed for each stochastic optimizer. These optimizers are PS, GA, and GA hybrid with PS (GA-PS). The values of the absolute error are provided in Table5 for input at 0 and 1. Mean and Standard deviation (STD) were also calculated for the best 100 runs. The value of unsupervised error, ej, of the equation in ascending order is plotted against each independent run of the algorithm in Figure 3.  Moreover, the values of the absolute error for the solution as well as its derivative at 0 and 1 are plotted in Figure 4. It can be seen from Figures 3 and 4 and Table 5 that the best results are obtained by the use of GA-PS algorithm.Table 5
Statistical analysis based on the value of absolute unsupervised error for example I.
T01FDE-NNPSGAGA-PSPSGAGA-PSBest|y(t)-ŷ(t)|2.36e-24.87e-23.11e-32.35e-34.61e-41.06e-4|y´(t)-ŷ′(t)|2.91e-26.13e-68.31e-49.59e-31.48e-32.62e-2Worst|y(t)-ŷ(t)|1.002320.186950.171930.826037.19e-26.87e-2|y´(t)-ŷ′(t)|0.657256.63e-25.82e−21.003570.124910.11201Mean|y(t)-ŷ(t)|0.327890.146350.121110.248343.36e-23.31e-2|y´(t)-ŷ′(t)|0.272332.99e-22.90e-20.409127.54e-26.17e-2STD|y(t)-ŷ(t)|0.246793.25e-24.14e-20.203912.12e-22.00e-2|y´(t)-ŷ′(t)|0.190511.78e-21.70e-20.287783.35e-23.37e-2Figure 3
Comparison of different stochastic numerical solvers.Comparison of FDE-NN for different stochastic numerical solvers, (a) and (b) are for the value of|y(t)-ŷ(t)| at t=0 and t=1 respectively, while (c) and (d) are for the value of |y´(t)-ŷ′(t)| at t=0 and t=1, respectively.
(a)(b)(c)(d)


## 5.2. Example II

Another fractional order system of Bagley-Torvik equation is taken to investigate the strength and weaknesses of the proposed stochastic algorithm. Consider the Bagley-Torvik equation [13, 15](5.5)Ad2y(t)dt2+Bd3/2y(t)dt3/2+Cy(t)=C(t+1),0<t≤1
with initial condition as y(0)=y′(0)=1. It has the exact solution given as(5.6)y(t)=t+1.
This problem can be made simpler by use of following substitution:(5.7)Y(t)=y(t)-1-t.
Equation (5.5) will be (5.8)Ad2Y(t)dt2+Bd3/2Y(t)dt3/2+CY(t)=0,Y(0)=Y′(0)=0.
This problem is solved on the same methodology adopted for previous example. However, the problem-specific fitness function for (5.5) by taking the values of constant coefficient as unity can be formulated as(5.9)ej=15∑i=15[d2ŷ(ti)dti2+d3/2ŷ(ti)dti3/2+ŷ(ti)-(ti+1)]2+12[ŷ(0)+ddtŷ(0)]2|j,j=1,2,3,…,
where j is the generations index, ŷ, d2ŷ/dt2, and d3/2ŷ/dt3/2 are FDE-NN networks provided in (3.2), (3.3), and (3.4), respectively. The one of unknown set of weights of FDE-NN networks trained stochastically by using PS, GA, and GA-SA is given in Table 6. These weights are used in expression (3.2) for finding the solution of the equations for some inputs between 0 and 1.The results are summarized in Table 7.  It can be inferred from the table that the best results are obtained by FDE-NN networks trained by GA-PS algorithm. The average accuracy achieved in the given scheme lies in the range of 10−3 to 10−4.Table 6
The weights trained for neural network modeling for example II.
IwiαiβiPSGAGA-PSPSGAGA-PSPSGAGA-PS1−7.52590.450940.450940.46874−0.6587−0.65870.769316−1.19230.480452−7.7316−0.0968−0.09680.90230−0.5674−0.56740.8710640.677350.844483−5.9519−0.3544−0.3544−6.69400.515340.515340.504503−0.74610.138034−1.58971.758591.758840.43284−0.1467−0.1467−4.777910.337950.5799450.20093−0.8300−4.58000.138290.141950.141710.702424−1.75440.6626460.720300.057500.057500.379760.498430.490620.748012−0.88220.688447−5.45541.105071.107020.371351.511261.509300.199868−0.17090.003858−7.46831.558241.558246.05083−0.7505−0.75050.918857−1.0854−5.94919−7.87360.240570.240570.955880.220790.220791.2702131.829210.8359310−6.9500−0.1263−0.12630.59958−0.2013−0.20130.8730720.473010.63207Table 7
Comparison of results for example II.
Ty(t)ŷ(t)|y(t)-ŷ(t)|PSGAGA-PSPSGAGA-PS0.01.000.6916041.0248621.0160070.308392.30e-21.60e-20.11.100.6237491.1212061.1047330.476252.69e-24.73e-30.21.200.8596971.2208211.1998040.340303.13e-21.95e-40.31.301.1222661.3230411.2993330.177733.45e-26.66e-40.41.401.3377361.4269521.4016296.22e-23.45e-21.62e-30.51.501.5018391.5313301.5049721.83e-32.87e-24.97e-30.61.601.6289071.6345691.6074292.89e-21.36e-27.42e-30.71.701.7344581.7345911.7067053.44e-21.49e-26.70e-30.81.801.8305201.8287381.7999873.05e-22.30e-21.27e-50.91.901.9253081.9136401.8837852.53e-22.69e-21.62e-21.02.002.0240991.9850571.9537622.40e-23.13e-24.62e-2The results computed for the algorithm are also plotted graphically in Figure5. In Figure 5(a), the comparison of the results is given from exact solution on interval (0, 1) with step of 0.1. Moreover, the results are also plotted for larger interval (0, 4) in Figure 5(b) by using same weights, and it can be seen that error is starting to grow for inputs larger than 1 and it is tending to for inputs greater than 2. One can increase the accuracy slightly on large intervals by taking large number of neurons in FDE-NN networks or by training of networks for large input span or increasing the number of steps.  However, it is worth mentioning that in these cases computational complexity will increase exponentially [36, 37].Comparison of the results (a) is for interval (0, 1) and (b) is for interval (0, 4).
(a)(b)


## 5.3. Example III

In this example the fractional order system based on Bagley-Torvik equation is taken for which the exact solution is not available. However, its numerical solutions are obtained by different state of art solvers. The designed scheme is applied for such problem in order to analyze further the applicability, reliability, effectiveness, and efficiency.Consider the fractional order system [3, 17](5.10)d2y(t)dt2+12d3/2y(t)dt3/2+12y(t)=f(t),0<t,
where, the forcing function and the initial conditions are given as(5.11)f(t)={8,(0≤t≤1),0,(t>1),y(0)=y′(0)=0.
The example is also solved on the same manner as the previous one. However, the objective function used in this case can be given as(5.12)ej=110∑i=110[d2ŷ(ti)dti2+12d3/2ŷ(ti)dti3/2+12ŷ(ti)-8]2+12[ŷ(0)+ddtŷ(0)]2|j,j=1,2,3,….
The aim of our algorithm is to tune weights for which the value of fitness function as given in (5.12) is at its minimum. One such set of unknown weights of FDE-NN networks trained stochastically using PS, GA, and GA-SA is given in Figures 6(a), 6(b), and 6(c), respectively. The results obtained based on these weights are summarized in Table 8. It also includes the results of PMA algorithm provided with same set of parameters as taken in example I. It can be inferred from the results of the given scheme that it can perform comparable to the specialized state of art numerical solvers.Table 8
Comparison of results for the solution of the equation in example III.
TPMAŷ(t)PSGAGA-PS0.00.0000000.0164280.0015340.0014370.10.0327340.1010480.0372050.0361020.20.1333000.2763680.1324770.1328430.30.2966540.3278190.2992320.2925620.40.5190240.4423890.5125090.5204110.50.7971070.6829390.7995980.8012900.61.1278220.1236481.1289621.1269230.71.5081990.1602371.5020851.5048720.81.9353172.0098711.9450761.9282390.92.4062762.5364792.3609272.3827381.02.9181752.9783562.9081202.910234The weights trained for FDE-NN networks for example III, (a) is for PS, (b) is for GA, and (c) is for GA-PS algorithms.
(a)(b)(c)


## 5.4. Some Further Discussion

In this section, some necessary further discussion is added to elaborate the results. The advantages and limitation of the proposed method is also presented, so that one should know about its effectiveness and reliability.In the design scheme, the number of stepssis taken as 2, 5, and 10, respectively, for corresponding (5.4), (5.9), and (5.12) in the interval (0, 1) randomly.  It means that the mesh size used in our scheme is 0.5, 0.2, and 0.1 for examples I, II, and III, respectively. On the other hand in PMA technique the results were obtained with number of step equals to 100, using mesh size h=0.01 in the interval (0, 1). It is well known that, for the numerical method, decreasing the mesh size increases the accuracy of algorithm, but at the cost of extensive computational budget. It can be inferred from Tables 3 and 8 that the results of our scheme are close to PMA method with less number of steps. Another advantage of our algorithm is that once weights are obtained by the training of FDE-NN networks, the solution of the equation can be obtained for any continuous input time within the interval (0, 1), whereas PMA approach results are available only on predefined discrete grid of inputs within the interval (0, 1). Therefore, if one desired the result on any new inputs timing, the whole cumbersome iterative procedure has to run.It can be seen from the result given in Table3 that on average the value of absolute error for GA-PS lies in the range 10−2 to 10−3, whereas reported results of VIM [14] have good approximation for the exact solution in closed vicinity of initial guess while the value of absolute error is going to rise with increase of time, for example, at t=1, it is 0.225. However, the results by PMA method are better than those of GA-PS algorithm, but it has limitation as discussed earlier.Moreover, it is a bit tricky to decide the appropriate number for steps in optimization of weights by our scheme. The decision of the input span interval and step is made on compromise between the computation complexity and accuracy of the algorithm. The time taken for the computation is measured in order to solve Bagley-Torvik equation with FDE-NN networks optimized with GA algorithm (200 individuals, 3000 generations) and PS technique (3000 generation). Optimization of weights with the help of fitness function provided in (5.12) the algorithm required executing  MLF function 20000 times for single generation, whereas the single generation using fitness functions in (5.4) and (5.9) required 10000 and 4000 time execution of MLF function, respectively. The average total time taken by the algorithm for its single run and time taken exclusively by MLF function is provided in Table 9.  It can be seen from Table 9 that about 80% of execution time is spent on calculation of MLF function only. The time analysis provided in this paper is carried out using Dell Latitude D630 laptop with Intel(R) Core(TM) 2 Duo CPU T9300, 2.50 GHz, and MATLAB version R2008b.Table 9
Computational complexity of the algorithms.
Span (0,T)No. of stepsFDE-NNCounts for MLF  (Million)Total timeTime (MLF)(0, 1)02GA12.0000462s371s(0, 1)02PS0.060007.66s4.55s(0, 1)05GA30.00001149s920s(0, 1)05PS0.1500015.4s9.34s(0, 1)10GA60.00002297s1852s(0, 1)10PS0.3000031,7s18.6s(0, 1)20GA120.0004593s3695s(0, 1)20PS0.6000060.8s37.5s


## 6. Conclusions

On the basis of the simulations and analysis, it can be concluded that fractional order system of Bagley-Torvik equation can be solved by designed heuristic computational intelligence algorithm. The fractional differential equation neural networks of the equation trained by GA-PS algorithm is the best stochastic optimizer compared to PS, GA algorithms. On the basis of the statistical analysis it can be inferred that our proposed computing approaches are reliable, effective, and easily applicable to such complex fractional order systems. In our future work, we intend to use other biologically inspired computational intelligence algorithms to solve these fractional order systems.



---

*Source: 675075-2011-03-22.xml*