# An Improved Teaching-Learning-Based Optimization with Differential Learning and Its Application

Authors: Feng Zou; Lei Wang; Debao Chen; Xinhong Hei
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

The teaching-learning-based optimization (TLBO) algorithm is a population-based optimization algorithm which is based on the effect of the influence of a teacher on the output of learners in a class. A variant of teaching-learning-based optimization (TLBO) algorithm with differential learning (DLTLBO) is proposed in the paper. In this method, DLTLBO utilizes a learning strategy based on neighborhood search of teacher phase in the standard TLBO to generate a new mutation vector, while utilizing a differential learning to generate another new mutation vector. Then DLTLBO employs the crossover operation to generate new solutions so as to increase the diversity of the population. By the integration of the local search and the global search, DLTLBO achieves a tradeoff between exploration and exploitation. To demonstrate the effectiveness of our approaches, 24 benchmark functions are used for simulating and testing. Moreover, DLTLBO is used for parameter estimation of digital IIR filter and experimental results show that DLTLBO is superior or comparable to other given algorithms for the employed examples.


---

## Body

## 1. Introduction

TLBO is a recently proposed population-based algorithm which simulates the teaching-learning process in a classroom. Rao et al. proposed TLBO for the optimization of mechanical design problems [1] and then applied TLBO to find global solutions to large-scale nonlinear optimization problems [2]. Toĝan [3] presented a design procedure that employed TLBO for the discrete optimization of planar steel frames. In [4], elitist concept is introduced in the TLBO algorithm and its effect on the performance of the algorithm is investigated. The effects of common controlling parameters such as the population size and the number of generations on the performance of the algorithm are also investigated. Degertekin and Hayalioglu [5] applied TLBO in the optimization of four truss structures. Rao and Patel [6] proposed an improved TLBO by introducing the concepts of the number of teachers, adaptive teaching factor, tutorial training, and self-motivated learning. Niknam et al. [7] presented a θ-multiobjective TLBO algorithm to solve the dynamic economic emission dispatch problem, where the optimization procedure considers phase angles attributed to the real value of the design parameters, instead of the design parameters themselves. Zou et al. [8] proposed an improved teaching-learning-based optimization algorithm with dynamic group strategy for global optimization problems. In [9], a ring neighborhood topology is introduced into the original TLBO algorithm to maintain the exploration ability of the population. Details of the conceptual basis of TLBO were given by Waghmare [10]. TLBO has emerged as one of the simplest and most efficient techniques, as it has been empirically shown to perform well on many optimization problems. However, in evolutionary computation research there have been always attempts to improve any given findings further and further.The problem of IIR system identification can also be viewed as a problem of parameter estimation of adaptive digital filtering. For adaptive digital IIR filter design, the objective of the adaptation is to adjust filter coefficients of a digital filter to estimate actual parameter values of an unknown system from its inputs and outputs. However, there are some potential disadvantages with the design of adaptive digital IIR filters [11–13]. A major concern in these disadvantages is the mean square error between the desired response and estimated filter output of digital IIR filter. This error surface is generally nonquadratic and multimodal with respect to the filter coefficients, and hence optimization algorithms are required to find out better global solution. In order to avoid the local optima problem encountered by gradient descent methods in IIR system identification, considering IIR system identification as an optimization problem, recently, evolutionary algorithm and swarm intelligence, such as Genetic Algorithms (GA) [14–16], Simulated Annealing (SA) [17], Tabu Search (TS) [18], Differential Evolution (DE) [19–21], Ant Colony Optimization (ACO) [22], Artificial Bee Colony (ABC) algorithm [23], Particle Swarm Optimizer (PSO) [24, 25], Gravitation Search Algorithm (GSA) [26, 27], and Cuckoo search optimization [28], have been made for studying alternative structures and algorithms for adaptive digital IIR filters.In order to find suitable filter coefficients for the digital IIR filter efficiently, this paper aims to develop a teaching-learning-based optimization (TLBO) approach. First, an improved TLBO with differential learning (DLTLBO) is proposed in paper. In the method, DLTLBO utilizes the integration of the learning strategy based on neighborhood search of teacher phase in the standard TLBO and the differential learning to generate two new mutation vectors, and DLTLBO employs the crossover operation to generate new solutions so as to increase the diversity of the population. Then, to evaluate the optimization performance of the proposed algorithm, the comparison study on 24 benchmark optimization problems is given. Moreover, DLTLBO is applied to the design of the digital IIR filter, and its performance is compared to that of other algorithms. The simulation results show that DLTLBO has better or comparable performance compared to the other algorithms for the employed examples.The remainder of this paper is organized as follows. Section2 describes the teaching-learning-based optimization. Section 3 presents the improved teaching-learning-based optimization with differential learning (DLTLBO). In Section 4, 24 benchmark functions are tested and the experiments are conducted along with statistical tests. DLTLBO is applied to the design of the digital IIR filter in Section 5. Conclusions are given in Section 6.


## 2. Teaching-Learning-Based Optimization

Inspired by the philosophy of the classical school teaching and learning process, Rao et al. [1, 2] first proposed a novel teaching-learning-based optimization (TLBO). In this method, population consists of learners in a class and design variables are courses offered. The brief description of the algorithm is given as follows.


### 2.1. Teacher Phase

During the teacher phase, learner will move their position toward the position of the best learner (Teacher), by taking into account the current mean value of the learners (Mean) that represents the average qualities of all learners in the population. The updating formula of the learning for a learner Xi in teaching phase is given by(1)​newXi=Xi+r×Teacher-TF×Mean,where TF is a teaching factor that decides the value of Mean to be changed and r is a random vector where each element is a random number in the range [0,1]. The value of TF can be either 1 or 2, which is again a heuristic step and is decided randomly with equal probability as(2)TF=round1+rand⁡0,1.Learner modification is expressed as follows:for each learnerXi
TF=round[1+rand⁡(0,1)]newXi=Xi+r×(Teacher-TF×Mean)AcceptnewXi if it gives a better function valueendfor


### 2.2. Learner Phase

During the learner phase, learners increase their knowledge by interacting with each other. A learnerXi learns something new if the other Xj learner has more knowledge than him or her:(3)newXi=Xi+r∗Xi-XjiffXi<fXjXi+r∗Xj-Xiotherwise,where r is a random vector where each element is a random number in the range [0,1], f(Xi) is the fitness value of the learner Xi, and f(Xj) is the fitness value of the learner Xj.Learner modification is expressed as follows:for each learnerXi of the class
Randomly select one learnerXk, such that k≠inewXi={Xi+r∗Xi-Xj,iffXi<fXj; Xi+r∗Xj-Xi,otherwise}AcceptnewX if it gives a better function valueend for


## 2.1. Teacher Phase

During the teacher phase, learner will move their position toward the position of the best learner (Teacher), by taking into account the current mean value of the learners (Mean) that represents the average qualities of all learners in the population. The updating formula of the learning for a learner Xi in teaching phase is given by(1)​newXi=Xi+r×Teacher-TF×Mean,where TF is a teaching factor that decides the value of Mean to be changed and r is a random vector where each element is a random number in the range [0,1]. The value of TF can be either 1 or 2, which is again a heuristic step and is decided randomly with equal probability as(2)TF=round1+rand⁡0,1.Learner modification is expressed as follows:for each learnerXi
TF=round[1+rand⁡(0,1)]newXi=Xi+r×(Teacher-TF×Mean)AcceptnewXi if it gives a better function valueendfor


## 2.2. Learner Phase

During the learner phase, learners increase their knowledge by interacting with each other. A learnerXi learns something new if the other Xj learner has more knowledge than him or her:(3)newXi=Xi+r∗Xi-XjiffXi<fXjXi+r∗Xj-Xiotherwise,where r is a random vector where each element is a random number in the range [0,1], f(Xi) is the fitness value of the learner Xi, and f(Xj) is the fitness value of the learner Xj.Learner modification is expressed as follows:for each learnerXi of the class
Randomly select one learnerXk, such that k≠inewXi={Xi+r∗Xi-Xj,iffXi<fXj; Xi+r∗Xj-Xi,otherwise}AcceptnewX if it gives a better function valueend for


## 3. Proposed Algorithm: DLTLBO

The search in TLBO is mainly based on the global information, and DE is mainly based on the distance and direction information which is a kind of local information [29]. To balance the global and local search ability, a modified interactive learning strategy is proposed in teacher phase. In this method, DLTLBO utilizes the learning strategy based on neighborhood search of teacher phase in the standard TLBO to generate a new mutation vector, while utilizing the differential learning to generate another new mutation vector. Different from our previous method [30], DLTLBO employs the DE crossover operation to generate new solutions so as to increase the diversity of the population. The complete flowchart of the DLTLBO algorithm is shown in Figure 1.Figure 1
Flowchart showing the working of DLTLBO algorithm.In the proposed DLTLBO algorithm, for each learnerXi, its associated mutant vector Ui based on differential mutation can be generated as follows:(4)Ui=Xi+r×NTeacher-TF×NMean,where NTeacher and NMean are the teacher and the mean of the learner’s corresponding neighborhood, respectively. In this paper, ring neighborhood topology with k-neighborhood radius is used to determine in the neighborhood area of each learner.For each learnerXi, its other associated mutant vector Vi can be generated based on differential mutation as follows:(5)Vi=Xr1+F∗Xr2-Xr3.After the mutation phase, crossover operation is applied to each pair of the mutant vectors in order to enhance the potential diversity of the population. That is, the updating formula of the learning for a learnerXi in teacher phase is given as follows:(6)newXij=Vijifrand⁡0,1≤CRUijotherwise.CR is a parameter called crossover probability; Uij is the learners which is calculated according to (4) and Vij is the learners which is calculated according to (5).Finally, the one-to-one greedy selection is employed by means of comparing a parentXi and its corresponding offspring newXi.In learner phase, interactive learning of the original TLBO algorithm in the learner phase is still used in our DLTLBO algorithm. In this learning method, one of the learners randomly learns from the other learner in the population. This learning method can be treated as the global search strategy. As explained above, the pseudocode for the implementation of DLTLBO is summarized in Algorithm1.Algorithm 1:
DLTLBO(⋅).
(
1)  Begin
(2) Initialize N (number of learners), D (number of dimensions) F and CR
(3) Initialize learners X and evaluate all learners X
(4) Donate NTeacher and NMean of the learner’s corresponding neighborhood
(5) while (stopping condition not met)
(6) for each learner Xi of the class    % Teacher phase
(7)      TF = round(1 + rand(0, 1))
(8)      for  j=1:D
(9)       Uij  =  Xij + rand ∗ (NTeacherj-TF∗NMeanj)
(10)     endfor
(11)     Select three learners Xr1Xr2Xr3 randomly from the current class where i≠r1≠r2≠r3
(12)     Vi=Xr1+F∗(Xr2-Xr3)
(13)     for  j=1:D
(14)       if rand(0,1)<CR,  newXij = Vij;  else,  newXij  = Uij;  endif
(15)     endfor
(16)     Accept  newXi  if  f(newXi) is better than f(Xi)
(17) endfor
(18) for each learner Xi of the class    % Learner phase
(19)     Randomly select one learner Xk, such that i≠k
(20)     iff(Xi) better f(Xk)
(21)      for  j=1:D
(22)       newXij = Xij + rand ∗ (Xij-Xkj)
(23)      endfor
(24)     else
(25)      forj=1:D
(26)       newXij = Xij + rand ∗ (Xkj-Xij)
(27)      endfor
(28)     endif
(29)     Accept  newXi if  f(newXi) is better than f(Xi)
(30)  endfor
(31) endwhile
(32) endAs explained in Algorithm1, it can be observed that the probability CR determines adopting the learning strategy of the learners in the teacher phase of the DLTLBO algorithm. That is, a random number r between 0 and 1 is generated for each learner; if r < CR, the learning strategy of the original TLBO will be adopted by the learner; otherwise, the differential learning strategy will be adopted by the learner. This will be helpful in balancing the advantages of fast convergence rate (the attraction of the learning strategy of TLBO learning) and exploration for local areas (neighborhood search strategy and the differential learning) in DLTLBO. By this means, DLTLBO achieves a tradeoff between exploration and exploitation, improves the global search capability, and accelerates the convergence.


## 4. Functions Optimization

In this section, to compare the performance of DLTLBO with some other methods, 7 algorithms are also simulated in the paper. The details of benchmark functions, the simulation settings, and experimental results are given as follows.


### 4.1. Benchmark Functions

The details of 24 benchmark functions are shown in Table1. Among 24 benchmark functions, F1 to F4 are unimodal functions, F5 to F10 are multimodal functions, and F11 to F18 are the rotated version of F3 to F10, respectively. F19 to F24 are taken from the CEC 2008 test suite. The searching range and theory optima for all functions are also shown in Table 1.Table 1
Details of numerical benchmarks used.
Function
Formula
Range
Optima
Sphere
F
1
x
=
∑
i
=
1
D
x
i
2
[
-
100,100
]
0
Quadric
F
2
x
=
∑
i
=
1
D
∑
j
=
1
i
x
j
2
[
-
100
,
100
]
0
Sum square
F
3
x
=
∑
i
=
1
D
i
x
i
2
[
-
10
,
10
]
0
Zakharov
F
4
x
=
∑
i
=
1
D
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
4
[
-
10
,
10
]
0
Rosenbrock
F
5
x
=
∑
i
=
1
D
-
1
100
x
i
2
-
x
i
+
1
2
+
x
i
-
1
2
[
-
2.048,2.048
]
0
Ackley
F
6
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
x
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
x
i
+
e
[
-
32.768
,
32.768
]
0
Rastrigin
F
7
x
=
∑
i
=
1
D
x
i
2
-
10
cos
⁡
2
π
x
i
+
10
[
-
5.12
,
5.12
]
0
Weierstrass
F
8
x
=
∑
i
=
1
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
x
i
+
0.5
-
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
×
0.5
a
=
0.5
b
=
3
k
m
a
x
=
20
[
-
0.5
,
0.5
]
0
Griewank
F
9
x
=
∑
i
=
1
D
x
i
2
4000
-
∏
i
=
1
n
cos
⁡
x
i
i
+
1
[
-
600
,
600
]
0
Schwefel
F
10
x
=
418.9829
D
+
∑
i
=
1
D
-
x
i
sin
⁡
a
b
s
x
i
[
-
500
,
500
]
0
Rotated sum square
F
11
x
=
∑
i
=
1
D
i
y
i
2
y
=
M
×
x
[
-
10
,
10
]
0
Rotated Zakharov
F
12
x
=
∑
i
=
1
D
x
i
2
+
∑
i
=
1
D
0.5
i
y
i
2
+
∑
i
=
1
D
0.5
i
y
i
4 y=M×x
[
-
10
,
10
]
0
Rotated Rosenbrock
F
13
x
=
∑
i
=
1
D
-
1
100
y
i
2
-
y
i
+
1
2
+
y
i
-
1
2
y
=
M
×
x
[
-
2.048
,
2.048
]
0
Rotated Ackley
F
14
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
y
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
y
i
+
e y=M×x
[
-
32.768
,
32.768
]
0
Rotated Rastrigin
F
15
x
=
∑
i
=
1
D
y
i
2
-
10
cos
⁡
2
π
y
i
⁡
+
10
y
=
M
×
x
[
-
5.12
,
5.12
]
0
Rotated Weierstrass
F
16
x
=
∑
i
=
1
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
y
i
+
0.5
-
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
×
0.5
a
=
0.5
b
=
3
k
m
a
x
=
20
y
=
M
×
x
[
-
0.5
,
0.5
]
0
Rotated Griewank
F
17
x
=
∑
i
=
1
D
y
i
2
4000
-
∏
i
=
1
n
cos
⁡
y
i
i
+
1
y
=
M
×
x
[
-
600
,
600
]
0
Rotated Schwefel
F
18
x
=
418.9829
D
+
∑
i
=
1
D
-
y
i
sin
⁡
a
b
s
y
i
y
=
M
×
x
[
-
500
,
500
]
0
Shifted sphere
F
19
x
=
∑
i
=
1
D
z
i
2
-
450 z=x-o o: theshiftedglobaloptimum
[
-
100
,
100
]
o
Shifted Schwefel’s Problem 2.21
F
20
x
=
max
⁡
i
z
i
,
1
≤
i
≤
D
-
450 z=x-o+1 o: the  shifted  global  optimum
[
-
100
,
100
]
o
Shifted Rosenbrock
F
21
x
=
∑
i
=
1
D
-
1
100
z
i
2
-
z
i
+
1
2
+
z
i
-
1
2
+
390 z=x-o+1 o: the  shifted  global  optimum
[
-
100
,
100
]
o
Shifted Rastrigin
F
22
x
=
∑
i
=
1
D
z
i
2
-
10
cos
⁡
2
π
z
i
+
10
-
330 z=x-o o: the  shifted  global  optimum
-
5,5
o
Shifted Griewank
F
23
x
=
∑
i
=
1
D
z
i
2
4000
-
∏
i
=
1
n
cos
⁡
z
i
i
+
1
-
180 z=x-o o: the  shifted  global  optimum
[
-
600
,
600
]
o
Shifted Ackley
F
24
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
z
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
z
i
+
e
-
140 z=x-o o: the  shifted  global  optimum
[
-
32
,
32
]
o


### 4.2. Parameter Settings

In this paper, all the experiments are carried out on the same machine with a Celoron 2.26 GHz CPU, 2 GB memory, and Windows XP operating system with MATLAB 7.9. For the purpose of reducing statistical errors, each function is independently simulated 50 runs, and their mean results are used in the comparison. For all approaches, the population size was set to 50, and the maximal number of cost function evaluations (FEs) is used as termination condition of algorithm, namely, 100,000 for 10D problems. The parameter settings for all algorithms in comparison are extracted from their corresponding literatures and are described as follows:(i)
jDE [31]: F=0.5, CR=0.9;(ii)
SaDE [32]: F~N(0.5,0.3), CR0=0.5, CR~N(CRm,0.1), LP = 50;(iii)
PSO-wFIPS [33]:w = 0.7298;(iv)
PSO-FDR [34]:wmin = 0.4,wmax = 0.9, ψ1 = 1, ψ2 = 1, ψ3 = 2;(v)
ETLBO [4]: elite size = 2;(vi)
Proposed DLTLBO:F=0.5, CR=0.9, neighborhood size = 3.


### 4.3. Comparisons on the Solution Accuracy

The results are shown in Table2 in terms of the average best solution and the standard deviation of the solutions obtained in the 50 independent runs by each algorithm for 10D problems on 24 test functions, where “w/t/l” summarizes the competition results among DLTLBO and other algorithms in the last row of the table, which means that DLTLBO wins in w functions, ties in t functions, and loses in l functions. The best results among the algorithms are shown in bold. Figure 2 presents the convergence graphs of different benchmark functions in terms of the median fitness values achieved by 7 algorithms for 50 runs.Table 2
Mean± Std. of the solutions for 10D problems.
Fun
jDE
SaDE
PSO-wFIPS
FDR-PSO
TLBO
ETLBO
DLTLBO
F
1
4.30
e
-
093
±
4.93
e
-
093
1.76
e
-
088
±
3.90
e
-
088
4.25
e
-
020
±
2.62
e
-
020
1.80
e
-
110
±
2.36
e
-
110
1.16
e
-
220
±
0.00
e
+
000
3.79
e
-
196
±
0.00
e
+
000
1.05
e
-
262
±
0.00
e
+
000
F
2
8.21
e
-
036
±
1.59
e
-
035
9.29
e
-
044
±
2.07
e
-
043
5.65
e
-
008
±
2.66
e
-
008
2.14
e
-
044
±
4.75
e
-
044
4.94
e
-
097
±
7.18
e
-
097
2.87
e
-
094
±
3.82
e
-
094
7.57
e
-
119
±
9.09
e
-
119
F
3
1.91
e
-
095
±
3.85
e
-
095
1.03
e
-
090
±
2.19
e
-
090
1.13
e
-
021
±
1.09
e
-
021
6.01
e
-
112
±
8.88
e
-
112
4.78
e
-
221
±
0.00
e
+
000
8.20
e
-
198
±
0.00
e
+
000
4.98
e
-
265
±
0.00
e
+
000
F
4
8.29
e
-
039
±
1.83
e
-
038
2.94
e
-
059
±
3.98
e
-
059
3.53
e
-
013
±
1.94
e
-
013
1.26
e
-
061
±
1.87
e
-
061
1.73
e
-
104
±
3.80
e
-
104
1.93
e
-
100
±
2.26
e
-
100
8.41
e
-
129
±
2.64
e
-
130
F
5
5.98
e
-
019
±
1.24
e
-
018
1.10
e
+
000
±
1.54
e
+
000
3.84
e
+
000
±
1.57
e
-
001
3.39
e
-
002
±
1.19
e
-
002
5.00
e
-
003
±
1.01
e
-
002
1.53
e
-
003
±
2.30
e
-
003
2.88
e
+
000
±
4.11
e
-
001
F
6
2.84
e
-
015
±
1.59
e
-
015
2.84
e
-
015
±
1.59
e
-
015
4.77
e
-
011
±
1.47
e
-
011
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
2.84
e
-
015
±
1.59
e
-
015
3.55
e
-
015
±
0.00
e
+
000
F
7
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
8.30
e
-
001
±
8.07
e
-
001
1.99
e
+
000
±
9.95
e
-
001
5.88
e
-
001
±
5.49
e
-
001
2.19
e
+
000
±
1.30
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
8
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.07
e
-
010
±
1.90
e
-
010
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
9
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.62
e
-
002
±
1.29
e
-
002
6.93
e
-
002
±
4.92
e
-
002
1.48
e
-
003
±
3.31
e
-
003
4.44
e
-
017
±
9.93
e
-
017
0.00
e
+
000
±
0.00
e
+
000
F
10
1.27
e
-
004
±
0.00
e
+
000
1.27
e
-
004
±
0.00
e
+
000
2.41
e
+
002
±
7.65
e
+
001
5.65
e
+
002
±
1.88
e
+
002
4.82
e
+
002
±
2.04
e
+
002
4.80
e
+
002
±
1.81
e
+
002
5.92
e
+
001
±
8.37
e
+
001
F
11
1.55
e
-
070
±
3.46
e
-
070
4.39
e
-
066
±
9.81
e
-
066
1.30
e
-
019
±
1.51
e
-
019
2.85
e
-
102
±
6.37
e
-
102
2.92
e
-
209
±
0.00
e
+
000
8.70
e
-
190
±
0.00
e
+
000
3.00
e
-
250
±
0.00
e
+
000
F
12
1.75
e
-
041
±
1.99
e
-
041
1.18
e
-
053
±
2.64
e
-
053
5.02
e
-
013
±
3.20
e
-
013
5.22
e
-
061
±
1.15
e
-
060
5.76
e
-
106
±
1.24
e
-
105
2.04
e
-
100
±
4.28
e
-
100
5.21
e
-
128
±
7.36
e
-
128
F
13
8.96
e
-
011
±
2.00
e
-
010
2.31
e
+
000
±
2.19
e
+
000
4.17
e
+
000
±
2.89
e
-
001
4.54
e
+
000
±
3.59
e
+
000
7.53
e
-
002
±
8.83
e
-
002
8.29
e
-
001
±
1.78
e
+
000
2.94
e
+
000
±
1.62
e
+
000
F
14
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
5.76
e
-
011
±
2.92
e
-
011
3.55
e
-
015
±
0.00
e
+
000
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
1.78
e
-
015
±
2.51
e
-
015
F
15
3.87
e
+
000
±
1.28
e
+
000
3.66
e
+
000
±
1.55
e
+
000
3.97
e
+
000
±
3.24
e
+
000
7.56
e
+
000
±
3.76
e
+
000
2.42
e
+
000
±
1.81
e
+
000
2.98
e
+
000
±
1.41
e
+
000
1.40
e
-
004
±
1.97
e
-
004
F
16
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
1.48
e
-
007
±
2.50
e
-
007
7.13
e
-
002
±
1.26
e
-
001
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
17
1.48
e
-
003
±
3.31
e
-
003
3.94
e
-
003
±
5.67
e
-
003
1.04
e
-
001
±
8.74
e
-
002
1.13
e
-
001
±
5.80
e
-
002
5.41
e
-
003
±
8.59
e
-
003
1.48
e
-
003
±
3.31
e
-
003
0.00
e
+
000
±
0.00
e
+
000
F
18
1.38
e
+
002
±
1.22
e
+
002
1.07
e
+
002
±
1.21
e
+
002
2.80
e
+
002
±
1.28
e
+
002
5.29
e
+
002
±
2.00
e
+
002
5.46
e
+
002
±
3.23
e
+
002
5.44
e
+
002
±
2.81
e
+
002
1.18
e
+
002
±
0.00
e
+
000
F
19
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.21
e
-
020
±
2.43
e
-
020
3.03
e
+
000
±
4.14
e
+
000
0.00
e
+
000
±
0.00
e
+
000
1.01
e
-
029
±
2.26
e
-
029
2.52
e
-
029
±
3.57
e
-
029
F
20
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
5.87
e
-
008
±
2.46
e
-
008
0.00
e
+
000
±
0.00
e
+
000
3.06
e
-
011
±
5.16
e
-
011
2.47
e
-
010
±
5.33
e
-
010
8.77
e
-
001
±
1.23
e
+
000
F
21
1.83
e
-
017
±
3.99
e
-
017
1.50
e
-
001
±
2.13
e
-
001
4.50
e
+
000
±
1.57
e
-
001
1.53
e
+
000
±
2.06
e
+
000
1.05
e
-
001
±
1.49
e
-
001
8.73
e
+
000
±
1.78
e
+
001
4.55
e
+
000
±
8.73
e
-
001
F
22
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.26
e
+
000
±
9.23
e
-
001
3.58
e
+
000
±
2.06
e
+
000
7.76
e
+
000
±
3.02
e
+
000
9.55
e
+
000
±
4.37
e
+
000
3.48
e
+
000
±
7.04
e
-
001
F
23
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.32
e
-
002
±
1.71
e
-
002
6.88
e
-
001
±
5.35
e
-
001
5.86
e
-
002
±
2.77
e
-
002
6.75
e
-
002
±
3.32
e
-
002
5.04
e
-
002
±
3.65
e
-
002
F
24
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
5.73
e
-
011
±
2.17
e
-
011
3.54
e
+
000
±
1.98
e
+
000
6.34
e
-
001
±
9.19
e
-
001
1.22
e
+
000
±
7.14
e
-
001
5.78
e
-
001
±
8.17
e
-
001
w
/
t
/
l
10/4/10
9/4/11
19/0/5
19/2/3
16/2/6
17/2/5Figure 2
Comparison of the performance curves of 7 algorithms.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)The comparisons in Table2 show that DLTLBO outperforms jDE on 10 functions, while jDE achieves better results on 10 functions. SaDE obtains better performance on 11 functions, while DLTLBO only outperforms jDE for 9 functions. PSO-wFIPS achieves better results than DLTLBO on 5 functions, while DLTLBO performs better for the remaining 19 functions. FDR-PSO only obtains better performance than DLTLBO on F5, F20, and F21, while DLTLBO outperforms FDR-PSO for the remaining 19 functions except F6 and F8. DLTLBO outperforms TLBO on 16 functions and outperforms ETLBO on 17 functions, while TLBO, ETLBO, and DLTLBO perform well on functions F8 and F16. It can be concluded that the proposed DLTLBO significantly improves the algorithm’s performance, although DLTLBO is not always better than other algorithms.


## 4.1. Benchmark Functions

The details of 24 benchmark functions are shown in Table1. Among 24 benchmark functions, F1 to F4 are unimodal functions, F5 to F10 are multimodal functions, and F11 to F18 are the rotated version of F3 to F10, respectively. F19 to F24 are taken from the CEC 2008 test suite. The searching range and theory optima for all functions are also shown in Table 1.Table 1
Details of numerical benchmarks used.
Function
Formula
Range
Optima
Sphere
F
1
x
=
∑
i
=
1
D
x
i
2
[
-
100,100
]
0
Quadric
F
2
x
=
∑
i
=
1
D
∑
j
=
1
i
x
j
2
[
-
100
,
100
]
0
Sum square
F
3
x
=
∑
i
=
1
D
i
x
i
2
[
-
10
,
10
]
0
Zakharov
F
4
x
=
∑
i
=
1
D
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
4
[
-
10
,
10
]
0
Rosenbrock
F
5
x
=
∑
i
=
1
D
-
1
100
x
i
2
-
x
i
+
1
2
+
x
i
-
1
2
[
-
2.048,2.048
]
0
Ackley
F
6
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
x
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
x
i
+
e
[
-
32.768
,
32.768
]
0
Rastrigin
F
7
x
=
∑
i
=
1
D
x
i
2
-
10
cos
⁡
2
π
x
i
+
10
[
-
5.12
,
5.12
]
0
Weierstrass
F
8
x
=
∑
i
=
1
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
x
i
+
0.5
-
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
×
0.5
a
=
0.5
b
=
3
k
m
a
x
=
20
[
-
0.5
,
0.5
]
0
Griewank
F
9
x
=
∑
i
=
1
D
x
i
2
4000
-
∏
i
=
1
n
cos
⁡
x
i
i
+
1
[
-
600
,
600
]
0
Schwefel
F
10
x
=
418.9829
D
+
∑
i
=
1
D
-
x
i
sin
⁡
a
b
s
x
i
[
-
500
,
500
]
0
Rotated sum square
F
11
x
=
∑
i
=
1
D
i
y
i
2
y
=
M
×
x
[
-
10
,
10
]
0
Rotated Zakharov
F
12
x
=
∑
i
=
1
D
x
i
2
+
∑
i
=
1
D
0.5
i
y
i
2
+
∑
i
=
1
D
0.5
i
y
i
4 y=M×x
[
-
10
,
10
]
0
Rotated Rosenbrock
F
13
x
=
∑
i
=
1
D
-
1
100
y
i
2
-
y
i
+
1
2
+
y
i
-
1
2
y
=
M
×
x
[
-
2.048
,
2.048
]
0
Rotated Ackley
F
14
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
y
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
y
i
+
e y=M×x
[
-
32.768
,
32.768
]
0
Rotated Rastrigin
F
15
x
=
∑
i
=
1
D
y
i
2
-
10
cos
⁡
2
π
y
i
⁡
+
10
y
=
M
×
x
[
-
5.12
,
5.12
]
0
Rotated Weierstrass
F
16
x
=
∑
i
=
1
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
y
i
+
0.5
-
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
×
0.5
a
=
0.5
b
=
3
k
m
a
x
=
20
y
=
M
×
x
[
-
0.5
,
0.5
]
0
Rotated Griewank
F
17
x
=
∑
i
=
1
D
y
i
2
4000
-
∏
i
=
1
n
cos
⁡
y
i
i
+
1
y
=
M
×
x
[
-
600
,
600
]
0
Rotated Schwefel
F
18
x
=
418.9829
D
+
∑
i
=
1
D
-
y
i
sin
⁡
a
b
s
y
i
y
=
M
×
x
[
-
500
,
500
]
0
Shifted sphere
F
19
x
=
∑
i
=
1
D
z
i
2
-
450 z=x-o o: theshiftedglobaloptimum
[
-
100
,
100
]
o
Shifted Schwefel’s Problem 2.21
F
20
x
=
max
⁡
i
z
i
,
1
≤
i
≤
D
-
450 z=x-o+1 o: the  shifted  global  optimum
[
-
100
,
100
]
o
Shifted Rosenbrock
F
21
x
=
∑
i
=
1
D
-
1
100
z
i
2
-
z
i
+
1
2
+
z
i
-
1
2
+
390 z=x-o+1 o: the  shifted  global  optimum
[
-
100
,
100
]
o
Shifted Rastrigin
F
22
x
=
∑
i
=
1
D
z
i
2
-
10
cos
⁡
2
π
z
i
+
10
-
330 z=x-o o: the  shifted  global  optimum
-
5,5
o
Shifted Griewank
F
23
x
=
∑
i
=
1
D
z
i
2
4000
-
∏
i
=
1
n
cos
⁡
z
i
i
+
1
-
180 z=x-o o: the  shifted  global  optimum
[
-
600
,
600
]
o
Shifted Ackley
F
24
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
z
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
z
i
+
e
-
140 z=x-o o: the  shifted  global  optimum
[
-
32
,
32
]
o


## 4.2. Parameter Settings

In this paper, all the experiments are carried out on the same machine with a Celoron 2.26 GHz CPU, 2 GB memory, and Windows XP operating system with MATLAB 7.9. For the purpose of reducing statistical errors, each function is independently simulated 50 runs, and their mean results are used in the comparison. For all approaches, the population size was set to 50, and the maximal number of cost function evaluations (FEs) is used as termination condition of algorithm, namely, 100,000 for 10D problems. The parameter settings for all algorithms in comparison are extracted from their corresponding literatures and are described as follows:(i)
jDE [31]: F=0.5, CR=0.9;(ii)
SaDE [32]: F~N(0.5,0.3), CR0=0.5, CR~N(CRm,0.1), LP = 50;(iii)
PSO-wFIPS [33]:w = 0.7298;(iv)
PSO-FDR [34]:wmin = 0.4,wmax = 0.9, ψ1 = 1, ψ2 = 1, ψ3 = 2;(v)
ETLBO [4]: elite size = 2;(vi)
Proposed DLTLBO:F=0.5, CR=0.9, neighborhood size = 3.


## 4.3. Comparisons on the Solution Accuracy

The results are shown in Table2 in terms of the average best solution and the standard deviation of the solutions obtained in the 50 independent runs by each algorithm for 10D problems on 24 test functions, where “w/t/l” summarizes the competition results among DLTLBO and other algorithms in the last row of the table, which means that DLTLBO wins in w functions, ties in t functions, and loses in l functions. The best results among the algorithms are shown in bold. Figure 2 presents the convergence graphs of different benchmark functions in terms of the median fitness values achieved by 7 algorithms for 50 runs.Table 2
Mean± Std. of the solutions for 10D problems.
Fun
jDE
SaDE
PSO-wFIPS
FDR-PSO
TLBO
ETLBO
DLTLBO
F
1
4.30
e
-
093
±
4.93
e
-
093
1.76
e
-
088
±
3.90
e
-
088
4.25
e
-
020
±
2.62
e
-
020
1.80
e
-
110
±
2.36
e
-
110
1.16
e
-
220
±
0.00
e
+
000
3.79
e
-
196
±
0.00
e
+
000
1.05
e
-
262
±
0.00
e
+
000
F
2
8.21
e
-
036
±
1.59
e
-
035
9.29
e
-
044
±
2.07
e
-
043
5.65
e
-
008
±
2.66
e
-
008
2.14
e
-
044
±
4.75
e
-
044
4.94
e
-
097
±
7.18
e
-
097
2.87
e
-
094
±
3.82
e
-
094
7.57
e
-
119
±
9.09
e
-
119
F
3
1.91
e
-
095
±
3.85
e
-
095
1.03
e
-
090
±
2.19
e
-
090
1.13
e
-
021
±
1.09
e
-
021
6.01
e
-
112
±
8.88
e
-
112
4.78
e
-
221
±
0.00
e
+
000
8.20
e
-
198
±
0.00
e
+
000
4.98
e
-
265
±
0.00
e
+
000
F
4
8.29
e
-
039
±
1.83
e
-
038
2.94
e
-
059
±
3.98
e
-
059
3.53
e
-
013
±
1.94
e
-
013
1.26
e
-
061
±
1.87
e
-
061
1.73
e
-
104
±
3.80
e
-
104
1.93
e
-
100
±
2.26
e
-
100
8.41
e
-
129
±
2.64
e
-
130
F
5
5.98
e
-
019
±
1.24
e
-
018
1.10
e
+
000
±
1.54
e
+
000
3.84
e
+
000
±
1.57
e
-
001
3.39
e
-
002
±
1.19
e
-
002
5.00
e
-
003
±
1.01
e
-
002
1.53
e
-
003
±
2.30
e
-
003
2.88
e
+
000
±
4.11
e
-
001
F
6
2.84
e
-
015
±
1.59
e
-
015
2.84
e
-
015
±
1.59
e
-
015
4.77
e
-
011
±
1.47
e
-
011
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
2.84
e
-
015
±
1.59
e
-
015
3.55
e
-
015
±
0.00
e
+
000
F
7
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
8.30
e
-
001
±
8.07
e
-
001
1.99
e
+
000
±
9.95
e
-
001
5.88
e
-
001
±
5.49
e
-
001
2.19
e
+
000
±
1.30
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
8
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.07
e
-
010
±
1.90
e
-
010
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
9
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.62
e
-
002
±
1.29
e
-
002
6.93
e
-
002
±
4.92
e
-
002
1.48
e
-
003
±
3.31
e
-
003
4.44
e
-
017
±
9.93
e
-
017
0.00
e
+
000
±
0.00
e
+
000
F
10
1.27
e
-
004
±
0.00
e
+
000
1.27
e
-
004
±
0.00
e
+
000
2.41
e
+
002
±
7.65
e
+
001
5.65
e
+
002
±
1.88
e
+
002
4.82
e
+
002
±
2.04
e
+
002
4.80
e
+
002
±
1.81
e
+
002
5.92
e
+
001
±
8.37
e
+
001
F
11
1.55
e
-
070
±
3.46
e
-
070
4.39
e
-
066
±
9.81
e
-
066
1.30
e
-
019
±
1.51
e
-
019
2.85
e
-
102
±
6.37
e
-
102
2.92
e
-
209
±
0.00
e
+
000
8.70
e
-
190
±
0.00
e
+
000
3.00
e
-
250
±
0.00
e
+
000
F
12
1.75
e
-
041
±
1.99
e
-
041
1.18
e
-
053
±
2.64
e
-
053
5.02
e
-
013
±
3.20
e
-
013
5.22
e
-
061
±
1.15
e
-
060
5.76
e
-
106
±
1.24
e
-
105
2.04
e
-
100
±
4.28
e
-
100
5.21
e
-
128
±
7.36
e
-
128
F
13
8.96
e
-
011
±
2.00
e
-
010
2.31
e
+
000
±
2.19
e
+
000
4.17
e
+
000
±
2.89
e
-
001
4.54
e
+
000
±
3.59
e
+
000
7.53
e
-
002
±
8.83
e
-
002
8.29
e
-
001
±
1.78
e
+
000
2.94
e
+
000
±
1.62
e
+
000
F
14
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
5.76
e
-
011
±
2.92
e
-
011
3.55
e
-
015
±
0.00
e
+
000
3.55
e
-
015
±
0.00
e
+
000
2.84
e
-
015
±
1.59
e
-
015
1.78
e
-
015
±
2.51
e
-
015
F
15
3.87
e
+
000
±
1.28
e
+
000
3.66
e
+
000
±
1.55
e
+
000
3.97
e
+
000
±
3.24
e
+
000
7.56
e
+
000
±
3.76
e
+
000
2.42
e
+
000
±
1.81
e
+
000
2.98
e
+
000
±
1.41
e
+
000
1.40
e
-
004
±
1.97
e
-
004
F
16
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
1.48
e
-
007
±
2.50
e
-
007
7.13
e
-
002
±
1.26
e
-
001
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
F
17
1.48
e
-
003
±
3.31
e
-
003
3.94
e
-
003
±
5.67
e
-
003
1.04
e
-
001
±
8.74
e
-
002
1.13
e
-
001
±
5.80
e
-
002
5.41
e
-
003
±
8.59
e
-
003
1.48
e
-
003
±
3.31
e
-
003
0.00
e
+
000
±
0.00
e
+
000
F
18
1.38
e
+
002
±
1.22
e
+
002
1.07
e
+
002
±
1.21
e
+
002
2.80
e
+
002
±
1.28
e
+
002
5.29
e
+
002
±
2.00
e
+
002
5.46
e
+
002
±
3.23
e
+
002
5.44
e
+
002
±
2.81
e
+
002
1.18
e
+
002
±
0.00
e
+
000
F
19
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.21
e
-
020
±
2.43
e
-
020
3.03
e
+
000
±
4.14
e
+
000
0.00
e
+
000
±
0.00
e
+
000
1.01
e
-
029
±
2.26
e
-
029
2.52
e
-
029
±
3.57
e
-
029
F
20
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
5.87
e
-
008
±
2.46
e
-
008
0.00
e
+
000
±
0.00
e
+
000
3.06
e
-
011
±
5.16
e
-
011
2.47
e
-
010
±
5.33
e
-
010
8.77
e
-
001
±
1.23
e
+
000
F
21
1.83
e
-
017
±
3.99
e
-
017
1.50
e
-
001
±
2.13
e
-
001
4.50
e
+
000
±
1.57
e
-
001
1.53
e
+
000
±
2.06
e
+
000
1.05
e
-
001
±
1.49
e
-
001
8.73
e
+
000
±
1.78
e
+
001
4.55
e
+
000
±
8.73
e
-
001
F
22
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.26
e
+
000
±
9.23
e
-
001
3.58
e
+
000
±
2.06
e
+
000
7.76
e
+
000
±
3.02
e
+
000
9.55
e
+
000
±
4.37
e
+
000
3.48
e
+
000
±
7.04
e
-
001
F
23
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
2.32
e
-
002
±
1.71
e
-
002
6.88
e
-
001
±
5.35
e
-
001
5.86
e
-
002
±
2.77
e
-
002
6.75
e
-
002
±
3.32
e
-
002
5.04
e
-
002
±
3.65
e
-
002
F
24
0.00
e
+
000
±
0.00
e
+
000
0.00
e
+
000
±
0.00
e
+
000
5.73
e
-
011
±
2.17
e
-
011
3.54
e
+
000
±
1.98
e
+
000
6.34
e
-
001
±
9.19
e
-
001
1.22
e
+
000
±
7.14
e
-
001
5.78
e
-
001
±
8.17
e
-
001
w
/
t
/
l
10/4/10
9/4/11
19/0/5
19/2/3
16/2/6
17/2/5Figure 2
Comparison of the performance curves of 7 algorithms.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)The comparisons in Table2 show that DLTLBO outperforms jDE on 10 functions, while jDE achieves better results on 10 functions. SaDE obtains better performance on 11 functions, while DLTLBO only outperforms jDE for 9 functions. PSO-wFIPS achieves better results than DLTLBO on 5 functions, while DLTLBO performs better for the remaining 19 functions. FDR-PSO only obtains better performance than DLTLBO on F5, F20, and F21, while DLTLBO outperforms FDR-PSO for the remaining 19 functions except F6 and F8. DLTLBO outperforms TLBO on 16 functions and outperforms ETLBO on 17 functions, while TLBO, ETLBO, and DLTLBO perform well on functions F8 and F16. It can be concluded that the proposed DLTLBO significantly improves the algorithm’s performance, although DLTLBO is not always better than other algorithms.


## 5. IIR System Identification Using DLTLBO

The goal of filter modeling is to alter filter coefficients of a digital filter to match an unknown system transfer function. Evolutionary algorithms are heuristic search methods, which are characterized by features as stochastic, adaptive, and learning in order to produce intelligent optimization schemes. In this section, DLTLBO is used to optimize the coefficients vector of the digital IIR filter.


### 5.1. Digital IIR Filter Identification

Application of IIR filter in system identification has been widely studied since many problems encountered in signal processing can be characterized as a system identification problem. Therefore, in the simulation study, digital IIR filters are designed for the system identification purpose. In this case, the parameters of the filters are successively adjusted by evolutionary optimization algorithms until the error between the output of the filter and the unknown system is minimized. In general, we expect that the number of zero points is more than the number of pole points. A schematic of filter modeling problem using the evolutionary algorithms is shown in Figure3.Figure 3
Filter design using a heuristic search algorithm.The general expression of digital IIR filters is given by the following difference equation [35]:(7)yk=∑i=0Maixk-i-∑i=1Nbiyk-i,where x(k) is the input of the digital IIR filter at time k, y(k) is the output of the digital IIR filter at time k, M and N are denominator and numerator orders, N(N≥M) is the filter’s order, ai and bi are adjustable filter coefficients of the model, and b0=1. The number of the digital IIR filter coefficients is (N+M+1).For adaptive digital IIR filter design, the objective of the adaptation is to adjust filter coefficients of a digital filter to estimate actual parameter values of an unknown system from its inputs and outputs. The design of this filter can be considered as a minimization problem of the cost function. The most commonly used approach to digital IIR filter design is to formulate the problem as an optimization problem with the mean square error (MSE) as the cost function to be minimized:(8)Jv=Ee2k=1S∑k=1SidealHk-actualHk2,where v=a0a1⋯aMb1b2⋯bN denotes the digital IIR filter coefficient vector, S is the number of samples used for the computation of the time-averaged cost function, and ideal(k) and actual(k) are the filter’s ideal and actual responses, respectively.To achieve higher stop band attenuation and to have better control on the transition width, the frequency cost function [27] to be minimized is defined as(9)Jv=1Sp∑ωpabsabsactualHω-1-δp+1Ss∑ωsabsactualHω-δs,where idealH(w) and actualH(w) are the filter’s ideal and actual frequency responses, respectively. δp and δs are the ripples in the pass band and stop band, ωp and ωs are pass band and stop band normalized cutoff frequencies, respectively, and Sp and Ss are the number of samples used in the pass band and stop band. In this case, the best fitness J(v)=0 represents that the magnitude response of the solution is within [1-δp,1] in the pass band and [0,δs] in the stop band.


### 5.2. Description of the DLTLBO for the Digital IIR Filter

The steps of the DLTLBO for the digital IIR filter are given as follows.Step 1.
Initialize every real coded learnerv=a0a1⋯aMb1b2⋯bN(N≥M) of the population. Each learner consists of M numerator filter coefficients and N denominator filter coefficients; the total number of filter coefficients is N+M+1 for Nth order filter to be designed. To maintain the stability of IIR filter, the efficient way of [17] is adopted. Hence, the coefficients ai are formed by the constraints [-2,2] and all the lattice-reflection-form coefficients ki of the direct-form coefficients bi are less than 1.Step 2.
Compute the error fitness value for each learner according to the cost function.Step 3 (teacher phase).
Each learner learns from the teacher according to (4)–(6). Compute the error fitness value for each learner and update each learner.Step 4 (learner phase).
Each learner randomly learns from the other learner according to (3). Compute the error fitness value for each learner and update each learner.Step 5 (termination criterion).
Stop if the termination criterion is met; the dimensional values of the global teacher are the coefficients of the designed digital IIR filter; otherwise, repeat from Step3.


### 5.3. Experimental Simulation

This section presents experimental evaluation of DLTLBO on designing digital IIR filters. Some benchmark problems are selected to compare the performance of GA [36], PSOwFIPS [33], SaDE [32], TLBO [1, 2], and DLTLBO algorithms. In all cases, the maximal value of FEs is used as ended condition of algorithm, namely, 25000 for GA, PSOwFIPS, SaDE, TLBO, and DLTLBO, respectively; the number of population is set to 50, and the initial population is selected randomly. For GA, the crossover rate Pc is set to 0.9 and the mutation rate Pm is set to 0.1. For PSOwFIPS,w = 0.7298. For SaDE, F~N(0.5,0.3), CR0=0.5, CR ~N(CRm, 0.1), and LP = 50. For our DLTLBO algorithm, the scaling factor F is set to 0.9 and the crossover probability CR is set to 0.1.Example 1.
In the first example, the transfer function of the unknown plant is unknown and only the performance parameters are given in Table3, including low-pass (LP), high-pass (HP), band-pass (BP), and band-stop (BS) filters.Table 3
Design parameters of IIR LP, HP, BP, and BS filters.
Type of filter
Pass band rippleδp
Stop band rippleδs
Pass band edgeωp
Stop band edgeωs
Low-pass (LP)
0.01
0.001
0.45
0.5
High-pass (HP)
0.01
0.001
0.5
0.45
Band-pass (BP)
0.01
0.001
0.3 and 0.7
0.25 and 0.75
Band-stop (BS)
0.01
0.001
0.25 and 0.75
0.3 and 0.7In this experiment, the filter orders are set to 8 for all IIR LP, HP, BP, and BS filters and the number of samples is set to 128. The minimum and maximum values of the coefficients are −2 and 2. The cost function is given as shown in (9). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 4 shows the maximum values and the mean values of stop band attenuation, the means and standard deviations of MSE, and the means and standard deviations of CPU times. Figure 4 shows the comparison of convergence curves, magnitude responses, and gain responses of five algorithms. In addition, a statisticalt-test [21] is given in Table 4.Table 4
Statistical data for stop band attenuation, MSE, and CPU times for 8th order IIR filters.
Type of filter
Algorithm
Maximum gain (dB)
Mean gain (dB)
MSE
t-stat
p value
t-value
Time (s)
Low-pass (LP)
GA
28.13
40.37
2.01
e
-
001
±
3.40
e
-
002
13.60792
3.01
E
-
11
2.093024
4.24
e
+
000
±
1.01
e
-
002
PSOwFIPS
30.22
43.51
1.10
e
-
001
±
1.94
e
-
002
5.14552
5.75
E
-
05
2.093024
5.17
e
+
000
±
4.69
e
-
003
SaDE
31.43
48.23
8.25
e
-
002
±
3.25
e
-
002
1.31278
2
.
0
5
E
-
01
2.093024
5.77
e
+
000
±
5.46
e
-
002
TLBO
28.74
41.18
9.40
e
-
002
±
3.62
e
-
002
3.03097
6.87
E
-
03
2.093024
4.80
e
+
000
±
2.12
e
-
002
DLTLBO
26.21
38.37
6.68
e
-
002
±
3.13
e
-
002
—
—
—
4.74
e
+
000
±
1.17
e
-
002
High-pass (HP)
GA
16.85
28.28
1.92
e
-
001
±
4.54
e
-
002
12.53964
1.23
E
-
10
2.09302
4.26
e
+
000
±
7.07
e
-
003
PSOwFIPS
20.16
32.46
1.24
e
-
001
±
2.59
e
-
002
8.96138
2.9
9
E
-
08
2.09302
5.20
e
+
000
±
6.41
e
-
003
SaDE
28.49
38.72
9.39
e
-
002
±
4.11
e
-
002
2.64240
1
.
6
1
E
-
02
2.09302
5.70
e
+
000
±
2.64
e
-
002
TLBO
25.39
35.37
1.12
e
-
001
±
2.89
e
-
002
5.73652
1.5
8
E
-
05
2.09302
4.78
e
+
000
±
3.74
e
-
002
DLTLBO
35.28
43.51
6.12
e
-
002
±
2.37
e
-
002
—
—
—
4.73
e
+
000
±
4.72
e
-
003
Band-pass (BP)
GA
24.60
37.28
3.24
e
-
001
±
5.96
e
-
002
7.12807
8
.
90
E
-
07
2.09302
4.81
e
+
000
±
1.21
e
-
002
PSOwFIPS
24.23
36.84
2.38
e
-
001
±
2.91
e
-
002
3.50304
2
.
3
8
E
-
03
2.09302
5.72
e
+
000
±
3.65
e
-
002
SaDE
29.73
42.19
2.00
e
-
001
±
2.77
e
-
002
-
0.1944
7
0.84787
2.09302
6.32
e
+
000
±
4.39
e
-
002
TLBO
25.91
39.52
2.34
e
-
001
±
4.00
e
-
002
2.80619
0.01127
2.09302
5.37
e
+
000
±
3.20
e
-
002
DLTLBO
27.90
40.27
2.01
e
-
001
±
3.25
e
-
002
—
—
—
5.28
e
+
000
±
5.08
e
-
003
Band-stop (BS)
GA
16.24
28.31
3.62
e
-
001
±
7.40
e
-
002
10.51046
2.3
5
E
-
09
2.09302
4.81
e
+
000
±
5.23
e
-
003
PSOwFIPS
17.75
30.28
2.60
e
-
001
±
3.90
e
-
002
6.70940
2.0
6
E
-
06
2.09302
5.77
e
+
000
±
8.13
e
-
002
SaDE
21.17
35.22
2.04
e
-
001
±
5.80
e
-
002
2.44724
0.0242
9
2.09302
6.34
e
+
000
±
2.20
e
-
002
TLBO
14.01
26.25
2.30
e
-
001
±
7.20
e
-
002
3.56938
2
.
0
5
E
-
03
2.09302
5.34
e
+
000
±
3.13
e
-
002
DLTLBO
36.07
46.27
1.60
e
-
001
±
4.62
e
-
002
—
—
—
5.33
e
+
000
±
1.01
e
-
002Figure 4
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for LP
(b)
Magnitude response for LP
(c)
Gain response for LP
(d)
Convergence curves for HP
(e)
Magnitude response for HP
(f)
Gain response for HP
(g)
Convergence curves for BP
(h)
Magnitude response for BP
(i)
Gain response for BP
(j)
Convergence curves for BS
(k)
Magnitude response for BS
(l)
Gain response for BSAs seen from Table4, it is observed that the proposed 8th order IIR filter design using DLTLBO spends the less CPU running time among all the three algorithms except GA. For low-pass (LP) filter, it is observed that the maximum stop band attenuations 28.13 dB, 30.22 dB, 31.43 dB, 28.74 dB, and 26.21 dB are obtained for GA, PSOwFIPS, SaDE, TLBO, and DLTLBO, respectively. Statisticalt-test between other algorithms and DLTLBO also indicates that DLTLBO performs well. Comparative curves in Figures 4(a)~4(c) also explore that the proposed 8th order IIR filter design using SaDE attains the highest stop band attenuation, although DLTLBO obtains the least MSE. Figure 4(a) shows the convergence curves for the designed 8th order IIR LP filter obtained for different algorithms, Figure 4(b) shows the comparative magnitude plots in dB, and Figure 4(c) represents the comparative normalized gain plots for 8th order IIR LP filter. For high-pass (HP) filter and band-stop (BS) filter, it is seen that the proposed 8th order IIR filter design using DLTLBO attains the highest stop band attenuation and the least MSE. It is also observed from Table 4 and Figure 4 that, for band-pass (BP) filter, the proposed 8th order IIR filter design using SaDE attains the highest stop band attenuation and the least MSE, and DLTLBO obtains the second highest stop band attenuation and the second least MSE. It can be concluded that DLTLBO has a good performance of the solution accuracy, although DLTLBO needs more CPU running time.Example 2.
In the first example (taken from [21, 22, 26]), a second-order IIR filter is estimated by a second-order IIR filter. The unknown plant and the filter had the following transfer functions:(10)idealHz=11-1.2z-1+0.6z-2,actualHz=11+b1z-1+b2z-2.The inputx(k) to the system and the filter is a white sequence. The sequence length used is N=100.Since the filter order is equal to that of the system, the error surface is unimodal. The best solution should be located at{-1.2,0.6}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 5 shows the best filter coefficients, the means and standard deviations of MSE, the means and standard deviations of CPU times, and statisticalt-test between DLTLBO and other algorithms. Figure 5 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 5
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
b
1
b
2
MSE
t-stat
p value
t-value
Time (s)
GA
-
1.20001
0.60000
1.38
e
-
04
±
3.37
e
-
04
1.83706
0.08190
2.09302
1.41
e
+
01
±
2.52
e
-
02
PSOwFIPS
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.50
e
+
01
±
3.44
e
-
02
SaDE
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.60
e
+
01
±
7.57
e
-
02
TLBO
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.43
e
+
01
±
4.70
e
-
02
DLTLBO
-
1.20000
0.60000
0.0
±
0.0
—
—
—
1.48
e
+
01
±
1.28
e
-
02Figure 5
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example2
(b)
Magnitude response for Example2From the values of MSE in Table5, one can see that PSOwFIPS, SaDE, TLBO, and DLTLBO can find the global minimum. From the values oft-test, it is also seen that DLTLBO is statistically better than GA. As seen from Table 5 and Figure 5, although DLTLBO spends more time than TLBO when the number of FEs reaches the given maximum FEs, DLTLBO has the faster convergence speed than other algorithms. It can be concluded that DLTLBO has a good performance of the solution accuracy.Example 3.
In the second example (taken from [17, 18, 21, 22, 26]), a second-order IIR filter is estimated by a first-order IIR filter. The unknown plant and the filter had the following transfer functions:(11)idealHz=0.05-0.4z-11-1.1314z-1+0.25z-2,actualHz=a01+b1z-1.The inputx(k) to the system and the filter is a Gaussian white noise with the mean of zero and unit variance. The sequence length used is N=100.Since the reduced order filter is employed for the identification, the error surface of the cost function is multimodal. The error surface has a global minimum at{-0.311,-0.906} and a local minimum at {0.114,0.519}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 6 shows the comparative results between DLTLBO and other algorithms. Figure 6 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 6
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
b
1
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.25418
-
0.92619
1.83
e
-
01
±
6.74
e
-
05
2582.69885
3.76
E
-
54
2.09302
1.32
e
+
01
±
6.23
e
-
02
PSOwFIPS
-
0.26621
-
0.92312
2.11
e
-
01
±
3.69
e
-
02
8.08353
1.4
4
E
-
07
2.09302
1.41
e
+
01
±
7.63
e
-
02
SaDE
-
0.31900
-
0.92058
2.01
e
-
01
±
7.43
e
-
17
16.75618
1.4
4
E
-
12
2.09302
1.46
e
+
01
±
7.22
e
-
02
TLBO
-
0.26675
-
0.91228
2.07
e
-
01
±
4.37
e
-
17
18.25943
1.4
4
E
-
15
2.09302
1.33
e
+
01
±
4.84
e
-
02
DLTLBO
-
0.36707
-
0.89206
1.44
e
-
01
±
5.37
e
-
17
—
—
—
1.40
e
+
01
±
9.93
e
-
02Figure 6
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example3
(b)
Magnitude response for Example3Table6 shows the best filter coefficients, the means and standard deviations of MSE, and CPU times. It is observed that DLTLBO has the smaller mean value of MSE, while the standard deviations of MSE obtained by SaDE, TLBO, and DLTLBO are approximate. It can be observed from Table 6 that when the number of FEs reaches the given maximum FEs, GA spends less time among all other three algorithms and DLTLBO spends less time than PSOwFIPS and SaDE. It is also seen that the differences of the means of MSE are statistically significant from the values oft-test between DLTLBO and other corresponding algorithms. As seen from Table 6 and Figure 6, the convergence speed of DLTLBO is faster than those of other algorithms.Example 4.
In the third example (taken from [24]), a third-order IIR filter is estimated by a third-order IIR filter. The unknown plant and the filter had the following transfer functions:(12)idealHz=-0.2+0.4z-1-0.5z-21-0.6z-1+0.25z-2-0.2z-3,actualHz=a0+a1z-1+a2z-21+b1z-1+b2z-2+b3z-3.The inputx(k) to the system and the filter is a Gaussian white noise with the mean of zero and unit variance. The sequence length used is N=100.Since the filter order is equal to that of the system, the error surface is unimodal. The best solution should be located at{-0.2,0.4,-0.5,-0.6,0.25,-0.2}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 7 shows the comparative results between DLTLBO and other algorithms. Figure 7 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 7
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
a
1
a
2
b
1
b
2
b
3
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.19546
0.39125
-
0.49149
-
0.57708
0.22420
-
0.18713
1.47
e
-
03
±
7.77
e
-
04
8.46158
7.21
E
-
08
2.09302
1.44
e
+
01
±
9.72
e
-
02
PSOwFIPS
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.54
e
-
17
±
1.40
e
-
17
4.9250
3
9.40
E
-
05
2.09302
1.53
e
+
01
±
1.32
e
-
01
SaDE
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
00
±
00
-
1.70613
0.104277
2.09302
1.62
e
+
01
±
7.40
e
-
02
TLBO
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.20
e
-
32
±
2.74
e
-
32
1.6650
4
0.11231
2.09302
1.48
e
+
01
±
4.75
e
-
02
DLTLBO
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.89
e
-
33
±
4.96
e
-
33
—
—
—
1.52
e
+
01
±
1.82
e
-
02Figure 7
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example4
(b)
Magnitude response for Example4As seen from Table7, the filter coefficients obtained by PSO and DLTLBO are exactly located at the best solution. Table 7 lists that PSOwFIPS, SaDE, TLBO, and DLTLBO have the same best filter coefficients. SaDE offers the best MSE performance and DLTLBO outperforms GA, PSOwFIPS, and TLBO. Thet-test in terms of the means of MSE between DLTLBO and other algorithms indicates that DLTLBO is statistically better than the corresponding algorithms except for SaDE. From Table 7 and Figure 7, we can see that the convergence speed of DLTLBO is faster than those of other algorithms except SaDE. It can be concluded that DLTLBO has a good performance of the solution accuracy, although DLTLBO needs more CPU running time when the number of FEs reaches the given maximum FEs.Example 5.
In the fourth example (taken from [17, 18]), a third-order filter is estimated by a second-order IIR filter. The unknown plant and the filter had the following transfer functions:(13)idealHz=-0.3+0.4z-1-0.5z-21-1.2z-1+0.5z-2-0.1z-3,actualHz=a0+a1z-11+b1z-1+b2z-2.The inputx(k) to the system and the filter is a uniformly distributed white sequence; taking values from (-0.5,0.5), the SNR = 30 dB. The sequence length used is N=100.Since the reduced order filter is employed for the identification, the error surface of the cost function is multimodal. The error surface has a global minimum at{-0.39,-0.077,-0.22,-0.58}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. The comparative results between DLTLBO and other algorithms are given in Table 8. Figure 8 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 8
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
a
1
b
1
b
2
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.41403
-
0.06370
-
0.26851
-
0.52414
1.32
e
-
02
±
1.14
e
-
04
46.6421
4.61
E
-
21
2.09302
1.42
e
+
01
±
1.82
e
-
02
PSOwFIPS
-
0.15324
-
0.00954
-
1.57619
0.66054
1.53
e
-
02
±
1.57
e
-
03
9.45093
1.3
E
-
08
2.09302
1.50
e
+
01
±
2.84
e
-
02
SaDE
-
0.40588
-
0.03978
-
0.25151
-
0.54818
1.35
e
-
02
±
3.56
e
-
18
41.65782
5.82
E
-
20
2.09302
1.57
e
+
01
±
3.96
e
-
02
TLBO
-
0.37151
-
0.04151
-
0.20290
-
0.61175
1.50
e
-
02
±
3.60
e
-
04
36.70821
4.16
E
-
19
2.09302
1.44
e
+
01
±
8.82
e
-
02
DLTLBO
-
0.38470
-
0.09199
-
0.19445
-
0.60222
1.20
e
-
02
±
2.14
e
-
18
—
—
—
1.49
e
+
01
±
2.42
e
-
02Figure 8
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example5
(b)
Magnitude response for Example5As seen from Table8, DLTLBO offers the mean best values and standard deviations of MSE. From the values oft-test between DLTLBO and other algorithms, it is also seen that DLTLBO is statistically better than the corresponding algorithms. From Figure 8, one can see that convergence speed of TLBO is faster than the other two algorithms. It can also be observed from Table 8 that DLTLBO spends the third most times among the five algorithms when the number of FEs reaches the given maximum FEs. It can be concluded that DLTLBO could have the best MSE performance among all the five algorithms, although all five algorithms cannot find the global minimum.


## 5.1. Digital IIR Filter Identification

Application of IIR filter in system identification has been widely studied since many problems encountered in signal processing can be characterized as a system identification problem. Therefore, in the simulation study, digital IIR filters are designed for the system identification purpose. In this case, the parameters of the filters are successively adjusted by evolutionary optimization algorithms until the error between the output of the filter and the unknown system is minimized. In general, we expect that the number of zero points is more than the number of pole points. A schematic of filter modeling problem using the evolutionary algorithms is shown in Figure3.Figure 3
Filter design using a heuristic search algorithm.The general expression of digital IIR filters is given by the following difference equation [35]:(7)yk=∑i=0Maixk-i-∑i=1Nbiyk-i,where x(k) is the input of the digital IIR filter at time k, y(k) is the output of the digital IIR filter at time k, M and N are denominator and numerator orders, N(N≥M) is the filter’s order, ai and bi are adjustable filter coefficients of the model, and b0=1. The number of the digital IIR filter coefficients is (N+M+1).For adaptive digital IIR filter design, the objective of the adaptation is to adjust filter coefficients of a digital filter to estimate actual parameter values of an unknown system from its inputs and outputs. The design of this filter can be considered as a minimization problem of the cost function. The most commonly used approach to digital IIR filter design is to formulate the problem as an optimization problem with the mean square error (MSE) as the cost function to be minimized:(8)Jv=Ee2k=1S∑k=1SidealHk-actualHk2,where v=a0a1⋯aMb1b2⋯bN denotes the digital IIR filter coefficient vector, S is the number of samples used for the computation of the time-averaged cost function, and ideal(k) and actual(k) are the filter’s ideal and actual responses, respectively.To achieve higher stop band attenuation and to have better control on the transition width, the frequency cost function [27] to be minimized is defined as(9)Jv=1Sp∑ωpabsabsactualHω-1-δp+1Ss∑ωsabsactualHω-δs,where idealH(w) and actualH(w) are the filter’s ideal and actual frequency responses, respectively. δp and δs are the ripples in the pass band and stop band, ωp and ωs are pass band and stop band normalized cutoff frequencies, respectively, and Sp and Ss are the number of samples used in the pass band and stop band. In this case, the best fitness J(v)=0 represents that the magnitude response of the solution is within [1-δp,1] in the pass band and [0,δs] in the stop band.


## 5.2. Description of the DLTLBO for the Digital IIR Filter

The steps of the DLTLBO for the digital IIR filter are given as follows.Step 1.
Initialize every real coded learnerv=a0a1⋯aMb1b2⋯bN(N≥M) of the population. Each learner consists of M numerator filter coefficients and N denominator filter coefficients; the total number of filter coefficients is N+M+1 for Nth order filter to be designed. To maintain the stability of IIR filter, the efficient way of [17] is adopted. Hence, the coefficients ai are formed by the constraints [-2,2] and all the lattice-reflection-form coefficients ki of the direct-form coefficients bi are less than 1.Step 2.
Compute the error fitness value for each learner according to the cost function.Step 3 (teacher phase).
Each learner learns from the teacher according to (4)–(6). Compute the error fitness value for each learner and update each learner.Step 4 (learner phase).
Each learner randomly learns from the other learner according to (3). Compute the error fitness value for each learner and update each learner.Step 5 (termination criterion).
Stop if the termination criterion is met; the dimensional values of the global teacher are the coefficients of the designed digital IIR filter; otherwise, repeat from Step3.


## 5.3. Experimental Simulation

This section presents experimental evaluation of DLTLBO on designing digital IIR filters. Some benchmark problems are selected to compare the performance of GA [36], PSOwFIPS [33], SaDE [32], TLBO [1, 2], and DLTLBO algorithms. In all cases, the maximal value of FEs is used as ended condition of algorithm, namely, 25000 for GA, PSOwFIPS, SaDE, TLBO, and DLTLBO, respectively; the number of population is set to 50, and the initial population is selected randomly. For GA, the crossover rate Pc is set to 0.9 and the mutation rate Pm is set to 0.1. For PSOwFIPS,w = 0.7298. For SaDE, F~N(0.5,0.3), CR0=0.5, CR ~N(CRm, 0.1), and LP = 50. For our DLTLBO algorithm, the scaling factor F is set to 0.9 and the crossover probability CR is set to 0.1.Example 1.
In the first example, the transfer function of the unknown plant is unknown and only the performance parameters are given in Table3, including low-pass (LP), high-pass (HP), band-pass (BP), and band-stop (BS) filters.Table 3
Design parameters of IIR LP, HP, BP, and BS filters.
Type of filter
Pass band rippleδp
Stop band rippleδs
Pass band edgeωp
Stop band edgeωs
Low-pass (LP)
0.01
0.001
0.45
0.5
High-pass (HP)
0.01
0.001
0.5
0.45
Band-pass (BP)
0.01
0.001
0.3 and 0.7
0.25 and 0.75
Band-stop (BS)
0.01
0.001
0.25 and 0.75
0.3 and 0.7In this experiment, the filter orders are set to 8 for all IIR LP, HP, BP, and BS filters and the number of samples is set to 128. The minimum and maximum values of the coefficients are −2 and 2. The cost function is given as shown in (9). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 4 shows the maximum values and the mean values of stop band attenuation, the means and standard deviations of MSE, and the means and standard deviations of CPU times. Figure 4 shows the comparison of convergence curves, magnitude responses, and gain responses of five algorithms. In addition, a statisticalt-test [21] is given in Table 4.Table 4
Statistical data for stop band attenuation, MSE, and CPU times for 8th order IIR filters.
Type of filter
Algorithm
Maximum gain (dB)
Mean gain (dB)
MSE
t-stat
p value
t-value
Time (s)
Low-pass (LP)
GA
28.13
40.37
2.01
e
-
001
±
3.40
e
-
002
13.60792
3.01
E
-
11
2.093024
4.24
e
+
000
±
1.01
e
-
002
PSOwFIPS
30.22
43.51
1.10
e
-
001
±
1.94
e
-
002
5.14552
5.75
E
-
05
2.093024
5.17
e
+
000
±
4.69
e
-
003
SaDE
31.43
48.23
8.25
e
-
002
±
3.25
e
-
002
1.31278
2
.
0
5
E
-
01
2.093024
5.77
e
+
000
±
5.46
e
-
002
TLBO
28.74
41.18
9.40
e
-
002
±
3.62
e
-
002
3.03097
6.87
E
-
03
2.093024
4.80
e
+
000
±
2.12
e
-
002
DLTLBO
26.21
38.37
6.68
e
-
002
±
3.13
e
-
002
—
—
—
4.74
e
+
000
±
1.17
e
-
002
High-pass (HP)
GA
16.85
28.28
1.92
e
-
001
±
4.54
e
-
002
12.53964
1.23
E
-
10
2.09302
4.26
e
+
000
±
7.07
e
-
003
PSOwFIPS
20.16
32.46
1.24
e
-
001
±
2.59
e
-
002
8.96138
2.9
9
E
-
08
2.09302
5.20
e
+
000
±
6.41
e
-
003
SaDE
28.49
38.72
9.39
e
-
002
±
4.11
e
-
002
2.64240
1
.
6
1
E
-
02
2.09302
5.70
e
+
000
±
2.64
e
-
002
TLBO
25.39
35.37
1.12
e
-
001
±
2.89
e
-
002
5.73652
1.5
8
E
-
05
2.09302
4.78
e
+
000
±
3.74
e
-
002
DLTLBO
35.28
43.51
6.12
e
-
002
±
2.37
e
-
002
—
—
—
4.73
e
+
000
±
4.72
e
-
003
Band-pass (BP)
GA
24.60
37.28
3.24
e
-
001
±
5.96
e
-
002
7.12807
8
.
90
E
-
07
2.09302
4.81
e
+
000
±
1.21
e
-
002
PSOwFIPS
24.23
36.84
2.38
e
-
001
±
2.91
e
-
002
3.50304
2
.
3
8
E
-
03
2.09302
5.72
e
+
000
±
3.65
e
-
002
SaDE
29.73
42.19
2.00
e
-
001
±
2.77
e
-
002
-
0.1944
7
0.84787
2.09302
6.32
e
+
000
±
4.39
e
-
002
TLBO
25.91
39.52
2.34
e
-
001
±
4.00
e
-
002
2.80619
0.01127
2.09302
5.37
e
+
000
±
3.20
e
-
002
DLTLBO
27.90
40.27
2.01
e
-
001
±
3.25
e
-
002
—
—
—
5.28
e
+
000
±
5.08
e
-
003
Band-stop (BS)
GA
16.24
28.31
3.62
e
-
001
±
7.40
e
-
002
10.51046
2.3
5
E
-
09
2.09302
4.81
e
+
000
±
5.23
e
-
003
PSOwFIPS
17.75
30.28
2.60
e
-
001
±
3.90
e
-
002
6.70940
2.0
6
E
-
06
2.09302
5.77
e
+
000
±
8.13
e
-
002
SaDE
21.17
35.22
2.04
e
-
001
±
5.80
e
-
002
2.44724
0.0242
9
2.09302
6.34
e
+
000
±
2.20
e
-
002
TLBO
14.01
26.25
2.30
e
-
001
±
7.20
e
-
002
3.56938
2
.
0
5
E
-
03
2.09302
5.34
e
+
000
±
3.13
e
-
002
DLTLBO
36.07
46.27
1.60
e
-
001
±
4.62
e
-
002
—
—
—
5.33
e
+
000
±
1.01
e
-
002Figure 4
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for LP
(b)
Magnitude response for LP
(c)
Gain response for LP
(d)
Convergence curves for HP
(e)
Magnitude response for HP
(f)
Gain response for HP
(g)
Convergence curves for BP
(h)
Magnitude response for BP
(i)
Gain response for BP
(j)
Convergence curves for BS
(k)
Magnitude response for BS
(l)
Gain response for BSAs seen from Table4, it is observed that the proposed 8th order IIR filter design using DLTLBO spends the less CPU running time among all the three algorithms except GA. For low-pass (LP) filter, it is observed that the maximum stop band attenuations 28.13 dB, 30.22 dB, 31.43 dB, 28.74 dB, and 26.21 dB are obtained for GA, PSOwFIPS, SaDE, TLBO, and DLTLBO, respectively. Statisticalt-test between other algorithms and DLTLBO also indicates that DLTLBO performs well. Comparative curves in Figures 4(a)~4(c) also explore that the proposed 8th order IIR filter design using SaDE attains the highest stop band attenuation, although DLTLBO obtains the least MSE. Figure 4(a) shows the convergence curves for the designed 8th order IIR LP filter obtained for different algorithms, Figure 4(b) shows the comparative magnitude plots in dB, and Figure 4(c) represents the comparative normalized gain plots for 8th order IIR LP filter. For high-pass (HP) filter and band-stop (BS) filter, it is seen that the proposed 8th order IIR filter design using DLTLBO attains the highest stop band attenuation and the least MSE. It is also observed from Table 4 and Figure 4 that, for band-pass (BP) filter, the proposed 8th order IIR filter design using SaDE attains the highest stop band attenuation and the least MSE, and DLTLBO obtains the second highest stop band attenuation and the second least MSE. It can be concluded that DLTLBO has a good performance of the solution accuracy, although DLTLBO needs more CPU running time.Example 2.
In the first example (taken from [21, 22, 26]), a second-order IIR filter is estimated by a second-order IIR filter. The unknown plant and the filter had the following transfer functions:(10)idealHz=11-1.2z-1+0.6z-2,actualHz=11+b1z-1+b2z-2.The inputx(k) to the system and the filter is a white sequence. The sequence length used is N=100.Since the filter order is equal to that of the system, the error surface is unimodal. The best solution should be located at{-1.2,0.6}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 5 shows the best filter coefficients, the means and standard deviations of MSE, the means and standard deviations of CPU times, and statisticalt-test between DLTLBO and other algorithms. Figure 5 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 5
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
b
1
b
2
MSE
t-stat
p value
t-value
Time (s)
GA
-
1.20001
0.60000
1.38
e
-
04
±
3.37
e
-
04
1.83706
0.08190
2.09302
1.41
e
+
01
±
2.52
e
-
02
PSOwFIPS
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.50
e
+
01
±
3.44
e
-
02
SaDE
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.60
e
+
01
±
7.57
e
-
02
TLBO
-
1.20000
0.60000
0.0
±
0.0
NaN
NaN
NaN
1.43
e
+
01
±
4.70
e
-
02
DLTLBO
-
1.20000
0.60000
0.0
±
0.0
—
—
—
1.48
e
+
01
±
1.28
e
-
02Figure 5
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example2
(b)
Magnitude response for Example2From the values of MSE in Table5, one can see that PSOwFIPS, SaDE, TLBO, and DLTLBO can find the global minimum. From the values oft-test, it is also seen that DLTLBO is statistically better than GA. As seen from Table 5 and Figure 5, although DLTLBO spends more time than TLBO when the number of FEs reaches the given maximum FEs, DLTLBO has the faster convergence speed than other algorithms. It can be concluded that DLTLBO has a good performance of the solution accuracy.Example 3.
In the second example (taken from [17, 18, 21, 22, 26]), a second-order IIR filter is estimated by a first-order IIR filter. The unknown plant and the filter had the following transfer functions:(11)idealHz=0.05-0.4z-11-1.1314z-1+0.25z-2,actualHz=a01+b1z-1.The inputx(k) to the system and the filter is a Gaussian white noise with the mean of zero and unit variance. The sequence length used is N=100.Since the reduced order filter is employed for the identification, the error surface of the cost function is multimodal. The error surface has a global minimum at{-0.311,-0.906} and a local minimum at {0.114,0.519}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 6 shows the comparative results between DLTLBO and other algorithms. Figure 6 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 6
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
b
1
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.25418
-
0.92619
1.83
e
-
01
±
6.74
e
-
05
2582.69885
3.76
E
-
54
2.09302
1.32
e
+
01
±
6.23
e
-
02
PSOwFIPS
-
0.26621
-
0.92312
2.11
e
-
01
±
3.69
e
-
02
8.08353
1.4
4
E
-
07
2.09302
1.41
e
+
01
±
7.63
e
-
02
SaDE
-
0.31900
-
0.92058
2.01
e
-
01
±
7.43
e
-
17
16.75618
1.4
4
E
-
12
2.09302
1.46
e
+
01
±
7.22
e
-
02
TLBO
-
0.26675
-
0.91228
2.07
e
-
01
±
4.37
e
-
17
18.25943
1.4
4
E
-
15
2.09302
1.33
e
+
01
±
4.84
e
-
02
DLTLBO
-
0.36707
-
0.89206
1.44
e
-
01
±
5.37
e
-
17
—
—
—
1.40
e
+
01
±
9.93
e
-
02Figure 6
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example3
(b)
Magnitude response for Example3Table6 shows the best filter coefficients, the means and standard deviations of MSE, and CPU times. It is observed that DLTLBO has the smaller mean value of MSE, while the standard deviations of MSE obtained by SaDE, TLBO, and DLTLBO are approximate. It can be observed from Table 6 that when the number of FEs reaches the given maximum FEs, GA spends less time among all other three algorithms and DLTLBO spends less time than PSOwFIPS and SaDE. It is also seen that the differences of the means of MSE are statistically significant from the values oft-test between DLTLBO and other corresponding algorithms. As seen from Table 6 and Figure 6, the convergence speed of DLTLBO is faster than those of other algorithms.Example 4.
In the third example (taken from [24]), a third-order IIR filter is estimated by a third-order IIR filter. The unknown plant and the filter had the following transfer functions:(12)idealHz=-0.2+0.4z-1-0.5z-21-0.6z-1+0.25z-2-0.2z-3,actualHz=a0+a1z-1+a2z-21+b1z-1+b2z-2+b3z-3.The inputx(k) to the system and the filter is a Gaussian white noise with the mean of zero and unit variance. The sequence length used is N=100.Since the filter order is equal to that of the system, the error surface is unimodal. The best solution should be located at{-0.2,0.4,-0.5,-0.6,0.25,-0.2}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. Table 7 shows the comparative results between DLTLBO and other algorithms. Figure 7 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 7
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
a
1
a
2
b
1
b
2
b
3
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.19546
0.39125
-
0.49149
-
0.57708
0.22420
-
0.18713
1.47
e
-
03
±
7.77
e
-
04
8.46158
7.21
E
-
08
2.09302
1.44
e
+
01
±
9.72
e
-
02
PSOwFIPS
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.54
e
-
17
±
1.40
e
-
17
4.9250
3
9.40
E
-
05
2.09302
1.53
e
+
01
±
1.32
e
-
01
SaDE
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
00
±
00
-
1.70613
0.104277
2.09302
1.62
e
+
01
±
7.40
e
-
02
TLBO
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.20
e
-
32
±
2.74
e
-
32
1.6650
4
0.11231
2.09302
1.48
e
+
01
±
4.75
e
-
02
DLTLBO
-
0.20000
0.40000
-
0.50000
-
0.60000
0.25000
-
0.20000
1.89
e
-
33
±
4.96
e
-
33
—
—
—
1.52
e
+
01
±
1.82
e
-
02Figure 7
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example4
(b)
Magnitude response for Example4As seen from Table7, the filter coefficients obtained by PSO and DLTLBO are exactly located at the best solution. Table 7 lists that PSOwFIPS, SaDE, TLBO, and DLTLBO have the same best filter coefficients. SaDE offers the best MSE performance and DLTLBO outperforms GA, PSOwFIPS, and TLBO. Thet-test in terms of the means of MSE between DLTLBO and other algorithms indicates that DLTLBO is statistically better than the corresponding algorithms except for SaDE. From Table 7 and Figure 7, we can see that the convergence speed of DLTLBO is faster than those of other algorithms except SaDE. It can be concluded that DLTLBO has a good performance of the solution accuracy, although DLTLBO needs more CPU running time when the number of FEs reaches the given maximum FEs.Example 5.
In the fourth example (taken from [17, 18]), a third-order filter is estimated by a second-order IIR filter. The unknown plant and the filter had the following transfer functions:(13)idealHz=-0.3+0.4z-1-0.5z-21-1.2z-1+0.5z-2-0.1z-3,actualHz=a0+a1z-11+b1z-1+b2z-2.The inputx(k) to the system and the filter is a uniformly distributed white sequence; taking values from (-0.5,0.5), the SNR = 30 dB. The sequence length used is N=100.Since the reduced order filter is employed for the identification, the error surface of the cost function is multimodal. The error surface has a global minimum at{-0.39,-0.077,-0.22,-0.58}. The cost function is given as shown in (8). All the results are averaged over 20 random runs with randomly chosen initial positions. The comparative results between DLTLBO and other algorithms are given in Table 8. Figure 8 presents the normalized cost function value versus number of cost function evaluations and the magnitude responses of three algorithms.Table 8
Optimized coefficients and performance comparison of MES and CPU time.
Algorithm
a
0
a
1
b
1
b
2
MSE
t-stat
p value
t-value
Time (s)
GA
-
0.41403
-
0.06370
-
0.26851
-
0.52414
1.32
e
-
02
±
1.14
e
-
04
46.6421
4.61
E
-
21
2.09302
1.42
e
+
01
±
1.82
e
-
02
PSOwFIPS
-
0.15324
-
0.00954
-
1.57619
0.66054
1.53
e
-
02
±
1.57
e
-
03
9.45093
1.3
E
-
08
2.09302
1.50
e
+
01
±
2.84
e
-
02
SaDE
-
0.40588
-
0.03978
-
0.25151
-
0.54818
1.35
e
-
02
±
3.56
e
-
18
41.65782
5.82
E
-
20
2.09302
1.57
e
+
01
±
3.96
e
-
02
TLBO
-
0.37151
-
0.04151
-
0.20290
-
0.61175
1.50
e
-
02
±
3.60
e
-
04
36.70821
4.16
E
-
19
2.09302
1.44
e
+
01
±
8.82
e
-
02
DLTLBO
-
0.38470
-
0.09199
-
0.19445
-
0.60222
1.20
e
-
02
±
2.14
e
-
18
—
—
—
1.49
e
+
01
±
2.42
e
-
02Figure 8
Comparison of the performance curves using different algorithms.
(a)
Convergence curves for Example5
(b)
Magnitude response for Example5As seen from Table8, DLTLBO offers the mean best values and standard deviations of MSE. From the values oft-test between DLTLBO and other algorithms, it is also seen that DLTLBO is statistically better than the corresponding algorithms. From Figure 8, one can see that convergence speed of TLBO is faster than the other two algorithms. It can also be observed from Table 8 that DLTLBO spends the third most times among the five algorithms when the number of FEs reaches the given maximum FEs. It can be concluded that DLTLBO could have the best MSE performance among all the five algorithms, although all five algorithms cannot find the global minimum.


## 6. Conclusion and Future Work

In this paper, TLBO has been extended to DLTLBO which uses neighborhood search and differential learning to improve the TLBO algorithm’s computation performance. The proposed DLTLBO algorithm maintains the diversity of the population effectively and provides more useful information in the iterated process. To demonstrate the effectiveness of our approaches, 24 benchmark functions have been used for simulating and testing. Moreover, DLTLBO has been also developed to optimize the coefficients setting of digital IIR filters simultaneously. In the experimental study, DLTLBO gained good results on five digital IIR filter design cases even though DLTLBO needs to spend much more time when the number of FEs reaches the given maximum FEs. It is shown that the proposed algorithm is potential for function optimization and parameter estimation of digital IIR system in comparison with other approaches.Further works include research into the following topics. First, neighborhood topological structures and selection of parameters should be investigated to make the DLTLBO algorithm more efficient. Secondly, DLTLBO of this paper puts emphasis on the use of differential learning, rather than on analyzing the influence of different learning formations of differential evolution on DLTLBO. The latter problem would be interesting to investigate in future research. Thirdly, DLTLBO could be used in a wide variety of filter design applications. Tradeoff between various parameters of the filter can lead to designing different kinds of filters according to different requirements in various kinds of applications. Finally, it is expected that DLTLBO will be used in other real-world optimization problems.



---

*Source: 754562-2015-08-31.xml*