Adaptive Forecasting of Non-Stationary
Nonlinear Time Series based on the
Evolving Weighted Neuro-Neo-FuzzyANARX-Model
Zhengbing Hu
School of Educational Information Technology, Central China Normal University, Wuhan, China
Email: hzb@mail.ccnu.edu.cn

Yevgeniy V. Bodyanskiy
Kharkiv National University of Radio Electronics, Kharkiv, Ukraine,
Email: yevgeniy.bodyanskiy@nure.ua

Oleksii K. Tyshchenko and Olena O. Boiko
Kharkiv National University of Radio Electronics, Kharkiv, Ukraine,
Email: lehatish@gmail.com, olena.boiko@ukr.net

Abstract — An evolving weighted neuro-neo-fuzzy-ANARX model and its learning procedures are introduced in the article. This
system is basically used for time series forecasting. It’s based on neo-fuzzy elements. This system may be considered as a pool of
elements that process data in a parallel manner. The proposed evolving system may provide online processing data streams.
Index Terms — Computational Intelligence, time series prediction, neuro-neo-fuzzy System, Machine Learning, ANARX, Data
Stream.

I. INTRODUCTION
Mathematical forecasting of data sequences (time series) is nowadays well studied and there is a large number of
publications on this topic. There are many methods for solving this task: regression, correlation, spectral analysis,
exponential smoothing, etc., and more advanced intellectual systems that sometimes require rather complicated
mathematical methods and high user’s qualification. The problem becomes more complicated when analyzed time
series are non-stationary and nonlinear and contain unknown behavior trends, quasiperiodic, stochastic and chaotic
components. The best results are shown by nonlinear forecasting models based on mathematical methods of
computational intelligence [1-3], and, first of all, neuro-fuzzy systems [4-5] due to their approximating and
extrapolating properties, learning abilities, transparency and results’ interpretability. The models to be especially
noted are the so-called NARX-models [6] which have the form
yˆ  k   f y  k  1 , ..., y  k  n y  , x  k  1 , ..., x  k  nx  (1)





where ŷ  k  is an estimate of forecasted time series at discrete time k  1, 2, ... ; f    stands for a certain nonlinear
transformation implemented by a neuro-fuzzy system, x  k  is an observed exogenous factor that defines a behavior
of y  k  . It can be noticed that popular Box–Jenkins AR-, ARX-, ARMAX-models as well as nonlinear NARMAmodels can be described by the expression (1). These models have been widely studied; there are many architectures
and learning algorithms that implement these models, but it is assumed that models’ orders n y , nx are given a
priori. These orders are previously unknown in a case of structural non-stationarity for analyzed time series, and
they also have to be adjusted during a learning procedure. In this case, it makes sense to use evolving connectionist
systems [7-10] that adjust not only their synaptic weights and activation-membership functions, but also their
architectures. There are many algorithms that implement these learning methods both in a batch mode and in a
sequential mode. The problem becomes more complicated if data are fed to the system with high frequency in the
form of a data stream [11]. Here, the most popular evolving systems turn out to be too cumbersome for learning and
information processing in an online mode.

As an alternative, a rather simple and effective architecture can be considered. It’s the so-called ANARX-model
(Additive NARX) that has the form [12, 13]
yˆ  k   f1  y  k  1 , x  k  1   f 2  y  k  2  , x  k  2    ...
n

...  f n  y  k  n  , x  k  n     f l  y  k  l  , x  k  l  



(2)

l 1



(here n  max n y , nx ), an original task of the forecasting system’s synthesis is decomposed into many local tasks
of parametric identification for node models with two input variables y  k  l  , x  k  l  , l  1, 2, ..., n, ... .
Authors [12, 13] used elementary Rosenblatt perceptrons with sigmoidal activation functions as such nodes. The
ANARX-model provided high forecasting quality, but generally speaking it requires a large number of nodes in its
architecture [14].
Some synthesis problems of forecasting neuro-fuzzy [15] and neo-fuzzy [15-17] systems based on the ANARXmodels are considered in this work. These systems avoid the above mentioned drawbacks.
Since we consider a case of stochastic nonlinear dynamic signals in this article, the basic novelty has to do with
defining a model’s delay order in an online mode.
II. A NEURO-FUZZY-ANARX-MODEL
An architecture of the ANARX-model is shown in Fig.1. It is formed by two lines of time delay elements z 1

 z y  k   y  k  1 
1

and n nodes N [l ] which are simultaneously learned. These nodes are tuned independently

from each other. And adding new nodes or removing unnecessary ones doesn’t have any influence on other neurons,
i.e. the evolving process for this system is implemented by changing a number of the nodes.
It’s recommended to use a neuron with two inputs (its architecture is shown in Fig.2) as a node of this system
instead of the elementary Rosenblatt perceptron.

Figure 1. The ANARX-model

Figure 2. A neuro-fuzzy node of the ANARX-model
As one can see, this node is the Wang–Mendel neuro-fuzzy system [18] with two inputs. It possesses universal
approximation capabilities and is actually the zero-order Takagi–Sugeno–Kang system [19, 20]. A two-dimensional
vector of input signals zl  k    y  k  l  , x  k  l   is fed to an input of the node N [l ] , l  1, 2, ..., n, ... . The first
T

layer contains 2h membership functions iy  y  k  l   , ix  x  k  l   and fulfills fuzzification of the input

variables by calculating membership levels 0  iy  y  k  l    1 , 0  ix  x  k  l    1 . The Gaussian functions can

be used as membership functions



 y k  l   c
iy
iy  y  k  l    exp  
2

2 iy




2


,



  x  k  l   c 2 
ix

ix  x  k  l    exp  


2 ix2


(here ciy , cix stand for parameters that define centers of these membership functions;  iy ,  ix are width parameters)

or other bell-shaped functions with infinite support for avoiding “gaps” which appear in the fuzzified space.
The second layer of the node provides aggregation of the membership levels that were computed in the first
layer. Outputs of the second layer form h aggregated signals
zil  k   iy  y  k  l   ix  x  k  l   .
The third layer contains synaptic weights that are adjusted during a learning procedure. Outputs of the third layer
are values
wil iy  y  k  l   ix  x  k  l    wil zil  k  .
The fourth layer is formed by two summation units and calculates sums of the output signals in the second and
the third layers correspondingly. Outputs of the fourth layer are signals
h
 h l



w

y
k
l

x
k
l
wil zil  k ,










i
iy
ix

 i 1
i 1
 h
h
   y  k  l     x  k  l    z l  k .

iy
ix
i

i 1
i 1
Defuzzification (normalization) is implemented in the fifth (output) layer. Finally, the output signal of the node
yˆ l  k   fl  y  k  l  , x  k  l   is computed:
h

yˆ l  k  

 w   y  k  l    x  k  l 
l
i

i 1
h

iy

ix

 iy  y  k  l   ix  x  k  l  



i 1

h

zil  k 

i 1

 z  k 

  wil

h

i 1

l
i

h

  wilil  k   wlT l  k 
i 1

1

T
 h

, il  k   zil  k    zil  k   , wl  w1l , w2l , ..., whl .
 i 1

Considering that the output signal yˆ l  k  of each node depends linearly on the adjusted synaptic weights wil ,



where  l  k   1l  k  , 2l  k  , ..., hl  k 



T





one can use conventional algorithms of adaptive linear identification [21] for their tuning which are based on the
quadratic learning criterion.
If a training data set is non-stationary [22], one can use either the exponentially weighted recurrent least squares
method
 wl  k   wl  k  1 

 l  k  1 y  k   wlT  k  1  l  k   l  k 
,

   lT  k  l  k  1  l  k 

(3)
 l
l
  k     k  1 
 l
l
lT
l
   k  1   k    k    k  1
    lT  k  l  k  1  l  k  , 0    1,

or the Kaczmarz–Widrow–Hoff optimal gradient algorithm (in a case of the “rapid” non-stationarity)
y  k   wlT  k  1  l  k  l
  k  . (4)
wl  k   wl  k  1 
 lT  k   l  k 





In fact, it is possible to tune 4h membership functions’ parameters ciy , cix ,  iy ,  ix of each node, but taking
into consideration the fact that the signal yˆ l  k  depends nonlinearly on these parameters, a learning speed can’t be

sufficient for non-stationary conditions in this case.
III. A NEO-FUZZY-ANARX-MODEL

If a large data set to be processed is given (within the “Big Data” conception [23] when data processing speed
and computational simplicity come to the forefront, it seems reasonable to use neo-fuzzy neurons that were
proposed by T. Yamakawa and his co-authors [15-17] instead of the neuro-fuzzy nodes in the ANARX-model. An
architecture of the neo-fuzzy neuron as a node of the ANARX-model is shown in Fig.3. The neo-fuzzy neuron’s
advantages are a high learning speed, computational simplicity, good approximating properties and abilities to find a
global minimum of a learning criterion in an online mode.

Figure 3. A neo-fuzzy node for the ANARX-model
Structural elements of the neo-fuzzy neuron are nonlinear synapses NS y , NS x that implement the zero order
Takagi–Sugeno fuzzy inference, but it’s easy to notice that the neo-fuzzy neuron is much simpler constructively
than the neuro-fuzzy node shown in Fig.2.
An output value of this node is formed with the help of input signals y  k  l  , x  k  l 
h

yˆ l  k    yl  k   xl  k    wiyl iy  y  k  l   
i 1

(5)

h

  wixl ix  x  k  l  
i 1

and an output signal of the ANARX-model can be written down in the form
n
h
 h

yˆ  k      wiyl iy  y  k  l     wixl ix  x  k  l    ,
l 1  i 1
i 1

which means that since the neo-fuzzy neuron is also an additive model [24], the ANARX-model based on the neofuzzy neurons is twice additive.
Triangular membership functions are usually used in the neo-fuzzy neuron. They meet the unity partitioning
conditions
h

h

   y  k  l   1 ,    x  k  l   1 ,
i 1

iy

i 1

ix

which make it possible to simplify the node’s architecture by excluding the normalization layer from it.
It was proposed to use B-splines as membership functions for the neo-fuzzy neuron [25]. They provide a higher
approximation quality and also meet the unity partitioning conditions. A B-spline of the q  th order can be written
down:



1, if ciy  y  k  l   ci 1, y , 
 for q  1
0 otherwise




 y  k  l   ciy q 1
iy  y  k  l   

 ci  q 1, y  ciy
q
iy  y  k  l    
 ci  q , y  y  k  l  q 1
i 1, y  y  k  l   , for q  1
 c
c
 i  q , y i 1, y



i  1, ..., h  q,





1, if cix  x  k  l   ci 1, x , 
 for q  1
0 otherwise




 x  k  l   cix q 1
ix  x  k  l   


c
c


1,
i
q
x
ix

ixq  x  k  l    
 ci  q , x  x  k  l  q 1
i 1, x  x  k  l   , for q  1
 c
c
 i  q , x i 1, x



i  1, ..., h  q.



It should be mentioned that B-splines are a sort of generalized membership functions: when q  2 one gets
traditional triangular membership functions; when q  4 one gets cubic splines, etc.





Introducing a vector of variables  l  k   1 y  y  k  l   , ...,  hy  y  k  l   , 1x  x  k  l   , ...,  hx  x  k  l   ,



l
l
wl  w1l y , ..., why
, w1l x , ..., whx



T

T

, the expression (5) can be rewritten in the form

yˆ  k   w   k  .
l

lT

l

Either the algorithms (3), (4) or the procedure [26]





 wl  k   wl  k  1  rl 1  k  y  k   wlT  k  1  l  k   l  k 
(6)

lT
l
 rl  k    rl  k  1    k    k  , 0    1,

can be used for the neo-fuzzy neuron’s learning. This procedure possesses both filtering and tracking properties. It
should also be noticed that when   1 the equation (6) coincides completely with the Kaczmarz–Widrow–Hoff
optimal algorithm (4).
Evolving systems based on the neo-fuzzy neurons demonstrated its effectiveness for solving different tasks
(especially forecasting tasks [25-30]). The twice additive system is the most appropriate choice for data processing
in Data Stream Mining tasks [11] from a point of view of implementation simplicity and a processing speed.
IV. A WEIGHTED NEURO-NEO-FUZZY-ANARX-MODEL
Considering that every node N [l ] of the ANARX-model is tuned independently from each other and represents
an individual neuro(neo)-fuzzy system, it is possible to use a combination of neural networks’ ensembles [31] in
order to improve a forecasting quality. This approach results in an architecture of a weighted neuro-neo-fuzzyANARX-model (Fig.4).

Figure 4. The weighted ANARX-model
An output signal of the system can be written in the form
n

yˆ  k    cl yˆ l  k   c T yˆ  k 
l 1

T

T
where yˆ  k   yˆ 1  k  , yˆ 2  k  , ..., yˆ n  k  , c   c1 , c2 , ..., cn  is a vector of adjusted weight coefficients that define





proximity of the signals yˆ  k  to the forecasted process y  k  and meet the unbiasedness condition
l

n

c
l 1

where I n is a  n  1 -vector of unities.

 cT I n  1

l

The method of undetermined Lagrange multipliers can be used for finding the vector с in a batch mode. So, a
sequence of forecasting errors


v  k   y  k   yˆ  k   y  k   c T yˆ  k   c T I n y  k   c T yˆ  k  
,

 c T I n y  k   yˆ  k   c TV  k 





the Lagrange function





L  c,     c TV  k  V T  k  c   c T I n  1 
k





 c T Rc   c T I n  1

(7)

(here  is an undetermined Lagrange multiplier, R   V  k  V T  k  stands for an error correlation matrix) and the
k

Karush–Kuhn–Tucker system of equations

 с L  c,    2 Rc   I n  0,

 L
 cT I n  1  0

 
are introduced. A solution of the system (8) can be written down in the form
c  R 1 I I T R 1 I 1 ,
n
n
n


  2 I nT R 1 I n

where a value of the Lagrange function (7) at the saddle point is





L *  c,    I nT R 1 I n



1



(8)

(9)

.

An implementation procedure of the algorithm (9) can meet some problems in a case of data processing in an
online mode and high level of correlation between the signals yˆ l  k  that leads to the ill-conditioned matrix R
which has to be inverted at every time step k .
The Lagrange function (7) can be written down in the form
2

L  c,   
y  k   c T yˆ  k    c T I  1


k





n



and a gradient algorithm for finding its saddle point based on the Arrow–Hurwicz procedure [27, 32] can be written
in the form

c  k   c  k  1  c  k   c L  c,   ,


 k   k  1   k L  c,  
   
  


or

c  k   c  k  1  c  k  

 2 y  k   c T  k  1 yˆ  k  yˆ  k     k  1 I 
n


ˆ
 c  k  1  c  k  2v  k  y  k     k  1 I n ,



T
  k     k  1    k  c  k  I n  1



where c  k  ,   k  are learning rate parameters.













(10)



The Arrow–Hurwicz procedure converges to the saddle point with rather general assumptions on the values
c  k  ,   k  , but these parameters can be optimized to speed up the learning process as that is particularly
important in Data Stream Mining tasks.


That’s why the first ratio in the equation (10) should be multiplied by ŷ T  k 


yˆ T  k  c  k   yˆ T c  k  1 

2

 

c  k   2v  k  yˆ  k     k  1 yˆ T I n 


and an additional function that characterizes criterial convergence is introduced
2

y  k   yˆ T  k  c  k   v 2  k  





2

 

2c  k  v  k   2v  k  yˆ  k     k  1 yˆ T I n   .


2

 

c2  k   2v  k  yˆ  k     k  1 yˆ T I n 


A solution for the differential equation
2

 y  k   yˆ T  k  c  k 

c  k 



2



2

 

2v  k   2v  k  yˆ  k     k  1 yˆ T I n  


2

2

 

2c  k   2v  k  yˆ  k     k  1 yˆ T I n   0


gives the optimal learning rate values c  k  in the form

c  k  

v k 
2


2v  k  yˆ  k     k  1 yˆ T I n

.

Applying this value to the expression (10), final equations can be written down


v  k  2v  k  yˆ  k     k  1 I n
c k  c k  1 
,
 
2
  
ˆ
ˆ T
(11)
2v  k  y  k     k  1 y I n


  k     k  1    k  c T  k  I n  1 .

It’s easy to notice that when   k  1  0 the procedure (11) coincides with the Kaczmarz–Widrow–Hoff









algorithm (4).
V. EXPERIMENTS
In order to prove the effectiveness of the proposed system, a number of experiments should be carried out.
A. Electricity Demand
This data set describes 15 minutes averaged values of power demand in the full year 1997. Generally speaking,
this data set contains 15000 points but we took only 5000 points for the experiment. 3000 points were selected for a
training stage and 2000 points were used for testing. Prediction results of the ANARX and weighted ANARX

systems are in Fig.5 and Fig.6 correspondingly (signal values are marked with a blue color; prediction values are
marked with a magenta color; and prediction errors are marked with a grey line).
We used for comparison multilayer perceptrons (MLPs), radial-basis function neural networks (RBFNs), ANFIS
and two proposed systems ANARX and weighted ANARX (both based on neo-fuzzy nodes). Since MLP can’t work
in an online mode, it processed data in two different modes. It had just one epoch in the first case (something similar
to an online case), and it had 5 epochs in the second case (the MLP architecture had almost the same number of
adjustable parameters when compared to the proposed systems). A number of MLP inputs was equal to 4 and a
number of hidden nodes was equal to 7 in both cases. A total number of parameters to be tuned was 43 in both
modes. It took about two times more time to compute the result in the second case but prediction quality was like
almost two times higher. MLP (case 2) demonstrated the best result in this experiment.
Speaking of RBFNs, we also had two cases. The first-case RBFN was taken really close in the sense of
parameters’ number to our systems and the second-case RBFN’s architecture was chosen to show the best
performance. In the first case, RBFN had 3 inputs and 7 kernel functions. In the second case, it had 3 inputs as well
but 12 kernel functions which generally led to higher prediction quality (+30% precision compared to RBFN in the
first case) but took longer to compute the result. A number of parameters to be tuned was 36 in the first case and 61
in the second case.
ANFIS showed one of the best prediction qualities in this experiment. It had 4 inputs, 55 nodes and it was
processing data during 5 epochs. It contained 80 parameters to be tuned.
The proposed ANARX system based on neo-fuzzy nodes had 2 inputs, 2 nodes, 9 membership functions, and its
 parameter was equal to 0.62. This system had 37 parameters. Its prediction quality was rather high and it was
definitely one of the fastest system on this data set. We should also notice that we used B-splines (q=2, which means
that we used triangular membership functions) as membership functions for the proposed systems.
The proposed weighted ANARX system based on neo-fuzzy nodes had 2 inputs, 2 nodes, 8 membership
functions; its  parameter was equal to 0.9. It had 37 adjustable parameters. This system demonstrated better
performance when compared to ANARX and the fastest results.
TABLE I. COMPARISON OF THE SYSTEMS’ RESULTS
Parameters RMSE
RMSE Time,
Systems
to be tuned (training) (test)
s
MLP
43
0.0600
0.0700 0.4063
(case 1)
MLP
43
0.0245
0.0381 0.9219
(case 2)
RBFN
36
0.0681
0.0832 0.6562
(case 1)
RBFN
61
0.0463
0.0604 1.0250
(case 2)
ANFIS
80
0.0237
0.0396 0.7031
ANARX
(neo37
0.0903
0.0922 0.4300
fuzzy
nodes)
Weighted
ANARX
(neo36
0.0427
0.0573 0.3750
fuzzy
nodes)

Figure 5. Identification of a nonlinear system. Prediction results of the ANARX system

Figure 6. Identification of a nonlinear system. Prediction results of the weighted ANARX system

B. Monthly sunspot number
This data set was taken from datamarket.com. The data set describes monthly sunspot number in Zurich. It was
collected between 1749 and 1983. It contains 2820 points. 2256 points were selected for a training stage and 564
points were used for testing. Prediction results of the ANARX and weighted ANARX systems are in Fig.7 and
Fig.8 correspondingly (signal values are marked with a blue color; prediction values are marked with a magenta
color; and prediction errors are marked with a grey line).
We used a set of systems which is similar to the previous experiment to compare results. MLP_1 (MLP for the
first case) had 3 inputs, 6 hidden nodes and it was processing data during just one epoch. MLP_2 basically had the
same parameter settings except the fact that it was processing data during 5 epochs. Both MLP systems had 31
parameters to be tuned. MLP_1 was 2 times faster than MLP_2 but its prediction quality was also almost 2 times
worse.
Let’s denote our RBFN architectures as RBFN_1 (RBFN for the first case) and RBFN_2 (RBFN for the second
case). Both of them had 3 inputs. RBFN_1 had 4 kernel functions unlike RBFN_2 which had 19 hidden nodes. A
number of parameters to be tuned was 21 in the first case and 96 in the second case. RBFN_2 showed a better
prediction quality but it was much slower than RBFN_1.
ANFIS had 4 inputs and 55 nodes. It was processing data during 3 epochs. It had 80 adjustable parameters.
The proposed ANARX system based on neo-fuzzy nodes had 2 inputs, 2 nodes, 4 membership functions, and its
 parameter was equal to 0.9. This system contained 17 parameters. Its prediction quality was rather high and it
was definitely one of the fastest systems on this data set.
The proposed weighted ANARX system based on neo-fuzzy nodes had 2 inputs, 2 nodes, 4 membership
functions; its  parameter was equal to 0.9. This system contained 20 parameters to be tuned. This system
demonstrated better performance when compared to ANARX and one of the fastest results.

TABLE II. COMPARISON OF THE SYSTEMS’ RESULTS
Systems
Parameters RMSE
RMSE Time, s

MLP
(case 1)
MLP
(case 2)
RBFN
(case 1)
RBFN
(case 2)
ANFIS
ANARX
(neofuzzy
nodes)
Weighted
ANARX
(neofuzzy
nodes)

to be tuned

(training)

(test)

31

0.1058

0.1407

0.2813

31

0.0600

0.0808

0.5938

21

0.1066

0.2155

0.2219

96

0.0702

0.1638

0.9156

80

0.0559

0.1965

0.8906

17

0.1297

0.1350

0.2252

20

0.0784

0.1081

0.2981

Figure 7. Identification of a nonlinear system. Prediction results of the ANARX system

Figure 8. Identification of a nonlinear system. Prediction results of the weighted ANARX system
VI. CONCLUSION
The evolving forecasting weighted neuro-neo-fuzzy-twice additive model and its learning procedures are
proposed in the paper. This system can be used for non-stationary nonlinear stochastic and chaotic time series’
forecasting where time series are processed in an online mode under the parametric and structural uncertainty. The
proposed weighted ANARX-model is rather simple from a computational point of view and provides fast data
stream processing in an online mode.
So, the proposed evolving forecasting model has demonstrated its efficiency for solving real-world tasks. A
number of experiments has been performed to show high efficiency of the proposed neuro-neo-fuzzy system.
ACKNOWLEDGMENT
The authors would like to thank anonymous reviewers for their careful reading of this paper and for their helpful
comments.
This scientific work was supported by RAMECS and CCNU16A02015.
REFERENCES

[1]
[2]
[3]
[4]
[5]

[6]
[7]
[8]
[9]
[10]
[11]
[12]

[13]

[14]
[15]

[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]

Rutkowski L. Computational Intelligence. Methods and Techniques. Springer-Verlag, Berlin-Heidelberg, 2008.
Kruse R, Borgelt C, Klawonn F, Moewes C, Steinbrecher M, Held P. Computational Intelligence. Springer, Berlin, 2013.
Du K-L, Swamy M N S. Neural Networks and Statistical Learning. Springer-Verlag, London, 2014.
Jang J-S, Sun C-T, Mizutani E. Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine
Intelligence. Prentice Hall, Upper Saddle River, 1997.
Rezaei A, Noori L, Taghipour M. The Use of ANFIS and RBF to Model and Predict the Inhibitory Concentration Values
Determined by MTT Assay on Cancer Cell Lines. International Journal of Information Technology and Computer
Science(IJITCS), 2016, 8(4): 28-34.
Nelles O. Nonlinear System Identification. Springer, Berlin, 2001.
Kasabov N. Evolving fuzzy neural networks – algorithms, applications and biological motivation. Proc. “Methodologies for
the Conception, Design and Application of Soft Computing”, Singapore, 1998:271-274.
Kasabov N. Evolving fuzzy neural networks: theory and applications for on-line adaptive prediction, decision making and
control. Australian J. of Intelligent Information Processing Systems, 1998, 5(3):154-160.
Kasabov N. Evolving Connectionist Systems. Springer-Verlag, London, 2003.
Lughofer E. Evolving Fuzzy Systems – Methodologies, Advanced Concepts and Applications. Springer, Berlin, 2011.
Bifet A. Adaptive Stream Mining: Pattern Learning and Mining from Evolving Data Streams. IOS Press, 2010, Amsterdam.
Belikov J, Vassiljeva K, Petlenkov E, Nõmm S. A novel Taylor series based approach for control computation in NNANARX structure based control of nonlinear systems. Proc. 27th Chinese Control Conference, Kunming, China,
2008:474-478.
Vassiljeva K, Petlenkov E, Belikov J. State-space control of nonlinear systems identified by ANARX and neural network
based SANARX models. Proc. WCCI 2010 IEEE World Congress on Computational Intelligence, Barcelona, Spain,
2010:3816-3823.
Cybenko G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Systems, 1989, 2:303-314.
Yamakawa T, Uchino E, Miki T, Kusanagi H. A neo fuzzy neuron and its applications to system identification and
prediction of the system behavior. Proc. 2nd Int. Conf. on Fuzzy Logic and Neural Networks “IIZUKA-92”, Iizuka, Japan,
1992:477-483.
Uchino E, Yamakawa T. Soft computing based signal prediction, restoration, and filtering. In: Intelligent Hybrid Systems:
Fuzzy Logic, Neural Networks, and Genetic Algorithms, 1997:331-349.
Miki T, Yamakawa T. Analog implementation of neo-fuzzy neuron and its on-board learning. In: Computational
Intelligence and Applications, 1999:144-149.
Wang L-X, Mendel J M. Fuzzy basis functions, universal approximation, and orthogonal least-squares learning. IEEE Trans.
on Neural Networks, 1992, 3(5):807-814.
Takagi T, Sugeno M. Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. on Systems,
Man, and Cybernetics, 1985, 15:116-132.
Sugeno M, Kang G T. Structure identification of fuzzy model. Fuzzy Sets and Systems, 1998, 28:15-33.
Ljung L. System Identification: Theory for the User. Prentice Hall, Inc., Upper Saddle River, 1987.
Polikar R, Alippi C. Learning in nonstationary and evolving environments. IEEE Trans. on Neural Networks and Learning
Systems, 2014, 25(1):9-11.
Jin Y, Hammer B. Computational Intelligence in Big Data. IEEE Computational Intelligence Magazine, 2014, 9(3):12-13.
Friedman J, Hastie T, Tibshirani R. The Elements of Statistical Learning. Data Mining, Inference and Prediction. Springer,
Berlin, 2003.
Bodyanskiy Ye, Kolodyazhniy V. Cascaded multiresolution spline-based fuzzy neural network. Proc. Int. Symp. on
Evolving Intelligent Systems, Leicester, UK, 2010:26-29.
Bodyanskiy Ye, Kokshenev I, Kolodyazhniy V. An adaptive learning algorithm for a neo-fuzzy neuron. Proc. 3rd Int. Conf.
of European Union Soc. for Fuzzy Logic and Technology (EUSFLAT’03), Zittau, Germany, 2003:375-379.
Bodyanskiy Ye, Otto P, Pliss I, Popov S. An optimal algorithm for combining multivariate forecasts in hybrid systems.
Lecture Notes in Artificial Intelligence, 2003, 2774:967-973.
Bodyanskiy Ye, Viktorov Ye. The cascade neo-fuzzy architecture using cubic-spline activation functions. Int. J. Information
Theories and Applications, 2009, 16(3):245-259.
Bodyanskiy Ye, Teslenko N, Grimm P. Hybrid evolving neural network using kernel activation functions. Proc. 17th Zittau
East-West Fuzzy Colloquium, Zittau/Goerlitz, Germany, 2010:39-46.
Bodyanskiy Ye V, Tyshchenko O K, Kopaliani D S. A multidimensional cascade neuro-fuzzy system with neuron pool
optimization in each cascade. Int. J. Information Technology and Computer Science, 2014, 6(8):11-17.
Sharkey A J C. On combining artificial neural nets. Connection Science, 1996, 8:299-313.
Bodyanskiy Ye, Deineko A, Stolnikova M. Adaptive generalization of neuro-fuzzy systems ensemble. Proc. of the Int. Conf.
“Computer Science and Information Technologies”, Lviv, Ukraine, November 16-19, 2011:13-14.

