Extension neural network. Extension neural network is a pattern recognition method found by M. H. Wang and C. P. Hung in 2003 to classify instances of data sets. Extension neural network is composed of artificial neural network and extension theory concepts. It uses the fast and adaptive learning capability of neural network and correlation estimation property of extension theory by calculating extension distance. 
ENN was used in:
 Failure detection in machinery.
 Tissue classification through MRI.
 Fault recognition in automotive engine.
 State of charge estimation in lead-acid battery.
 Classification with incomplete survey data.

Extension Theory 
Extension theory was first proposed by Cai in 1983 to solve contradictory problems. While classical mathematics is familiar with quantity and forms of objects, extension theory transforms these objects to matter-element models.

where in matter ,  is the name or type,  is its characteristics and  is the corresponding value for the characteristic. There is a corresponding example in equation 2.

where  and  characteristics form extension sets. These extension sets are defined by the  values which are range values for corresponding characteristics. Extension theory concerns the extension correlation function between matter-element models like shown in equation 2, and extension sets. Extension correlation function is used to define extension space which is composed of pairs of elements and their extension correlation functions. The extension space formula is shown in equation 3.

where,  is the extension space,  is the object space,  is the extension correlation function,  is an element from the object space and  is the corresponding extension correlation function output of element .  maps  to a membership interval . Negative region represents an element not belonging membership degree to a class and positive region vice versa. If  is mapped to , extension theory acts like fuzzy set theory. The correlation function can be shown with the equation 4.

where,  and  are called concerned and neighborhood domain and their intervals are (a,b) and (c,d) respectively. The extended correlation function used for estimation of membership degree between  and ,  is shown in equation 5.

Extension Neural Network 
Extension neural network has a neural network like appearance. Weight vector resides between the input nodes and output nodes. Output nodes are the representation of input nodes by passing them through the weight vector. 

There are total number of input and output nodes are represented by  and , respectively. These numbers depend on the number of characteristics and classes. Rather than using one weight value between two layer nodes as in neural network, extension neural network architecture has two weight values. In extension neural network architecture, for instance ,  is the input which belongs to class  and  is the corresponding output for class . The output  is calculated by using extension distance as shown in equation 6.

Estimated class is found through searching for the minimum extension distance among the calculated extension distance for all classes as summarized in equation 7, where  is the estimated class.

Learning Algorithm 
Each class is composed of ranges of characteristics. These characteristics are the input types or names which come from matter-element model. Weight values in extension neural network represent these ranges. In the learning algorithm, first weights are initialized by searching for the maximum and minimum values of inputs for each class as shown in equation 8

where,  is the instance number and  is represents number of input. This initialization provides classes' ranges according to given training data.

After maintaining weights, center of clusters are found through the equation 9.

Before learning process begins, predefined learning performance rate is given as shown in equation 10

where,  is the misclassified instances and  is the total number of instances. Initialized parameters are used to classify instances with using equation 6. If the initialization is not sufficient due to the learning performance rate, training is required. In the training step weights are adjusted to classify training data more accurately, therefore reducing learning performance rate is aimed. In each iteration,  is checked to control if required learning performance is reached. In each iteration every training instance is used for training. 
Instance , belongs to class  is shown by:

Every input data point of  is used in extension distance calculation to estimate the class of . If the estimated class  then update is not needed. Whereas, if  then update is done. In update case, separators which show the relationship between inputs and classes, are shifted proportional to the distance between the center of clusters and the data points. 
The update formula:

To classify the instance  accurately, separator of class  for input  moves close to data-point of instance , whereas separator of class  for input  moves far away. In the above image, an update example is given. Assume that instance  belongs to class A, whereas it is classified to class B because extension distance calculation gives out . After the update, separator of class A moves close to the data-point of instance  whereas separator of class B moves far away. Consequently, extension distance gives out , therefore after update instance  
 is classified to class A.

References 
 
 Kuei-Hsiang Chao, Meng-Hui Wang, and Chia-Chang Hsu. A novel residual capacity estimation method based on extension neural network for lead-acid batteries. International Symposium on Neural Networks, pages 1145–1154, 2007
 Kuei-Hsiang Chao, Meng-Hui Wang, Wen-Tsai Sung, and Guan-Jie Huang. Using enn-1 for fault recognition of automotive engine. Expert Systems with Applications, 37(4):29432947, 2010
  
  
 Juncai Zhang, Xu Qian, Yu Zhou, and Ai Deng. Condition monitoring method of the equipment based on extension neural network. Chinese Control and Decision Conference, pages 1735–1740, 2010
 

Artificial neural networks