# Language Inference Using Elman Networks with Evolutionary Training
## 1 Introduction  
Recurrent neural networks (RNN) are well-established mathematical models, and they have been used widely in many scientific and industrial areas such as text processing [1; 2], speech recognition [3], sentiment classification [4], handwriting recognition [5], medical signal processing [6], etc. Elman [7] is a multi-layer neural network with recurrent units in the first layer. In its simplest form, the output of the Elman network is derived from the recurrent neuron outputs of the first layer. In the neuron internal states, the information from previous time non-linear transformations is stored and processed, also known as the memory of the cell. These models have been implemented in a wide variety of complex multidimensional classification problems [8; 9]. Due to their complex structure, error propagation [10] and hill-climbing techniques [11; 12] inherit many deficiencies. A well-known alternative, which avoids trapping the model in local minima, is the use of evolutionary methods such as the standard genetic algorithm (SGA), or some variants of the SGA [13], to estimate the internal weights or/and the structure of the Elman network [14; 15], also referred to in the literature as the GA-Elman algorithm. Furthermore, the decoding of the model's output differs significantly from application to application. In natural language classification problems, it is common to use the final output of the network for determining the class of a data instance [16], while in more complex models, the output of the recurrent nodes [17] is part of a greater network. The use of evolutionary algorithms with Elman networks has been successful in several applications [18; 19].  
In this article, a novel approach to decoding the outputs of the Elman network for binary classification and a hybrid GA-Elman algorithm, which fully automatically estimates the synaptic weights, are presented. The input data of the network are treated as positive or negative symbolic sequences, while the output of the Elman network is decoded to a scalar value representing the probability of a data instance being recognized as a positive symbolic sequence by the classifier. The hybrid GA-Elman training algorithm estimates the weights of the model using both genetic operators and invoking gradient -ased local search [20; 21] every few iterations. Even though this approach limits this classifier to binary classification problems, this method is still applicable to a wide range of problems, withseveral advantages discussed in Section 5, and extended easily to multi-class problems. The datasets used, either artificially created or obtained from popular databases, are transformed to fit the encoding scheme of our method. All the experiments were conducted tenfold by randomly splitting the instances in test and evaluation data.  
The rest of this article has the following structure: in Section 2, the proposed neural classifier and the corresponding training method are described in detail. The implementation details and the specific optimization techniques and tools are presented in Section 3. In Section 4, evaluation experiments from various scientific areas such as biology, chemistry, mathematics, etc., show the capabilities of the proposed algorithm. The accuracy of the proposed classifier and the computational advantages of the proposed classifier are summarized. In the last section, some conclusions from the use of the proposed method and implementation advances are given.