# Using Optimization Techniques in Grammatical Evolution
## 1 Introduction  
Genetic Algorithms belong to the field of evolutionary techniques [1] and were originally formulated by [PERSON] and his team [2]. Genetic Algorithms are initiated by generating a series of random candidate solutions to an optimization problem. These candidate solutions are called chromosomes, and they iteratively undergo a series of operations that have their foundation in physical processes, such as selection, crossover, and mutation [3; 4; 5]. Grammatical Evolution [6] is considered a special case of Genetic Algorithms, where the chromosomes are series of integer numbers. Chromosomes in the Grammatical Evolution process are rules for generating a Backus-Naur form (BNF) grammar [7] and can be used to create functional programs in any programming language. Grammatical Evolution has been applied on a wide series of real-world applications, such as function approximation [8; 9], credit classification [10], network security and prevention of attacks [11], monitoring of water quality [12], modeling glycemia in humans [13], automatic design of Ant algorithms [14], temperature prediction in data centers [15], solving trigonometric equations [16], composing music [17], neural network construction [18; 19], producing numeric constants [20], video games [21; 22], energy demand estimation [23], combinatorial optimization [24], cryptography [25], evolving of decision trees [26], automatic design of analog electronic circuits [27], etc. The method of Grammatical Evolution has been extended by various researchers during recent years and some examples of these extensions are Structured Grammatical Evolution [28; 29], which applies a one-to-one mapping between the chromosomes and the non-terminal symbols of the grammar; the \(\pi\)Grammatical Evolution method [30], an application of the Particle Swarm Optimization(PSO) [31] to produce programs with Grammatical Evolution denoted as Grammatical Swarm [32; 33]; the Probabilistic Grammatical Evolution [34], which introduced a new mapping mechanism for the Grammatical Evolution method, incorporation of parallel programming techniques [35; 36], usage of Christiansen grammars [37], etc. Also, a variety of software has been developed for Grammatical Evolution, such as the GEVA v2.0 [38] that proposes a GUI application for Grammatical Evolution, the gramEvol v2.1-4 software [39] that provides a package in the R programming language, the GeLab v2.0 [40] that implements a Matlab toolbox for Grammatical Evolution, the GenClass v1.0 [41] software used to produce classification programs, the QFc v1.10 software [42] for feature construction, etc. Although the method of Grammatical Evolution has proven to be extremely efficient, it can often become trapped in the local minimum of the objective problem and its performance may not be as expected. A common method to get out of such situations is the use of local optimization techniques, which have been applied many times in combination with Genetic Algorithms [43; 44]. In the case of Grammatical Evolution and due to the integral representation of the candidate solutions, local optimization techniques are not directly applicable, such as, for example, the BFGS method [45] and hence more suitable methods should be adopted. In this paper, an attempt is made to improve upon the excellent results shown by Grammatical Evolution in the past on a variety of problems by applying local optimization techniques, periodically applied to randomly selected chromosomes of Grammatical Evolution. The current work utilizes a modified version of the Simulated Annealing method [46] as a local search procedure. The method of Simulated Annealing has been utilized in various cases, such as image processing [47], protein structure optimization [48], resource allocation [49], convex optimization [50], deep learning [51], etc. To verify the possibility of the proposed methodology to improve the results of Grammatical Evolution, the neural networks construction technique was chosen.