# On Modelling and Comparative Study of LMS and RLS Algorithms for Synthesis of MSA

Authors: Ahmad Kamal Hassan; Adnan Affandi
Journal: Modelling and Simulation in Engineering
 (2016)

---

## Abstract

This paper deals with analytical modelling of microstrip patch antenna (MSA) by means of artificial neural network (ANN) using least mean square (LMS) and recursive least square (RLS) algorithms. Our contribution in this work is twofold. We initially provide a tutorial-like exposition for the design aspects of MSA and for the analytical framework of the two algorithms while our second aim is to take advantage of high nonlinearity of MSA to compare the effectiveness of LMS and that of RLS algorithms. We investigate the two algorithms by using gradient decent optimization in the context of radial basis function (RBF) of ANN. The proposed analysis is based on both static and adaptive spread factor. We model the forward side or synthesis of MSA by means of worked examples and simulations. Contour plots, 3D depictions, and Tableau presentations provide a comprehensive comparison of the two algorithms. Our findings point to higher accuracies in approximation for synthesis of MSA using RLS algorithm as compared with that of LMS approach; however the computational complexity increases in the former case.


---

## Body

## 1. Introduction

Microstrip patch antenna (MSA) has been extensively used in wireless transmission links because of its simplicity in design and fabrication, less weight, low cost, and small size. Major parameters in the design consideration of MSA include bandwidth, gain, directivity, polarization, and center frequency. It is well understood that there is a tradeoff in selection of these parameters and design engineers have to assign appropriate weights based on their work objectives [1–3]. Usually MSA finds applications in areas where the bandwidth requirement is narrow and with multiband resonance frequencies to account for diversity issues [4, 5]. There exist many other applications [6–8] where MSA can be integrated with a given automation system to better the existing results.Terminology of patch is based on the aspects of radiating element “photoetchen” on dielectric constant. Patch can be of different geometries both deterministic and random. Deterministic geometry points to triangular, elliptical, circular, square, and last but not least rectangular (used in this work) shapes of patch. Choice of patch dimension such as length, width, permeability, and thickness of the substrate plays significant role in obtaining the resonance frequency. There exist numerous models in literature that account for the determination of dimension of patch based on artificial neural networks (ANN) for modelling purposes [9, 10] but an algorithmic comparison of such models is unavailable. Nevertheless, ANN in general and radial basis function (RBF) in particular have been extensively used with excellent results in modelling and simulation techniques for other nonlinear mechanical systems [11, 12] and signal processing routines [13] and in the design aspects of MSA [14–17]. The design of MSA using ANN is subdivided into forward modelling and backward modelling. Forward modelling accounts for the synthesis of MSA and hence it is useful in obtaining both length (L) and width (W) of the patch. In backward modelling, the primary task is the extraction of resonance frequency (f). Both forward side of the problem (synthesis) and reverse side of the problem (analysis) are basic building blocks for commercially available simulation software such as advanced design system (ADS) and Ansoft high frequency simulation system (HFSS) [18]. In this work, the analysis for the synthesis of MSA using RBF is performed. We present herein an exposition of significant algorithms such as least mean square (LMS) and recursive least square (RLS) and further investigate their deviants based on the gradient decent approach (GDA). The choice of the LMS and RLS algorithm is because they are considered fundamental in many subdisciplines of engineering such as adaptive filtering and signal processing. The parameters that are fed to ANN are resonance frequency (f), permeability constant (εr), and height of the substrate (h). Consequently, width (W) and length (L) are extracted from ANN. The rationale behind the proposed work is that the MSA design and its pertinent equations are of high nonlinearity; therefore choosing an algorithm that results in minimal error between the estimated data and the desired data can be useful in the development of the aforementioned commercial software. Moreover, computational complexity of algorithms is another area which needs to be looked into before choosing an algorithm; hence we outline the complexity difference in terms of complex multiplications and additions that are performed in an algorithm. To the best of the authors’ knowledge such investigation and comparative analysis have not been done previously for the synthesis of MSA.Organization of this paper is such that, after introduction in Section1, an overview of the design aspects of MSA is given in Section 2. Section 3 introduces RBF, LMS, and RLS approaches and mathematical formulation of adaptive spread factor. Complexity analysis is presented in Section 4. Simulation results and comparative analysis of the four algorithms are given in Section 5. Subsequent section deals with concluding remarks of this work and references.


## 2. Design of the Rectangular Microstrip Patch Antenna

Patch antennas are multilayered components with conductors, that is, patch and ground plane separated by dielectric substrate. Rectangular microstrip patch antenna (R-MSA) consists of width (W) and length (L) as presented in Figure 1. The other significant parameters required in designing and fabrication of MSA include substrate thickness (h) and its permeability (εr). Choice of substrate determines the size and its relevant application base. Most common substrates on the market are RT/duroid with dielectric constant of 2.2 and liquid crystal polymer (LCP) with the range of dielectric constant between 2.9 and 3.2 [19]. Generally, there are numerous substrates available on the market for usability in MSA and in practice the permeability of dielectric with MSA in perspective is in the range of 2.2≤εr≤12. Feed is another crucial aspect in design of patch antenna and there exist multiple methods by which a signal can be fed. The simplest method is direct contact method shown in Figure 1 and thereby used in this work. Other methods can be probe feed, microstrip proximity coupling, and microstrip aperture coupling [20].Figure 1
Rectangular microstrip antenna.In MSA, physical length (L) and electrical (effective) length (Leff) are different because of fringing fields at the edges of the patch. The effective electrical length of the patch antenna is given by [2, 19–22](1)Leff=L+2ΔL,where ΔL is the change in length given by(2)ΔL=0.412hεr,eff+0.3W/h+0.264εr,eff-0.258W/h+0.8.Effective dielectric constant of the dielectric material is expressed as [19](3)εr,eff=εr+12+εr-121+12hW-1/2.The actual length of the patch under specified conditions is(4)Leff=λ2⟹L=c2fεr,eff-2ΔL.Hence the practical width is determined as follows:(5)W=c2f2εr+1.Based on the above mathematical modelling, both synthesis and analysis can be designed using soft computing tools. We, however, focus on the synthesis of MSA; where the inputs to ANN are resonance frequency (f), substrate thickness (h), and dielectric constant (εr) while length (L) and width (W) are extracted from the RBF designs. Synthesis aspect of this work is presented in Figure 2. We utilize this model as a test bench to compare LMS and RLS algorithms in the forthcoming sections.Figure 2
Synthesis of ANN-MSA.


## 3. Synthesis Using RBF-ANN

In this section we examine the feedforward architecture of the ANN based on RBF. Layout of RBF is simplistic and it consists of hidden layer and output layer neurons. The function of hidden layer is to perform a nonlinear operation on the set of inputs. For the purposes of performing a nonlinear operation, one can resort to plethora of nonlinear functions such as multiquadrics, inverse multiquadrics, and Gaussian function [21–23]. Moreover, Kernels of the RBF have been extended in multiple ways with notable work in [24–26]. In this work, we however use the conventional approach expressed in [21, 22] for our hypothesis testing. The nonlinear function pertinent to jth neuron (Φj) is considered as Gaussian function which can be expressed by means of the following expressions:(6a)Φjn=exp⁡-∑l=1Mxl-ck22σj2(6b)σj=dmax2m,where n is the time index, σj is the spread factor of jth Gaussian function and it is determined empirically, m is the total number of basis functions employed, dmax is the maximum distance between any two bases, xl is the lth input data, and ck is the kth center of basis function.Once the nonlinear activation function is performed on the input set of data (xi), the output can be extracted from output layer neuron (yj) as(7)yin=∑j=1mwjnΦjn,where m is the total number of basis functions and wj are the weighted interconnection between the hidden neurons and the output neurons.Weight update is processed by using recursion principle of RBF-ANN and its optimality is an active research area with the synthesis of MSA in perspective. An effort is made herein to model algorithms with better convergence and stability properties of the objective function which is basically error minimization. The following subsections provide an overview of algorithms utilized in approximating MSA metrics and recursively update weights and the corresponding activation function. Figure3 presents an RBF neural network where input data is fed to Gaussian function; each nonlinear activation function has a weighted interconnection with the output neuron. Our primary focus here is the minimization of objective function which accounts for the mean square error (MSE) between the desired output (d) and the estimated output (y). We use two benchmark algorithms, namely, the LMS and the RLS independently. Lastly, spread can also be recursively updated based on gradient decent approach (GDA) which has the potential to further minimize the MSE.Figure 3
LMS, RLS, and adaptive spread factor (σ) modelled using RBF-ANN.


### 3.1. Least Mean Square (LMS)

Least mean square (LMS) algorithm is used widely in the domain of adaptive filtering [27] and it is also more often than not utilized in RBF for MSA design [14]. The LMS utilizes GDA approach to recursively update the weights of neurons based on the instantaneous MSE. Consider an objective function which is simply based on instantaneous error of all the output neurons given by(8)En=12∑i=1Iyin-d2,where d corresponds to the desired output which is compared with the approximated result of the output neuron yi. Gradient of the cost function is thus calculated as(9)∂En∂wjn=∑i=1Iyin-dΦjn.Hence, the corresponding weights wj update equation can be computed and updated as(10)wjn+1=wjn-η1∂En∂wjn,where η1 is the learning rate or step size and it plays an important role in the convergence properties of the given objective function.


### 3.2. Recursive Least Square (RLS)

Recursive least square (RLS) is another powerful adaptive algorithm where the cost function is minimized by recursively updating the weights. There is a plethora of literature available on the design of RLS algorithm; therefore [28] and references therein can be insightful for interested readers. The main aim herein is not to redrive the RLS algorithm but to briefly overview its core principles. The linear least square objective function is given by(11a)En=∑i=1nfn,iyin-d2(11b)fn,i=λn-i,where λ is the forgetting factor of RLS and has values in the range of 0<λ≤1.The order of RLS denoted asP(n) and the corresponding weights update (wi) expressions are given by(12)Pn=λ-1Pn-1-Pn-1xnxTnPn-1λ+xTnPn-1xn(13)wn+1=wn+Pnxnen.Convergence of RLS is much faster as compared with LMS though computational complexity of RLS substantially increases, and this aspect is shown in the subsequent section.


### 3.3. Gradient Decent Approach for Adaptive Spread

In this subsection, the spread factor has also been made adaptive using gradient decent approach for both LMS and RLS algorithms. The relevant gradient of objective function and recursive update of spread factor are given by [21, 22](14a)∂En∂σjn=-wjn∑l=1Lyin-d2G′xl-ckσjQlkn(14b)Qlkn=xl-cknxl-cknT(14c)σjn+1=σjn-η2∂En∂σjn,where the term G′(·) in (14a) is the first derivative of Green’s function G(·) with respect to its argument and η2 is the learning rate for adaptive spread.Depiction of GDA based spread update is shown using dashed-lines of Figure3. Recursive update of spread expressed in (14a), (14b), and (14c) can greatly enhance the approximation capabilities of the neural networks since it is tuning the most significant part of RBF, namely, the activation function defined in (6a). It is also to be noted herein that a further extension of the aforementioned model is also possible in which one can update centers of the Gaussian function as well [21–23]; however subtractive clustering method is used in this work.


## 3.1. Least Mean Square (LMS)

Least mean square (LMS) algorithm is used widely in the domain of adaptive filtering [27] and it is also more often than not utilized in RBF for MSA design [14]. The LMS utilizes GDA approach to recursively update the weights of neurons based on the instantaneous MSE. Consider an objective function which is simply based on instantaneous error of all the output neurons given by(8)En=12∑i=1Iyin-d2,where d corresponds to the desired output which is compared with the approximated result of the output neuron yi. Gradient of the cost function is thus calculated as(9)∂En∂wjn=∑i=1Iyin-dΦjn.Hence, the corresponding weights wj update equation can be computed and updated as(10)wjn+1=wjn-η1∂En∂wjn,where η1 is the learning rate or step size and it plays an important role in the convergence properties of the given objective function.


## 3.2. Recursive Least Square (RLS)

Recursive least square (RLS) is another powerful adaptive algorithm where the cost function is minimized by recursively updating the weights. There is a plethora of literature available on the design of RLS algorithm; therefore [28] and references therein can be insightful for interested readers. The main aim herein is not to redrive the RLS algorithm but to briefly overview its core principles. The linear least square objective function is given by(11a)En=∑i=1nfn,iyin-d2(11b)fn,i=λn-i,where λ is the forgetting factor of RLS and has values in the range of 0<λ≤1.The order of RLS denoted asP(n) and the corresponding weights update (wi) expressions are given by(12)Pn=λ-1Pn-1-Pn-1xnxTnPn-1λ+xTnPn-1xn(13)wn+1=wn+Pnxnen.Convergence of RLS is much faster as compared with LMS though computational complexity of RLS substantially increases, and this aspect is shown in the subsequent section.


## 3.3. Gradient Decent Approach for Adaptive Spread

In this subsection, the spread factor has also been made adaptive using gradient decent approach for both LMS and RLS algorithms. The relevant gradient of objective function and recursive update of spread factor are given by [21, 22](14a)∂En∂σjn=-wjn∑l=1Lyin-d2G′xl-ckσjQlkn(14b)Qlkn=xl-cknxl-cknT(14c)σjn+1=σjn-η2∂En∂σjn,where the term G′(·) in (14a) is the first derivative of Green’s function G(·) with respect to its argument and η2 is the learning rate for adaptive spread.Depiction of GDA based spread update is shown using dashed-lines of Figure3. Recursive update of spread expressed in (14a), (14b), and (14c) can greatly enhance the approximation capabilities of the neural networks since it is tuning the most significant part of RBF, namely, the activation function defined in (6a). It is also to be noted herein that a further extension of the aforementioned model is also possible in which one can update centers of the Gaussian function as well [21–23]; however subtractive clustering method is used in this work.


## 4. Computational Complexity

Computation complexity of the four algorithms based on matrix and vector dimension denoted asD is summarized in Table 1. It can be seen that algorithm 4 has the highest computational complexity and it increases rapidly with increasing the dimension of the system, for example, number of neurons of RBF. In order to calculate the complexity of overall system, input vectors are needed to be considered alongside the inherited routines of a particular algorithm. For the detailed formation of computational complexity, we suggest that the reader follows mechanics of [29]. It is shown in the next section that RLS algorithm which needs more computational power for the present problem benefits in terms of minimizing the MSE which drops from 152 to just over 1 for the testing data. There are certainly some emerging algorithms [30–34] and conventional techniques of algorithm manipulation [35, 36] with varying degree of complexity which can lead to the potential expansions of this work.Table 1
Comparison of computational complexity.
Algorithms Number of complex multiplications Number of complex additions Algorithm1: (9) and (10) D 2 + D D 2 + D Algorithm2: (12) and (13) 4 D 2 + 2 D 3 D 2 + D Algorithm3: (9), (10), (14a), (14b), (14c) 3 D 2 + D 3 D 2 + 3 D Algorithm4: (12), (13), (14a), (14b), (14c) 6 D 2 + 2 D 5 D 2 + 3 D


## 5. Experimental Results

In this section, four methods have been investigated and analyzed for synthesis of MSA, namely, LMS, gradient decent approach for updating spread fused with LMS (σ-GDA-LMS), RLS, and lastly gradient decent approach for updating spread integrated with RLS (σ-GDA-RLS). Four algorithms mentioned are tested based on the frequency range between 2.2 GHz and 5 GHz with substrate thickness variable between 0.2175 mm and 0.5175 mm and for a scalar value of dielectric constant set as 2.33 which is Rogers RT/duroid, and similar frequency band has been designed using rectangular slot antenna in [37] and therefore adopted herein. Normalized spread used for LMS algorithm is 0.151. The number of epochs used in training is 100, and subtractive clustering approach is utilized as in the previous work of [13]. Training results for LMS is shown in Figure 4. It can be seen that, during the training phase, the estimated values of length (L) and width (W) superimpose the actual output values. The mean square error (MSE) for this algorithm is 156 for the testing batched data. By making the spread adaptive for the LMS algorithm based on gradient decent approach, MSE decreases to 153 which is not a substantive decrease in overall error for this approach. On the other hand, with the similar set of data as in LMS case, RLS is employed with forgetting factor (λ) set at 0.93 and training of ANN is done. On testing it is found that the MSE drops to 129 which is better than both LMS and LMS with adaptive spread. Further, using the adaptive spread technique in RLS algorithm, performance of ANN is enhanced tremendously and the MSE is reduced to 1.396. Training results with ANN based on fourth approach is shown Figure 5 whereas Table 2 represents the comparative MSE analysis for the 4 algorithms discussed. Please note that the difference between the initial spread and final spread for the second and fourth algorithm in Table 2 is disproportionate which is indicative of the overall performance metric of the two algorithms.Table 2
Mean square error (MSE) in testing phase.
Algorithm Initialσ Finalσ MSE LMS 0.151 0.151 1.5626 e + 02 LMS with adaptiveσ 0.151 0.149 1.5322 e + 02 RLS 0.151 0.151 1.2916 e + 02 RLS with adaptiveσ 0.151 1.396 1.3365 e + 00Figure 4
Training results for LMS algorithm.Figure 5
Training results for RLS algorithm with the adaptive spread.In Figure6, we present a 3D depiction for the variables involved in the synthesis design of MSA, namely, resonance frequency (f), substrate thickness (h), and length (L) and width (W), using adaptive spread based RLS algorithm. There is excellent match of the algorithm output and the desired output. The flouring (contour) of Figure 6 indicates the degree of match. Building on the contour of Figure 6 and to better analyze the four algorithms, contour plots for the testing phase of all four algorithms are provided in Figure 7. It can be seen in the contour plots that approximated results of W and L are represented by “red” and “green” lines while the actual or target for both W and L is shown using “blue” lines. In algorithm 4, results almost superimpose desired “blue” lines; the least matching contours are for LMS. The degree of match between approximated and desired output is shown to improve from algorithm 1 to algorithm 4 in respective order. Table 3 represents the desired and approximated width and length for the four algorithms, it can be seen that most significant approximation using RBF is L of the patch which contributes significantly to MSE for LMS; however, it is better approximated using RLS algorithms. From Table 3, it can also be seen that algorithms 1 and 3 are almost similar in approximation of W while, in all four test cases, L is better approximated using algorithm 3 as compared with LMS approach of algorithm 1. Mean square error (MSE) among all four cases is minimal while using adaptive spread based RLS algorithm. The accuracy of our fourth algorithm is 99.89% as compared with 99.09% achieved by authors of [14] which is their highest in the synthesis work of MSA.Table 3
Comparison of the desired and approximated width and length for the four algorithms.
Synthesis parameters f = 4.5 GHz f = 3.3 GHz f = 2.33 GHz f = 2.3 GHz h = 0.238 mm h = 0.387 mm h = 0.338 mm h = 0.588 mm ε r = 2.33 ε r = 2.33 ε r = 2.33 ε r = 2.33 Desired width 25.832808609737622 35.226557195096753 43.054681016229374 50.542451627747518 Algorithm1 25.503126915201797 34.727797095017110 42.546963629337398 53.123805223190246 Algorithm2 25.399012168041605 34.595677176141677 42.368210233612288 52.787288550277708 Algorithm3 25.503126711966839 34.727794498976266 42.546963026145313 53.123800213238567 Algorithm4 25.844082198839743 35.219397833997661 43.044311230214660 50.575068292414450 Desired length 21.589078002479315 29.529980036520211 36.147398398220965 42.477102918108628 Algorithm1 21.180298009632367 28.977178217363399 35.587003693004334 44.507734041334601 Algorithm2 21.096419382083074 28.866795189303346 35.436986349108103 44.225886126622122 Algorithm3 21.142603463604360 29.537218733244114 37.292493792678805 41.295986097468969 Algorithm4 21.513894367659727 29.466986874035420 36.059519035942024 42.453611546508384Figure 6
Testing results for adaptive spread RLS algorithm.Figure 7
Contour plots of the LMS, RLS, and adaptive spread algorithms in MSA synthesis.


## 6. Concluding Remarks

This paper presented the comparative analysis of four different algorithms for synthesis aspect of MSA using RBF. Estimation based on LMS, LMS with adaptive spread, RLS, and RLS with adaptive spread algorithms is utilized for approximating length and width of the MSA based on the input of resonance frequency, permeability constant, and substrate thickness. It is shown that, by augmenting the adaptive spread scheme with LMS and RLS algorithm, there is possibility of significantly improving the performance of RBF Kernel. It is also shown in this work that the most significant metric that accounts for maximum error is length of patch antenna and hence its estimation is of utmost importance which is accounted in RLS algorithm with good measure. The computational power that is desired by machines has been explicitly expressed in terms of complex multiplication and additions. Lastly, some potential extension of this work is discussed.



---

*Source: 9742483-2016-11-14.xml*