# Comparative Study of a Deterministic Adaptive Beamforming Technique with Neural Network Implementations

Ioannis Mallioras, Zaharias Zaharis, Pavlos Lazaridis, Traianos V. Yioultsis, Nikolaos V. Kantartzis, Ioannis P. Chochliouros

► To cite this version:

Ioannis Mallioras, Zaharias Zaharis, Pavlos Lazaridis, Traianos V. Yioultsis, Nikolaos V. Kantartzis, et al.. Comparative Study of a Deterministic Adaptive Beamforming Technique with Neural Network Implementations. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.98-107, 10.1007/978-3-031-08341-9_9.hal-05335626

HAL Id: hal-05335626

https://inria.hal.science/hal-05335626v1

Submitted on 28 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Comparative Study of a Deterministic Adaptive Beamforming Technique with Neural Network Implementations

Ioannis Mallioras<fnref n="1" />,<fnref n="2" />✉, Zaharias D. Zaharis<fnref n="1" />✉, Pavlos Lazaridis<fnref n="3" />✉, Traianos V. Yioultsis<fnref n="1" />, Nikolaos V. Kantartzis<fnref n="1" />✉, and Ioannis P. Chochliouros<fnref n="4" />✉

**Abstract.** Future wireless networks depend on the development of new mech-
anisms that can increase the efficiency of the network. Antenna array adaptive
beamforming (ABF) is an antenna operation that can be significantly improved
with the use of machine learning. In this paper, a deterministic beamforming tech-
nique is compared with two different types of neural networks (NNs). These are
the non-linear autoregressive network with exogenous inputs (NARX) and the
recurrent NN (RNN) with long short-term memory (LSTM) units. To train the
NNs, we produce a dataset using the minimum variance distortionless algorithm
(MVDR) applied to a realistic antenna array. Using grid search, we find the best
architecture for both NNs. Then, we train the final models and evaluate them by
comparing their accuracy to that of the MVDR algorithm. We demonstrate how
the use of NNs is preferable to that of deterministic algorithms as they appear
to maintain high accuracy while having a much lower response time than that of
deterministic algorithms. The RNN with LSTM units is the most promising out of
the two NN models as it achieves higher accuracy with a slightly shorter training
time.

**Keywords:** Adaptive beamforming · Antenna array · Long short-term memory (LSTM) · Neural network (NN) · Non-linear autoregressive network with exogenous inputs (NARX) · Recurrent neural network (RNN)

# 1 Introduction

The complexity of a modern telecommunications environment is constantly changing and becoming increasingly demanding in terms of response time and data management. There is a huge need for new mechanisms that could elevate the efficiency of the network

<footnote n="1">¹ School of Electrical and Computer Engineering, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
mallioras@auth.gr</footnote>

<footnote n="2">² Maggioli S.P.A., 47822 Santarcangelo di Romagna, Italy</footnote>

<footnote n="3">³ School of Computing and Engineering, University of Huddersfield, Huddersfield H1 3DH, UK</footnote>

<footnote n="4">⁴ Hellenic Telecommunications Organization S.A. Member of the Deutsche Telekom Group of Companies, 15122 Athens, Greece</footnote>
and allow it to handle the demands of current and future network traffic. Given the continuous increase of data load, older and high-complexity algorithms are not able to respond properly and therefore they become less reliable for future use. Already, neural networks (NNs) and other machine learning techniques have proven to be a fast and reliable alternative to many high-complexity algorithms of different scientific fields and so their use in the field of wireless networks is very promising and has already shown a lot of potential [1–6]. By training our models using pre-recorded data from the application of deterministic algorithms, we can create reliable and accurate machine learning alternatives that can replace current mechanisms and improve the overall quality of service.

## 1.1 Antenna Array Adaptive Beamforming

In this regard, we aim to improve the efficiency of smart antennas and specifically one of their most important operations, i.e., adaptive beamforming (ABF). By applying ABF, a smart antenna can dynamically adapt its feeding weights to maintain high gain towards the direction of a desired user while eliminating signals of interference. By steering the main lobe of its radiation pattern towards the direction of a desired incoming signal (i.e., signal of interest or SoI) while placing nulls towards the directions of respective interfering signals (i.e., signals of avoidance or SoAs) the antenna establishes a reliable connection with the desired source and a high signal to interference-plus-noise ratio (SINR). The radiation pattern produced by the antenna array can be altered to fit the desired criteria by tweaking the feeding weight of each array element appropriately. This can be done using an algorithm, also known as a beamformer, to which we give information about the directions of arrival (DoAs) of new incoming signals, and it calculates the appropriate feeding weights. To achieve the same task using NNs, we must train the network to accurately map the angles of arrival (AoAs) of the incoming signals to the desired weight vectors, thus bypassing the high-complexity and time-consuming operations that deterministic algorithms come with. To compare the performance of the NN approach with a deterministic algorithm, we test two types of NNs, namely the non-linear autoregressive network with exogenous inputs (NARX), and a recurrent neural network (RNN) both of which have already been used in the field [3, 6, 7].

# 2 Theoretical Background

## 2.1 Theoretical MVDR Beamformer

For this study, we utilize the minimum variance distortionless algorithm (MVDR) as the deterministic beamforming algorithm for comparison. This beamformer keeps the incoming signal undistorted at the output and minimizes the output signal variance, which practically translates into reducing the output power due to the unwanted signal component thus maximizing SINR. If we consider an M-element linear antenna array and N + 1 incoming signals (where the first is SoI and the rest N are SoAs), the desired weights are calculated using the following expression:

$$ \mathbf{w}_{\text{MV}} = \mathbf{R}_{xx}^{-1} \mathbf{a}(\theta_1, \varphi_1) \quad (1) $$
where $\mathbf{R}_{xx}$ is the input correlation matrix of the beamformer, and $\mathbf{a}(\theta_1, \varphi_1)$ is the steering vector that corresponds to the desired signal, which arrives at the antenna from DoA described in the spherical coordinate system by $(\theta_1, \varphi_1)$. This steering vector is calculated considering an antenna array composed of isotropic point sources, but in order to pursue a more realistic approach we use a modified version as shown below.

## 2.2 Realistic Version of MVDR Beamformer

Modifying the previous equation for a more realistic approach (as seen in [8]) we use the following equation for calculating the desired feeding weights:

$$ \mathbf{w}_{MV} = (\mathbf{E}_{\varphi} \mathbf{E}_{\varphi}^{\mathrm{H}} + \mathbf{R}_{nn})^{-1} \mathbf{e}_{\varphi}(\theta_1, \varphi_1) \quad (2) $$

where $\mathbf{E}_{\varphi}$ is a $M \times (N+1)$ matrix representing the total realistic steering matrix, $\mathbf{R}_{nn}$ is the noise correlation matrix and $\mathbf{e}_{\varphi}(\theta_1, \varphi_1)$ is the realistic steering vector that corresponds to DoA of the desired signal. All matrices and vectors shown in Eq. (2) are calculated given that the incoming AoAs are known. These calculations can be found in [8] and [9], and are beyond the scope of this paper.

## 2.3 NARX as a Beamformer

A NARX is a type of NN that differs from the common feed-forward operation in that it utilizes values of its previous outputs to improve future predictions. As shown in Fig. 1, the outcome of each NARX training iteration, is calculated using the previous attempt's output. This way, this type of NN can process incoming data in a sequential manner and produce an output by having each new input "affecting" the previous prediction.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Two-layer NARX implementation</figcaption></figure>

To use a NARX as a beamformer, the incoming AoAs $x_n(n=0, 1, ..., N)$ enter the NN in series and at each prediction step $n$ the outcome of the network $y_n$ is calculated
using both the current input $x_n$ and the feedback of the output $y_{n-1}$ derived during the previous iteration. The only exception to this process is of course the first input $x_0$, which also represents the SoI, as it enters the first layer of the network without the additional feedback information, since there is no $y_{-1}$.

For this application we also introduce another parameter, called as the *feedback contribution factor* (FCF), and it is used to regulate the compromise between the contribution of the feedback over the current input during the prediction process. The utilization of this parameter can also be seen in Fig. 1 and its value is decided after grid searching within the range [0, 1] as shown in Fig. 2. For this purpose, we use $10^4$ records for training and another $10^3$ records for testing, while considering the root mean square error (RMSE) as a cost function and performing a 3-fold cross validation to further validate our findings. The results in Fig. 2 show that the most promising value for this parameter is 0.4.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Grid search to adjust the feedback contribution factor</figcaption></figure>

## 2.4 RNN as a Beamformer

RNNs belong to another type of NNs, which are mostly used for processing sequential data. This type of NN uses its processing units not only to process the current input, but also to use valuable information from previous inputs. For this study, we build an RNN with long short-term memory (LSTM) processing units.

In this scenario (as shown in Fig. 3), at each time step ($t = 0, 1, ..., N$), the current AoA input $x_t$ is processed by the LSTM units to influence their hidden states, which are consequently passed on to the next time step's units. In this way, each input affects the outcome of the RNN. Once all inputs are processed, and the output $y_N$ has been produced, we pass it through a linear transformation layer, which is also included in the training process, in order to have the correct weight vector size. The idea behind this approach is to have the weight vector initially configured based on AoA of SoI ($x_0$) and then, to let the interferences ($x_n, n = 1,...,N$) shape the final form of the RNN output.
<figure><img src="image_5.png" /><figcaption>Fig. 3. Example of the LSTM-RNN implementation with two hidden layers.</figcaption></figure>

## 3 Dataset Production

For this study, we perform beamforming on a 16-element antenna array, where each element is excited with a complex feeding weight. Therefore, the antenna needs 32 weight numbers to produce the radiation pattern (i.e., 16 real and 16 imaginary parts). To produce the dataset, we implement the modified MVDR algorithm on MATLAB, using the realistic steering vectors derived from the realistic antenna array model simulated in CST [10]. Further restrictions are applied to the produced records so that the NN models are trained based on realistic and demanding circumstances. AoAs of the incoming signals must lie within the angular sector [30°, 150°] and have a minimum distance of $Δθ = 6°$ between each other, while the signal to noise ratio (SNR) of the incoming signals is considered to be equal to 0 dB, thus considering high noise conditions.

Hence, each record consists of $N + 1$ AoAs (with the first one corresponding to SoI and the rest of them to SoAs), and 32 weight numbers created by the MVDR algorithm for the specified AoAs. To make the process of searching the best architecture for each type of NN faster, we assume the simple case of 1 SoI and 2 SoAs. Additionally, we normalize both AoAs and output weights in the range [0,1] to improve the performance of the optimization algorithm. Thus, we produce two datasets, one with $1.1 \times 10^4$ records ($10^4$ for training and $10^3$ for testing) to be used for the purpose of finding each best NN architecture, and one with $1.1 \times 10^6$ records ($10^6$ for training and $10^5$ for testing) for the training and evaluation of the final NN models.

## 4 Training and Evaluation of NNs

### 4.1 NARX

We start by investigating the best architecture for NARX. We test upon a two and a three-hidden layer approach with different values for the layer size. These variations are
shown in Fig. 4 using the following notation: [Size of 1st hidden layer, Size of 2nd hidden layer, ..., Size of last hidden layer]. To train these models, we use the smaller dataset we produced (as explained in Sect. 3) with a 3-fold cross validation to confirm our findings. Each configuration has been trained for 500 epochs with a learning rate of 0.001 and a batch size of 512. Figure 4 demonstrates that the two hidden layer architecture described as [256, 512] (also depicted in Fig. 1) has the most satisfactory results with good train and test error values. It also appears that the increase in the network depth, and thus in its complexity and training time, does not come with a significant decrease in test RMSE values. Therefore, the three-layer approach is disregarded.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Comparison between 2 and 3-hidden layer NARX implementations</figcaption></figure>

Now that we know the best architecture for this type of NN, we proceed to train the final model using the big dataset of 1.1 × 10⁶ records (as explained in Sect. 3), a batch size of 512, and a learning rate of 0.001. Moreover, we use the Pytorch function *ReduceLRonPlateau*, which reduces the learning rate by a factor of 0.8 if the training RMSE is decreasing at a very slow rate. This function increases the training performance by a factor between 2 and 10 [11]. When the learning rate has dropped to a point where it no longer contributes to the training process, or when the test error does not seem to improve over time, we stop the training process. The results are presented in Table 1.

<table><caption>Table 1. NARX training and test results</caption><thead><tr><td>Epochs</td><td>Training RMSE</td><td>Test RMSE</td><td>Final learning rate</td><td>Training time (hours)</td></tr></thead><tbody><tr><td>380</td><td>0.0256</td><td>0.0257</td><td>0.0003</td><td>3.5</td></tr></tbody></table>

We can now test the trained model in terms of beamforming accuracy using 10⁴ triads of AoAs (1 SoI and 2 SoAs) on which the trained model has no prior “experience”. In Table 2, we present a statistical analysis of the performance of the NARX in comparison to that of the MVDR algorithm. It is evident that this model is accurate regarding the placement of the main lobe with similar accuracy to that of the MVDR algorithm, but faces some difficulty in placing nulls at DoAs of SoAs as shown by the higher mean
value of nulls divergence. This also explains the lower SINR mean value achieved by the NARX compared to that of the MVDR algorithm.

<table><caption>**Table 2.** Comparison between MVDR and NARX</caption><thead><tr><th></th><th>MVDR<br>[Mean/Std]</th><th>NARX<br>[Mean/Std]</th></tr></thead><tbody><tr><td>Divergence of main lobe (°)</td><td>0.429/0.320</td><td>0.411/0.317</td></tr><tr><td>Divergence of nulls (°)</td><td>0.000/0.000</td><td>0.856/0.773</td></tr><tr><td>SINR (dB)</td><td>27.225/2.972</td><td>24.170/4.063</td></tr></tbody></table>

## 4.2 LSTM-RNN

In this case, we also need to find the best combination between number and size of the hidden layers. Once again, we proceed with a 3-fold cross validation grid search using the small dataset (as described in Sect. 3) and train each configuration for 500 epochs with a learning rate of 0.001 and a batch size of 512. The results are shown in Fig. 5, where it appears the best combination is that of a two-hidden layer architecture with a layer size of 256.

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Comparison between 2 and 3-hidden layer LSTM-RNN implementations</figcaption></figure>

We proceed to train the final model using the big dataset and the same configurations used in the training of the NARX model, as shown in Sect. 4.1. The training results are provided in Table 3.
<table><caption>**Table 3.** LSTM-RNN training and test results</caption><thead><tr><td>Epochs</td><td>Training RMSE</td><td>Test RMSE</td><td>Final learning rate</td><td>Training time (hours)</td></tr></thead><tbody><tr><td>245</td><td>0.0133</td><td>0.0133</td><td>0.0005</td><td>3.2</td></tr></tbody></table>

Once more, we evaluate the trained model in terms of beamforming accuracy using $10^4$ triads of AoAs on which the trained model has no prior “experience”. The comparison in terms of performance between the MVDR algorithm and the trained model can be seen in Table 4. It appears that the LSTM-RNN is very precise regarding the placement of the main lobe and demonstrates significant accuracy regarding the placement of the nulls. The competence of this model as a beamformer can also be validated by the mean SINR value, which is very similar to that of the MVDR algorithm.

<table><caption>**Table 4.** Comparison between MVDR and LSTM-RNN</caption><thead><tr><td></td><td>MVDR [Mean/Std]</td><td>LSTM-RNN [Mean/Std]</td></tr></thead><tbody><tr><td>Divergence of main lobe (°)</td><td>0.429/0.320</td><td>0.422/0.314</td></tr><tr><td>Divergence of nulls (°)</td><td>0.000/0.000</td><td>0.309/0.353</td></tr><tr><td>SINR (dB)</td><td>27.225/2.972</td><td>26.665/3.113</td></tr></tbody></table>

# 5 Comparison Between the Beamformers

In this section, we compare the performances of the different NN beamformers with that of the MVDR algorithm. In Table 5, we see the summary of the performance of each beamformer, so that we can easily compare them. It is evident that the LSTM-RNN implementation is the most promising model as it outperforms the NARX model in terms of both null placement accuracy and SINR values. We observe that both NN models are much faster than the MVDR algorithm (see mean response time). All measurements have been performed in the Google Colaboratory environment, using an Intel® Xeon® CPU @2.30 GHz with 12 GB of RAM (assigned by the Google Colaboratory environment).

We additionally provide a radiation pattern comparison in Fig. 6 for AoAs equal to 100°, 60°, and 140°. Here, we see that all beamformers have accurately placed the main lobe towards the direction of 100°, but the pattern produced by the LSTM-RNN is much closer to that of the MVDR algorithm, as it appears to almost coincide with it. It is also obvious that the LSTM-RNN model has been slightly more accurate at the null placement at 60°.
<table><caption>**Table 5.** Comparison between MVDR, NARX and LSTM-RNN</caption><thead><tr><th></th><th>MVDR</th><th>NARX</th><th>LSTM-RNN</th></tr></thead><tbody><tr><td>Mean divergence of main lobe (°)</td><td>0.429/0.320</td><td>0.411/0.317</td><td>0.422/0.314</td></tr><tr><td>Mean divergence of nulls (°)</td><td>0.000/0.000</td><td>0.856/0.773</td><td>0.309/0.353</td></tr><tr><td>SINR (dB)</td><td>27.225/2.972</td><td>24.170/4.063</td><td>26.665/3.113</td></tr><tr><td>Training time (hours)</td><td>—</td><td>3.5</td><td>3.2</td></tr><tr><td>Mean response time (sec)</td><td>1.48</td><td>0.0010</td><td>0.0048</td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Radiation patterns produced by MVDR, NARX and LSTM-RNN beamformers for a SoI received at 100° and two SoAs received at respective AoAs equal to 60° and 140°.</figcaption></figure>

# 6 Conclusion

In this study, we have compared two different types of NNs, namely a NARX and a LSTM-RNN, with a conventional deterministic ABF technique, i.e., MVDR, to find which one provides better behavior in the ABF operation. The comparative results shown in Table 5 prove that the most promising model is the LSTM-RNN with two hidden layers, and a size of 256 per hidden layer. Even though the NARX implementation has a slightly faster response time, the LSTM-RNN model achieves higher levels of accuracy with less training time. Given the results, it seems that the implementation of ABF using LSTM-RNN models has great potential and is highly recommended.

**Acknowledgements.** This research was supported by the European Union, partially through the Horizon 2020 Marie Skłodowska-Curie Innovative Training Networks Programme "Mobil-ity and Training for beyond 5G Ecosystems (MOTOR5G)" under grant agreement no. 861219, and partially through the Horizon 2020 Marie Skłodowska-Curie Research and Innovation Staff Exchange Programme "Research Collaboration and Mobility for Beyond 5G Future Wireless Networks (RECOMBINE)" under grant agreement no. 872857.
References

1. Senthilkumar, K.S., Pirapaharan, K., Hoole, P.R.P., Hoole, H.R.H.: Single perceptron model for smart beam forming in array antennas. Int. J. Electr. Comput. Eng. 6(5), 2300–2309 (2016)

2. Ramezanpour, P., Rezaei, M.J., Mosavi, M.R.: Deep-learning-based beamforming for rejecting interferences. IET Sign. Process. 14, 467–473 (2020)

3. Che, H., Li, C., He, X., Huang, T.: A recurrent neural network for adaptive beamforming and array correction. Neural Netw. 80, 110–117 (2016)

4. Zaharis, Z., Gotsis, K., Sahalos, J.N.: Comparative study of neural network training applied to adaptive beamforming of antenna arrays. Prog. Electromagn. Res 126, 269–283 (2012)

5. Zaharis, Z.D., et al.: Implementation of antenna array beamforming by using a novel neu- ral network structure. In: International Conference Telecommunications and Multimedia (TEMU), Heraklion, Greece (2016)

6. Bhadauria, P., Kumar, R., Sharma, S.: Performance dependency of LSTM and NAR beamformers with respect to sensor array properties in V2I scenario. arXiv (2021)

7. Mallioras, I., Zaharis, Z.D., Lazaridis, P.I., Chochliouros, I.P., Mistry, K.K., Loh, T.H.: A novel approach based on recurrent neural networks applied to adaptive beamforming. In: International Conference COMPUMAG, Cancun, Mexico (2021)

8. Zaharis, Z.D., Gravas, I.P., Lazaridis, P.I., Yioultsis, T.V., Antonopoulos, C.S., Xenos, T.D.: An effective modification of conventional beamforming methods suitable for realistic linear antenna arrays. IEEE Trans. Antennas Propag 68(7), 5269–5279 (2020)

9. Godara, L.C.: Smart Antennas, 1st edn. CRC Press, Boca Raton (2004)

10. CST Studio Suite. https://www.3ds.com

11. Dauphin, Y.N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., Bengio, Y.: Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. Adv. Neural Inf. Process. Syst. 2933–2941 (2014)