## 4 Proposed Solution (Intelligent Agent Project)
From Tables 2 and 3, it can be seen that power control was selected in more states, compared to beamforming, which demonstrates the independence of the algorithm in selecting the optimum policy, in contrast to other procedures [8, 9, 10, 17] in which only the power is updated once beamforming convergence is reached. Figure 5 illustrate the evolution of the SINR until the target value is reached (second simulation). \begin{table}
\begin{tabular}{c c c c c c c c} \hline  & \multicolumn{3}{c}{**Policy 10**} & \multicolumn{3}{c}{**Policy 50**} & \multicolumn{3}{c}{**Policy 250**} \\ \hline
**INDEX** & **SINR(dB)** & **BF** & **PC** & **BF** & **PC** & **BF** & **PC** \\ \hline
1 & \(-\)0.8 & 0.5 & 0.5 & 0 & 1 & 0 & 1 \\
2 & \(-\)0.6 & 0.5 & 0.5 & 0 & 1 & 0 & 1 \\
3 & \(-\)0.4 & 0.5 & 0.5 & 1 & 0 & 0 & 1 \\
4 & \(-\)0.2 & 0 & 1 & 1 & 0 & 0 & 1 \\
5 & 0 & 0.5 & 0.5 & 0.5 & 0.5 & 0 & 1 \\
6 & 0.2 & 0 & 1 & 0 & 1 & 0 & 1 \\
7 & 0.4 & 0 & 1 & 0 & 1 & 0 & 1 \\
8 & 0.6 & 0 & 1 & 0 & 1 & 0 & 1 \\
9 & 0.8 & 1 & 0 & 0 & 1 & 0 & 1 \\
10 & 1 & 0 & 1 & 0 & 1 & 0 & 1 \\
11 & 1.2 & 0.5 & 0.5 & 0 & 1 & 0 & 1 \\
12 & 1.4 & 0 & 1 & 0 & 1 & 1 & 0 \\
13 & 1.6 & 0 & 1 & 0 & 1 & 0 & 1 \\
14 & 1.8 & 0 & 1 & 0 & 1 & 0 & 1 \\ \hline
15 & 2 & **Destination** & **Destination** & **Destination** & **Destination** \\ \hline
16 & 3 & 0 & 1 & 1 & 0 & 1 & 0 \\
17 & 4 & 0 & 1 & 1 & 0 & 1 & 0 \\
18 & 5 & 1 & 0 & 1 & 0 & 1 & 0 \\ \hline \end{tabular}
\end{table} 
Table 3: Policy Improvement of the agent. Figure 4: The switching sequence among the two techniques. Figure 5: **(a)** System response. Agent started with \(\text{SINR}=-0.8\) dB; **(b)** System response. Agent started with \(\text{SINR}=0\) dB; **(c)** System response. Agent started with \(\text{SINR}=5\) dB. Figure 6 presents the switching sequence for an initial SINR of 5, where the ordinate axis shows the actions executed (beamforming or power control) and the curve indicates the order of execution in each processing cycle. Index 1 indicates the execution of beamforming and Index 2 indicates the execution of power control. It can be seen that even with increase of the variance of the noise from 0.1 to 0.3, the proposed algorithm was effective in selecting the optimum policy, requiring only one new training, given that the environment was modified. This is an important point, and shows the robustness of the algorithm when faced with new noise conditions. ## 6 Conclusions

The use of beamforming and power control individually in sensor arrays has its benefits, but it can be seen that by employing them jointly there is an increase in system performance. Several studies have used beamforming and power control jointly, but this paper presents a new control method employing a combination of beamforming and power control. The algorithm presented uses the technique of reinforcement learning to obtain the optimum policy for selection between beamforming and power control in sensor arrays. It can also be seen that the proposed technique reduces the computational cost, as the techniques are selected independently. For example (Table 2), using the optimum policy obtained after 250 episodes, it was found that in many cases it was not necessary to execute the LMS algorithm. This resulted in lower computational cost and reduced complexity of the proposed method. From the simulations performed, it could be concluded that reinforcement learning offers an effective way of implementing a policy of switching between beamforming and power control in sensor arrays, benefiting from the advantages of both techniques. Figure 6: The switching sequence among the two techniques. ## Author Contributions

All the authors have contributed in various degrees to ensure the quality of this work. [PERSON], [PERSON] and [PERSON] conceived the idea and experiments; [PERSON] designed and performed the experiments; [PERSON], [PERSON] and [PERSON] analyzed the data and wrote the paper. ## Conflicts of Interest

The authors declare no conflict of interest.