Distributed Beamforming
by Multi-Agent Active Inference
Tatsuya Otoshi

Masayuki Murata

Graduate School of Economics
Osaka University
Osaka, Japant-otoshi@econ.osaka-u.ac.jp

Graduate School of Information Science and Technology
Osaka University
Osaka, Japan
murata@ist.osaka-u.ac.jp

Abstract—Massive MIMO systems have emerged as a promisingtechnology for next-generation wireless communication networks.
Beamforming, a key technique in Massive MIMO, signiﬁcantly
enhances the system’s performance by exploiting the spatial domain. This paper presents a collaborative beamforming approachthat leverages vertical collaboration between base stations toimprove adaptation and mitigate fading variations, while also
applying the principle of free energy to optimize beamforming. The
approach involves sharing estimated channel states and learned
models among base stations, enabling them to collectively optimize
beamforming based on the principle of free energy. Evaluations insingle-user operation scenarios demonstrate that both with andwithout vertical collaboration, the expected free energy, representing beamforming performance, decreases over time. However,vertical collaboration reduces temporary decreases in signal-tointerference-plus-noise ratio (SINR) caused by independent adaptation to fading variations. Furthermore, in a multi-user switching
scenario, the proposed approach ensures stable control by utilizing
learned models and state estimation results, leading to improved
beamforming performance during switching.
Index Terms—Beamforming, Active Inference, Free Energy Principle, 5G NR, CoMP

I. I NTRODUCTION
In recent years, beamforming has gained signiﬁcant importance in the context of the ﬁfth-generation (5G) communication
technology [1]. Beamforming is a technique that enhances communication quality and capacity by controlling the directionalityof the transmitted radio waves using a transmit antenna array.
In particular, the combination of beamforming with Massive
MIMO (Massive Multiple Input Multiple Output) has further
improved the directional capabilities [2]. Massive MIMO utilizes
a large number of transmit and receive antennas, enabling
simultaneous communication with multiple User Equipment(UE) devices.
However, in the border regions of the coverage area, providing sufﬁcient communication quality solely through a single
base station can be challenging. This necessitates the need for
Coordinated Multi-Point (CoMP) operations, where multiple
base stations cooperate to perform beamforming and improve
communication quality for UE devices in the border regions [3],[4].
In CoMP, there is a trade-off between the accuracy of feedback information for beamforming and system performance.

Utilizing accurate feedback information enables optimal beamforming, but there can be delays or errors associated with obtaining such information. When accurate information is available,high throughput can be achieved through Joint Transmission(JT), where multiple base stations transmit the same signal.
On the other hand, when accurate information is not available,
Coordinated Scheduling/Beamforming (CS/CB) is employed,
where multiple base stations transmit different signals while
avoiding interference. In this case, the throughput is lower
compared to JT.
In addition, there is a trade-off between channel state estimation and data transmission in beamforming. To obtain accurate
information, it is necessary to acquire a substantial amountof feedback information. Typically, dedicated pilot signals aretransmitted for feedback purposes. The transmission of pilot
signals for channel state estimation limits the resources available
for data transmission. Even if the ACK (Acknowledgment) fordata transmission is used as feedback, the feedback information
obtained varies depending on the beam conﬁguration. Therefore,
it is necessary to switch the beams for measurement purposes.
This implies that there are occasions where the opportunity tosacriﬁce beams optimized for data transmission arises in orderto obtain feedback.
To address these challenges, we use the Free Energy Principle(FEP) [5], drawing inspiration from the workings of the humanbrain. FEP aims to estimate and utilize the information necessary
for the system to make optimal decisions. In FEP, the goal is
to minimize the free energy, which is a combination of the
exploration and exploitation value. By minimizing free energy,
FEP allows for the selection of actions that strike a balance
between exploration and exploitation. This means that FEP
enables the system to choose actions that are optimal in terms ofbalancing the trade-off between exploration and exploitation. In
the context of beamforming or similar scenarios where a tradeoff exists between observation and control, FEP can be leveraged
to make optimal decisions. By using FEP, the cooperation among
multiple base stations in beamforming can be achieved, leading
to an improvement in communication quality.
In this paper, we propose a cooperative beamforming technique using the Free Energy Principle (FEP). To achieve this,we deploy FEP agents at each base station and place a higher-

level FEP agent to coordinate their actions. The lower-level
agents perform beamforming using information from the higherlevel agent and feedback information from the User Equipment(UE). The higher-level agent observes the state of the lower-level
agents and provides feedback in the form of predicted states. By
employing this hierarchical structure, coordinated beamforming
is realized, ensuring coherence across the system.
The evaluation compares the performance of hierarchicalcoordination with non-coordination scenarios, elucidating theeffectiveness of hierarchical cooperation.
This paper is organized as follows. Section II discusses
related work. Section III describes the system model. Section IV
presents the proposed method. Section V evaluates the proposed
method. Finally, Section VI concludes the paper.

noise power. Additionally, for each user terminal, the decisionof whether to block it or not is made. Blocked terminals cannot
transmit radio waves, reducing interference to other terminals.
The beamforming and blocked user terminals are computedby solving an optimization problem to maximize the sum oftransmission rates for all terminals.
In the context of active inference based on the principle of freeenergy, the policy π = (a1 , · · · , aT ) representing a sequenceof selected actions is determined to minimize the following
expected free energy:

II. R ELATED W ORK

Here, o1:T represents the observations, s1:T represents thestates, Q represents the approximate posterior distribution, andP̃ represents the target distribution.
Given π, the expected free energy at each time step τ can be
independently calculated as follows:

In previous studies targeting hierarchical beamforming withmacrocells and small cells, most of them assumed sparse placement of small cells without overlapping with each other [6]–[8]. In this case, user terminals connect to the small cell at theirlocation, and if they do not belong to any small cell, they connect
to the macrocell, ensuring a unique correspondence between
cells and terminals. However, in practice, considerations must be
given to terminal allocation between cells, such as the overlap
between small cells or allocating terminals near the peripheryof small cells to the macrocell.
In the work by [9], a method for performing independent
beamforming in situations with overlapping cells and interference between them is investigated. However, this approach does
not consider the hierarchical structure of cells.
Each cell determines beamforming and transmission powerusing local information within the cell, such as the transmission
rate to the terminal, channel gains, and interference powerincluding noise. Beamforming and transmission power selection
are performed using Deep Q-network (DQN) from a discrete setof multiple candidates.
DQN learns the relationship between observed information
and reward (power efﬁciency as the transmission rate per unittransmission power) using a neural network. The observed
information uses only local information, while the reward incorporates global information by considering the overall power
efﬁciency of all cells. The learning of the Q-network involves
accumulating observed information and rewards, followed byofﬂine iterative retraining.
The evaluation demonstrates that the overall power efﬁciency
improves compared to randomly selected beamforming andtransmission power choices or when using a greedy approach.
In the study by [7], beamforming for small cells within the
coverage of macrocells is investigated. In this case, the interference between cells is only considered between the macrocell andsmall cells, without considering interference among other small
cells. The macrocell has the ability to determine the transmitted
radio waves with higher priority, while small cells perform
beamforming in the presence of interference from the macrocell.
In the small cells, beamforming decisions are made based onchannel coefﬁcients, interference power from the macrocell, andG(o1:T , s1:T , π) =EQ [log Q(s1:T , π)(1)

− log P̃ (o1:T , s1:T , π)]Gτ (π)=

EQ(oτ ,sτ |π) [log Q(sτ |π) − log P̃ (oτ , sτ |π)] (2)≥−EQ(oτ |π) [DKL (Q(sτ |oτ , π)||Q(sτ |π))](3)−EQ(oτ |π) [log P̃ (oτ )]The ﬁrst term in the above equation provides the information
gain of posterior distribution update by obtaining the newobservation oτ . The second term represents the expected utility
based on the observation oτ . The utility needs to be speciﬁed
in advance by the prior distribution P̃ (oτ ). Both the ﬁrst andsecond terms are negative, and as the information gain and utility
increase, the expected free energy decreases.
Hierarchical active inference models primarily focus on hierarchical organization within a single individual, where theupper layers provide goals to lower layers to determine speciﬁc
actions [10]–[13].
In the context of small cell-macrocell systems, the macrocell
not only modiﬁes the goals of the small cells but also controls its
own behavior. Additionally, the macrocell needs to interact withmultiple small cells. While the coordination of multiple agentshas been addressed in [13], it does not involve hierarchicalcoordination but rather coordination between equal agents.
In [12], hierarchical active inference is applied to robot
motion control. The hierarchical structure in this case refers to
the temporal hierarchy within a single robot, where the upper
layers determine long-term trajectories while the lower layers
decide short-term motor controls.
The upper layers aim to reach suitable positions based on
preference distributions, while the lower layers aim to minimize
deviations from the assumptions made by the upper layers using
the upper layers’ estimated distributions as prior beliefs.
In the upper layers, the path with the minimum expected freeenergy is determined by weighing the expected free energy ateach point along the trajectory. On the other hand, in the lower

layers, individual movements are determined through samplingbased on the magnitude of the expected free energy.
Using an actual robot and only RGB camera data, the ability
to correctly move to the target position has been conﬁrmed.
However, this work does not consider comparisons with othermethods or environmental changes.
In [13], a hierarchical active inference approach is applied
to a cooperative cooking game called ”overcooked,” where
agents (cooks) cooperate with each other. To facilitate agent
cooperation, each agent estimates the goals of other agents andshares those goals.
In this approach, hierarchical organization occurs within eachagent, with an upper layer that sets goals and a lower layer that
determines speciﬁc actions. The estimated goals of others andtop-down goals from the upper layer are reﬂected as the Kalman
gain in the Kalman ﬁlter, and the lower layer makes decisions
for integrated actions.
It has been demonstrated that this approach achieves comparable performance to Bayesian Delegation, which combines
Bayesian estimation and Q-learning, in a shorter computation
time.
III. S YSTEM M ODELThe system consists of M macrocells and S small cells that
communicate with D User Equipment (UE). We consider ascenario where downlink communication involves interference
among the beams transmitted from the base stations and interfering with each UE.
We deﬁne the set of base stations for macrocells as B (m) , theset of base stations for small cells as B (s) , and the set of UEs
as U :

B

(s)

U

(m)

|i = 1, · · · , M }p

Pb (t)H u,b (t)wb (t) ◦ xb (t)(7)

where ◦ denotes element-wise multiplication.
1) Electromagnetic Interference: UE u receives signals that
include noise and signals transmitted from other base stations,causing interference. The received signal at UE u is given by:y u (t) = y u,b (t) +Xy b′ (t) + σu (t)(8)

b′ ∈B\{b}The Signal-to-Interference-plus-Noise Ratio (SINR) is deﬁned
as follows:γu (t) =Pb (t)∥H u,b (t)wb (t)∥2
Iu (t) + σu2 (t)X

Iu (t) =(9)2

y b′ (t)(10)

b′ ∈B\{b}where I(t) represents the strength of interference signals, andσ 2 (t) represents the power of noise.
C. Objective Function

A. Components

B (m)y u,b (t) =

=

{bi

=

(s)
{bi |i = 1, · · · , S}

(5)=

{ui |i = 1, · · · , D}

(6)

(4)

The set of all base stations is denoted as B = B (m) ∪ B (s) .
B. Channel Coefﬁcients and Beamforming
Each base station b transmits signals using Mb antennas, andthe signals are transformed by the spatial characteristics andreach the Mu antennas of the receiving UE u. The spatial
characteristics are represented by the matrix H u,b (t), where
the element hu,bij (t) represents the channel coefﬁcient whentransmitting from antenna i to antenna j.
At each time, each UE u is connected to one of the basestations b, and this correspondence is represented by A(t). The
element au,b (t) of A(t) is 1 if u is connected to b, and 0
otherwise.
Each base station adjusts the phase and amplitude of thetransmitted signals for each antenna and performs beamforming.
The phase is represented by the beamforming vector wb (t), andthe amplitude is represented by Pb (t). The relationship between
the transmitted signal xb (t) and the received signal yu,b (t) is
given by:The data transmission rate to UE u is estimated using SINR
as follows:
Cu (t) = log(1 + γu (t))(11)

In the paper by [9], the reward is deﬁned as the totaltransmission rate of all cells per unit power consumed by allcells, aiming to maximize the overall power efﬁciency. In thiscase, the power efﬁciency at time t is given by:X
EE(t) =
EEu (t)(12)u

EEu (t) =C (t)Pub Pb (t)(13)

However, when using this objective function, fairness among
UEs is not considered, which may lead to unfairness amongUEs, such as prioritizing nearby UEs that can achieve higher
transmission rates while making communication difﬁcult for
distant UEs. In the paper by [14], for the case of beamforming
in a single base station, they propose using objective functions
that consider fairness, such as the logarithmic sum of power
efﬁciency for each UE or the maximization of the minimum
power efﬁciency. Similar objective functions can be appliedin the case of multiple base stations to achieve beamforming
considering fairness.
IV. D ISTRIBUTED B EAMFORMING BY FEP
A. Active Inference by Multiple Agents
To minimize the expected free energy, it is necessary to
calculate the expected free energy for all possible actions basedon the observations and states. In the case of beamformingwith multiple base stations, the number of observations, states,
and actions increases with the number of base stations. Inparticular, for states and actions, considering their combinations
is necessary for distribution calculations, making it impracticalto minimize the expected free energy of the entire system.
Therefore, it is desirable for the active inference agents deployedat each base station to decide local actions based on local
information.
In this regard, each base station b ∈ B determines a local ac(b)(b)
tion sequence π (b) = (a1 , · · · , aT ) to minimize the following(b)
local free energy under the local observation information oτ(b)and the local state sτ :Gτ (π)(b)=(b)
EQ(o(b) ,s(b) |π(b) ) [log Q(s(b)
)
τ |π
τ

τ

(14)

(b) (b)− log P̃ (o(b))]τ , sτ |π

In this case, collaboration with other base stations is achieved(b)through the exchange of local observation information oτ orprediction and prediction errors. In FEP, collaboration at thesame hierarchical level is achieved by other agents observing
the results of an agent’s actions through observations [13]. On
the other hand, collaboration between upper and lower levels
is achieved through the exchange of predictions and prediction
errors, where the upper level predicts the state of the lower levelfrom a higher level of abstraction, and the lower level conveys
the prediction error to the upper level [12].
In the case of beamforming, collaboration at the same hierarchical level involves collaboration where the settings of eachbeamforming are observed as equal partners, while collaboration between upper and lower levels involves the upper level
predicting the desired states of each small cell as a result ofmediating the interference of each lower level, and the lower
level providing feedback on prediction errors to the upper
level. In this case, collaboration between the upper and lower
levels requires more overhead for exchanging information since
it is necessary to transmit information about the distribution
rather than the determined value of beamforming results in thecase of collaboration at the same hierarchical level. Therefore,
collaboration between the upper and lower levels is expected tobe performed on a longer timescale than collaboration within
the same hierarchical level.
B. Active Inference in Vertical Collaboration
We provide a detailed description of the operation of active
inference agents in the vertical collaboration between basestations. Speciﬁcally, we deﬁne observations, states, actions, andpreference distributions in the framework of FEP.
1) Lower-level Agents: Lower-level agents are deployed ineach small cell and determine the shape of the beam based on the
feedback from the user equipment (UE). Although lower-level
agents are independent of each other, they achieve cooperation
among agents by receiving the predicted distribution from theupper level and implementing control along the preference
distribution.
a) Observations: We assume that base station b can observe the transmission rate to each UE based on the feedback
from the UE, denoted as C b (t). Therefore, it corresponds to
ob (t) = C b (t).

As feedback, the receiving side SINR is measured through
CSI or SS, and the transmission rate is estimated from the SINR.
b) States: Based on the observed transmission rate C b (t)and the information of its own beamforming, base station b
estimates the channel coefﬁcients H u,b (t). Therefore, the system
state is represented as sb (t) = (H u,b (t), C b (t)).
The prior distribution of states is given as the predicteddistribution of lower-level states by the upper level, and thelower level achieves cooperation by controlling to align with theassumptions of the upper level. Therefore, the prior distributionof states in the lower level is given as follows:P̃C (sb (t)) = P (sb (t)|obh (t))(15)

where bh represents the base station of the macrocell.
c) Actions: The base station speciﬁes the shape of thebeam by the beam vector wb (t) and determines the transmit
power Pb (t). Therefore, ab (t) = (wb (t), Pb (t)) corresponds tothe actions.
d) Preference Distribution: The preference distribution
sets the desired states for control as the prior distribution withrespect to the observations. When a speciﬁc objective function
Rb (oτ ) is set, such as power efﬁciency or maximization oftransmission rate, the preference distribution can reﬂect theobjective function R(oτ ) by using the Boltzmann distribution
as the preference, where utility is measured in terms of thelogarithm of the preference distribution. It can be represented
as follows:P̃R (oτ ) ∝ exp(βR(oτ ))(16)where β is a parameter that determines how much the preference
biases depending on the magnitude of the objective function.
2) Upper-level Agents: Upper-level agents are deployedin the macrocell and determine the Coordinated Multi-Point(CoMP) conﬁguration (i.e., the group of lower-level base stations performing joint transmission) based on the observed states
estimated by the lower-level agents. The predicted distributionof upper-level states is used as the prior distribution for the
lower-level states.
a) Observations: The estimated results of the states
P (sbl (t)|obl (t)) in the lower-level base stations bl are used as
observations. Therefore, ob (t) = P (sbl (t)|obl (t)) corresponds
to the observations.
b) States: In the upper level, control is performed based onlthe channel coefﬁcients H u,b (t) for each lower-level base stalltion. Therefore, sb (t) = (H u,b1 (t), H u,b2 (t), · · · ) corresponds
to the states.
c) Actions: In the upper level, the group of lower-levelbase stations performing joint transmission (JT) is determined.
In other words, a subset of small cells is determined as G ⊆B (s) , and ab (t) = G is deﬁned.
d) Preference Distribution: Similar to the lower level,
the preference distribution in the upper level is a Boltzmann
distribution based on power efﬁciency or transmission rate, butit uses aggregated power efﬁciency and transmission rate for theentire system instead of individual values for each base station.

10−3

gain

C. Beamforming for Multiple Terminals
In the case of multiple terminals, each UE is assigned a timeslot, and only one UE is served simultaneously. Since eachUE has physically different antennas, the channel coefﬁcients
H u,b between the UE and the base station are independent
for each terminal. Therefore, for each UE u, an agent state(u)sb is maintained, and the state is switched depending on the
UE to which the transmission is directed, thereby achieving
beamforming for multiple terminals.
Here, we assume that the assignment of time slots to UEs isdetermined in advance as an external control. Also, regarding
the feedback from UEs to base stations, it is assumed to be theobservation of the estimated time slots for each UE.
V. E VALUATION
To conﬁrm the effectiveness of beamforming coordination byvertical coordination using FEP, simulation will be used to verify
the operation.

10−5

0

r
hij (t)=gi,j (t, l)=ai,j (θl )=βi,j X
gi,j (t, l)a†i,j (θ)
L
l=1pρgi,j (t − 1, l) + 1 − ρ2 ei,j (t)1√ (1, exp(πil · 1 cos θl ),N· · · , exp(πil · (N − 1) cos θl ))
L

(17)
(18)

Here, ei,j (t) represents complex Gaussian noise, and ρ is aparameter that determines the temporal continuity of Rayleighfading. θl denotes the transmission angle of path l, which follows
a uniform distribution (θ¯l − ϑ/2, θ¯l + ϑ/2). Figure 1 illustrates
an example of the time variation of the channel gain ||H(t)||2
under fading conditions.

200

400

time

600

800

1000

Fig. 1. Example of the time variation of the channel gain under fading conditions2) Base Station Operation: Each base station observes thereceived SINR from the UE as feedback. Upon receiving the
feedback, the base station updates its internal models and statesand determines the transmit power and beamforming vector forthe next transmission.
The beamforming vector is selected from a pre-deﬁned set of
N codebooks [1]. The i-th beamforming vector wn is given by:
wn

A. Setting
To verify the effectiveness of coordinated beamforming
through vertical collaboration using FEP, we conducted simulations to conﬁrm its operation.
1) System Conﬁguration: We consider a system with 1 UE
equipped with a single antenna and 2 base stations equippedwith N = 4 antennas each. The goal is to transmit signals fromthe two base stations to the UE using beamforming techniques.
Initially, we investigate the performance in a static environment
where the channel coefﬁcients remain ﬁxed without temporalvariations. The noise intensity is set to -114 dBm.
Following the approach in [9], we assume that the channel
coefﬁcients undergo time variations modeled as L-path Rayleighfading. The channel coefﬁcient hij (t) between the i-th basestation and the j-th antenna of the UE at time t can be expressed
as:

10−4

1
2πi(n − 1)0= ( √ exp(),
NN
2πi(n − 1)(N − 1)
1
· · · , √ exp(
))
N
N

(19)

We use solver pymdp [15] for inference, action, and learning,in which, the SINR and channel coefﬁcients are discritized into
10 bins and 50 bins.
3) Internal Model Initialization: The internal models
P (oτ |sτ , π) and P (sτ |sτ −1 , π) can also be learned as FEP for
expected free energy minimization. For this simulation, we set
the initial models with random values as the initial models for
FEP.
B. Results
In this section, we present the results of the evaluation forboth single-user operation and multi-user switching scenarios.
C. Single-User Operation
Figures 2 and 3 show the results of the expected free energy
and SINR over time for single-user operation without and withvertical collaboration, respectively.
In Fig. 2, the results without vertical collaboration show the
temporal variations of expected free energy and SINR wheneach base station independently performs beamforming. Fig. 3
illustrates the results with vertical collaboration, where the basestations share the estimated states through higher layers.
From both ﬁgures, it can be observed that the expected
free energy decreases over time, indicating the improvement of
the beamforming performance. However, due to the stochastic
variations of the channel coefﬁcients, the expected free energy
shows small ﬂuctuations and does not converge to zero, unlike
in static environments.
The SINR, on the other hand, reaches a high level after
a certain number of iterations in both cases, indicating that

30

25

25

20
cell

15

0
1

10
5
0

expected free energy

expected free energy

30

20
cell

15

0
1

10
5

0

200

400

time

600

800

1000

0

0

200

time

600

800

1000

(a) Expected Free Energy

125

125

100

100

75

75

50

50

SINR

SINR

(a) Expected Free Energy

400

25

25

0

0

−25

−25

−50

−50
0

200

400

time

600

800

1000

0

200

400

time

600

800

1000

(b) SINR(b) SINR

Fig. 2. Temporal variations of expected free energy and SINR at the UE (without
vertical collaboration)

Fig. 3. Temporal variations of expected free energy and SINR at the UE (withvertical collaboration)

appropriate beams are selected. However, without vertical collaboration, temporary decreases in SINR are observed even after
it initially becomes high. These temporary decreases are causedby the response of each base station to ﬁne variations in fading,which affects the changes in beams of other base stations.
In particular, without vertical collaboration, the independent
adaptation of each base station to ﬁne variations in fading leads
to oscillations and further changes in the situation. In contrast,
the sharing of state information through vertical collaboration
allows the upper layers to predict the overall situation andsuppress oscillations in control responses to such ﬁne variations.

The SINR, in both cases, reaches a high level after a certain
number of iterations and is maintained thereafter. However,
without vertical collaboration, temporary decreases in SINR are
observed even after it initially becomes high. These temporary
decreases are caused by the response of each base station to ﬁnevariations in fading, which affects the changes in beams of otherbase stations. Nevertheless, the use of vertical collaboration
enables stable control by predicting the overall situation basedon higher-level information and transmitting it to lower-level
layers.
Although each UE requires iterations to improve SINR, once
the SINR improves, it remains at a high level. Even when a
UE is switched during the process, the learned models and state
estimation results are maintained, allowing the setting of beams
tailored to the situation of each UE. In practical operations,since the models and states are continuously utilized, the timerequired for the initial iterations is not a signiﬁcant issue.

D. Multi-User Switching
Next, we consider the scenario of switching the transmission
target among multiple UEs according to a given schedule. The
schedule switches the UE every 300 time units.
Figures 4 and 5 show the results of the expected free energy
and SINR over time for multi-user switching without and withvertical collaboration, respectively. Each UE is color-coded in
the ﬁgures.
Fig. 4 shows the results without vertical collaboration, where
each base station independently performs beamforming for theswitching UEs according to the given schedule. Fig. 5 illustrates
the results with vertical collaboration, where the base stations
share the estimated states through higher layers.
From both ﬁgures, it can be observed that the expected freeenergy decreases for each UE during the switching process,indicating the improvement in beamforming performance.

VI. C ONCLUSION
In this paper, we proposed a collaborative beamforming
approach, utilizing vertical collaboration between base stations
to enhance the overall performance. The proposed approachleverages the sharing of estimated channel states and learned
models among base stations to improve beamforming adaptationand mitigate the effects of fading variations.
Through evaluations in single-user operation scenarios, weobserved that the expected free energy, which represents the
beamforming performance, decreased over time in both caseswith and without vertical collaboration. However, without verti-

40

35

35

30

30

25
cell

20
15

0
1

10
5
0

expected free energy

expected free energy

40

25
cell

20
15
10
5

0

500

1000

1500
time

2000

2500

3000

0

0

500

(a) Expected Free Energy
125

125

100

100

75

75

50

50

25

0

25

25
0

500

1000

1500
time

2000

1500
time

2000

2500

3000

25

0
50

1000

(a) Expected Free Energy

SINR

SINR

0
1

2500

3000

50
0

500

1000

1500
time

2000

2500

3000

(b) SINR(b) SINR

Fig. 4. Temporal variations of expected free energy and SINR at the UE (without
vertical collaboration, with 3 UE switching)Fig. 5. Temporal variations of expected free energy and SINR at the UE (withvertical collaboration, with 3 UE switching)cal collaboration, the SINR showed temporary decreases due tothe independent adaptation of each base station to ﬁne variations
in fading. In contrast, with vertical collaboration, the sharingof state information allowed for the prediction of the overallsituation and suppression of oscillations in control responses,
resulting in improved stability.
Furthermore, we investigated the multi-user switching scenario, where the transmission target switched among multiple
users according to a predeﬁned schedule.
Future work can explore further enhancements to the collaborative beamforming approach. One potential direction is
to investigate horizontal collaboration between beamforming
agents, enabling coordinated actions and information exchange
to achieve better overall performance. Additionally, incorporating predictions of environmental time variations could improve
the adaptability and robustness of the beamforming system.

[4] T. K. Tandra, F. Tajrian, A. Hossain, M. T. Kawser, M. R. Akram, andA. B. Shams, “Joint transmission coordinated multipoint on mobile users in
5G heterogeneous network,” in 2022 IEEE 2nd Conference on Information
Technology and Data Science (CITDS). IEEE, 2022, pp. 273–278.
[5] K. Friston, “The free-energy principle: a uniﬁed brain theory?” Nature
reviews neuroscience, vol. 11, no. 2, pp. 127–138, 2010.
[6] G. Bartoli, R. Fantacci, K. B. Letaief, D. Marabissi, N. Privitera, M. Pucci,
and J. Zhang, “Beamforming for small cell deployment in lte-advanced andbeyond,” IEEE Wireless Communications, vol. 21, no. 2, pp. 50–56, 2014.
[7] M.-L. Ku, L.-C. Wang, and Y.-L. Liu, “Joint antenna beamforming, multiuser scheduling, and power allocation for hierarchical cellular systems,”
IEEE Journal on Selected Areas in Communications, vol. 33, no. 5, pp.
896–909, 2014.
[8] D. Castanheira, P. Lopes, A. Silva, and A. Gameiro, “Hybrid beamforming
designs for massive MIMO millimeter-wave heterogeneous systems,” IEEE
Access, vol. 5, pp. 21 806–21 817, 2017.
[9] K. Yu, G. Wu, S. Li, and G. Y. Li, “Local observations-based energyefﬁcient multi-cell beamforming via multi-agent reinforcement learning,”
Journal of Communications and Information Networks, vol. 7, no. 2, pp.
170–180, 2022.
[10] G. Pezzulo, F. Rigoli, and K. J. Friston, “Hierarchical active inference: atheory of motivated control,” Trends in cognitive sciences, vol. 22, no. 4,
pp. 294–306, 2018.
[11] J. Bauermeister and P. Lanillos, “The role of valence and meta-awarenessin mirror self-recognition using hierarchical active inference,” arXiv
preprint arXiv:2208.13213, 2022.
[12] O. Çatal, T. Verbelen, T. Van de Maele, B. Dhoedt, and A. Safron, “Robotnavigation as hierarchical active inference,” Neural Networks, vol. 142,
pp. 192–204, 2021.
[13] J. Pöppel, S. Kahl, and S. Kopp, “Resonating minds―emergent collaboration through hierarchical active inference,” Cognitive Computation, vol. 14,
no. 2, pp. 581–601, 2022.
[14] H. Al-Obiedollah, K. Cumanan, J. Thiyagalingam, A. G. Burr, Z. Ding, andO. A. Dobre, “Energy efﬁciency fairness beamforming designs for MISONOMA systems,” in 2019 IEEE Wireless Communications and Networking
Conference (WCNC). IEEE, 2019, pp. 1–6.
[15] C. Heins, B. Millidge, D. Demekas, B. Klein, K. Friston, I. Couzin, and
A. Tschantz, “pymdp: A python library for active inference in discrete
state spaces,” arXiv preprint arXiv:2201.03904, 2022.

ACKNOWLEDGMENT
This work was supported by JSPS KAKENHI Grant Number
JP20K14735.
R EFERENCES[1] L. Rao, M. Pant, L. Malviya, A. Parmar, and S. V. Charhate, “5G
beamforming techniques for the coverage of intended directions in modern wireless communication: in-depth review,” International Journal of
Microwave and Wireless Technologies, vol. 13, no. 10, pp. 1039–1062,
2021.
[2] F. A. P. de Figueiredo, “An overview of massive MIMO for 5G and 6G,”
IEEE Latin America Transactions, vol. 20, no. 6, pp. 931–940, 2022.
[3] M. S. J. Solaija, H. Salman, A. B. Kihero, M. İ. Sağlam, and H. Arslan,“Generalized coordinated multipoint framework for 5G and beyond,” IEEE
Access, vol. 9, pp. 72 499–72 515, 2021.