DISTRIBUTED BEAMFORMING BY MULTIAGENT ACTIVE INFERENCE
Tatsuya Otoshi † , Masayuki Murata ‡† Graduate School of Economics, Osaka University‡ Graduate School of Information Science and Technology, Osaka University

1

Beamforming• Effective use of a large number of antennas at the base station• Generally, base stations have many antennas and terminals have few antennas• Since the terminal side has only a few antennas, it is difficult to receive the throughput gain ofMIMO.
• Signal propagation can be made more directional by using a large number of antennas at thebase station• Signal control based on channel propagation conditions is necessary• Base station estimates the propagation state based on feedback from the terminal• Controls the signal to be transmitted to amplify the signal received by the terminal based on thepropagation state• Handling of channel state fluctuations is an issue

signalfeedback
2

Beamforming with multiple base stations• Efficient use of radio wave resources by cooperatively performing beamforming

between base stations• Appropriate collaboration methods vary depending on the accuracy of available
information• Joint transmission:• Send the same signal from multiple base stations and amplify the signal• Need accurate channel information• Coordinated beamforming:• Beamforming is performed between base stations to avoid interference• Rough location is more important than precise channel information3

Trade-off between estimation accuracy and control performance• In order to accurately grasp information, communication signal resources aresacrificed.
• Allocate resources to measurement signals to improve estimation accuracy• A certain level of accuracy is necessary for optimizing communication signals, butanything more than that will result in a decline in communication performance.
• People constantly make trade-offs between accuracy and goal achievement underuncertainty.
• Be proactive and obtain information to reduce uncertainty• Make decisions once a certain amount of information has been gathered, rather than aiming for zerouncertainty.
• Solving the trade-off between estimation accuracy and control performance byapplying human active inference4Active inference• Ordinary inference estimates a "good" state given observed values.
• Active inference estimates a "good" state, including changing observed values

through actions.
• Example: Peek under the table to see what is hidden under the table.
• Example: Switching between various beams to estimate channel conditions• Using free energy as a measure of goodness

observed value
outsideworld

inference

observed value

stateoutside
world

state

active inference
action5Free energy principle• A theory that comprehensively describes the functioning of the brain• Describe reasoning and actions as minimization of "free energy"• free energy• 𝐹 = 𝐷!" 𝑄 𝑠 𝑃 𝑠 𝑥− log 𝑃(𝑥)• First term: Posterior distribution of state s P 𝑠 𝑥 and approximate distribution Q 𝑠 Kullback-Leibler

information amount of• Second term : Shannon surprise for• inference• Estimate the posterior distribution• action• Select the action that will yield the observed value x that reduces the Shannon surprise in addition

to the accuracy of the inference

6Inference• observed value x• Feedback of signal strength from the device• condition s• Propagation channel information• State estimation
•#$!
=0#%"

7

Action• action u• beam vector w• Transmission power p• policy 𝑃(𝑢% |𝜋)• Determine behavior by estimating the policy using the behavior distribution as a policy ( control asinference)• The actual action shall be the one with the highest probability.
• Policy estimation
Direct gain
(preference for observed values)

Estimated value of obtaining the observed value o8Preference prior• A probability distribution that expresses the goodness of the observed value itself in

determining behavior.
• Corresponds to reward function and objective function• Predicted distribution of observations• Minimize surprise = Obtain observed values with high probability = Obtain observed values with strong

preferences• objective function• Transmission rate

• Preference distribution reflecting objective function• Boltzmann distribution with negative transmission rate as energy

𝑃/ 𝑜& ∝ exp −𝛽𝜖 = exp 𝛽𝑅 𝑜&9Learning• Observation model: A• A probabilistic model that expresses the relationship between• Used to estimate the state from observed values and predict observed values𝜕𝐹𝜕 ln 𝐴• State transition model : B• A probabilistic model that expresses the time change of state• Used to predict the state when deciding on actions

𝑢%&'𝑃(𝑠! |𝑠!"# , 𝑢!"# , 𝐵)𝑠%&'
𝑃(𝑜|𝑠, 𝐴)

𝑜%&'𝑠%
𝑃(𝑜|𝑠, 𝐴)𝑜%𝜕𝐹𝜕 ln 𝐵
10

Coordination between base stations• Perform cooperative operations by exchanging information between base stations• How to exchange information• Sharing via upper base station• Integrate information at upper base stations• Information transmission to lower base stations• Load may be concentrated on the upper layer• Sharing between adjacent base stations• Exchange information between neighboring base stations• Each base station makes its own decision• Control may conflict between base stations

11

Cooperation between hierarchical FEP agents• Upper layer agent• Observes the state resulting from the inference of lower layer
agentsupper layer• By reducing the dimensionality of the state, it is possible to reduce

the load concentrated on the upper layer.

state• Predicts the desired situation when lower layer cooperation isrealized and feed it back to the lower layer.

stateprediction

lower layer• Lower layer agent• Make control decisions by inferring the state from actual
observation
observed values• Achieving cooperation by using the predictions of upper-layer
agents as the prior distribution for inference

controlprediction
lower layer

observation control• Achieving cooperation through inference and control to minimize

prediction errors
12Simulation environment• base station• UE• 2 BSs• 1 or 3 UEs• 4 antennas/BS• 1 antenna/UE• channel coefficient• beam• multipath fading• 4 pathes• Direction: 4 types• Power: 5 levels• Each path is a complex Gaussian13

Convergence of Free Energy• Information exchange through upper layers speeds up control convergence.
• When upper layer agents are deployed, convergence of expected free energy is achieved inabout 200 steps.
• Without upper layer agents, it takes about 1000 steps to converge.
• Convergence is possible with decentralized control, but convergence is faster withhierarchical control.
Without upper layerWith upper layer

14

Communication performance• Information exchange through upper layers quickly selects the appropriate beam
• Without upper layer agents in place, large SINR drops occur many times before

convergence

Without upper layerWith upper layer
15

Switching Multiple UEs• Beamforming switched between three UEs.
• Red, blue, and green in the graph show beamforming for different UEs.

• High SINR is maintained after beamforming for each UE.

Without upper layerWith upper layer
16Summary and future work
• summary• Proposed a solution based on the free energy principle framework for beamforming withcoordination among multiple base stations• Achieved coordination as the aggregation of information by the upper layer and prediction for thelower layer, and the realization of prediction by the lower layer.
• As a result, appropriate beam selection can be achieved in a short time with little feedback
information exchange.
• Future work
• Simulation considering the movement of UE• Multimodal information processing such as UE location information• Realization of shortcuts for collaboration without upper layers

17