Load balancing in D2D networks Using Reinforcement Learning This work proposes a novel mechanism for management, orchestration and flow control in the context of the devicetodevice (D2D) to deal with load balancing using the deep Qlearning (DQN) technique. To do so, we implemented a D2D network simulation environment, using the ParticiptAct dataset to evaluate the load of the cell towers in a region of Italy. The GaussMarkov and GilbertElliott models were used for mobility and packet loss, respectively, where it was considered that the towers had a disconnected coverage area, hence forming a Voronoi space. We used a Gaussian process to predict the load of the towers when they receive the packet, and a DQN to perform the balance of load of the network. This proposal presents better results than the baseline, concerning the metrics used, as well as presenting some perspectives for a future unfolding of this work.

When Learning Joins Edge: RealTime Proportional Computation Offloading via Deep Reinforcement Learning Computation offloading makes sense to the interaction between users and computeintensive applications. Current researches focused on deciding locally or remotely executing an application, but ignored the specific offloading proportion of application. A full offloading cannot make the best use of client and server resources. In this paper, we propose an innovative reinforcement learning (RL) method to solve the proportional computation problem. We consider a common offloading scenario with timevariant bandwidth and heterogeneous devices, and the device generates applications constantly. For each application, the client has to choose locally or remotely executing this application, and determines the proportion to be offloaded. We formalize the problem as a longterm optimization problem, and then propose a RLbased algorithm to solve it. The basic idea is to estimate the benefit of posible decisions, of wihch the decision with the maximum benefit is selected. Instead of adopting the original Deep Q Network (DQN), we propose Advanced DQN (ADQN) by adding Priority Buffer Mechanism and Expert Buffer Mechanism, which improves the utilization of samples and overcomes the cold start problem, respectively. The experimental results show ADQNu0027s high feasibility and efficiency compared with several traditional policies, such as None Offloading Policy, Random Offloading Policy, Link Capacity Optimal Policy, and Computing Capability Optimal Policy. At last, we analyse the effect of expert buffer size and learning rate on ADQNu0027s performance.