Decentralized AP selection using MultiArmed Bandits: Opportunistic xcexb5Greedy with Stickiness WiFi densification leads to the existence of multiple overlapping coverage areas, which allows user stations (STAs) to choose between different Access Points (APs). The standard WiFi association method makes the STAs select the AP with the strongest signal, which in many cases leads to underutilization of some APs while overcrowding others. To mitigate this situation, Reinforcement Learning techniques such as MultiArmed Bandits can be used to dynamically learn the optimal mapping between APs and STAs, and so redistribute the STAs among the available APs accordingly. This is an especially challenging problem since the network response observed by a given STA depends on the behavior of the others, and so it is very difficult to predict without a global view of the network. In this paper, we focus on solving this problem in a decentralized way, where STAs independently explore the different APs inside their coverage range, and select the one that better satisfy its needs. To do it, we propose a novel approach called opportunistic varepsilongreedy with Stickiness that halts the exploration when a suitable AP is found, then, the STA remains associated to that same AP while it is satisfied, only resuming the exploration after several unsatisfactory association periods. With this approach, we reduce significantly the network response variability, improving the ability of the STAs to find a solution faster, as well as achieving a more efficient use of the network resources.

LoRaMAB: Toward an Intelligent Resource Allocation Approach for LoRaWAN For a seamless deployment of the Internet of Things (IoT), selfmanaging solutions are needed to overcome the challenges of IoT, including massively dense networks and careful management of constrained resources in terms of calculation, memory, and battery. Leveraging on artificial intelligence will enable IoT devices to operate autonomously by using inherently distributed learning techniques. Fully distributed resource management will free devices from draining their limited energy by constantly communicating with a centralized controller. The present work is devoted to a specific IoT context, that of LoRaWAN, where devices communicate with the access network via ALOHAtype access and spread spectrum technology. Concurrent transmissions on different spreading factors increase the network capacity. However, the bottleneck is inevitable with the expected massive deployment of LoRa devices. To address this issue, we resort to the popular EXP3 (Exponential Weights for Exploration and Exploitation) algorithm to steer autonomously the decision of LoRa devices towards the least solicited spreading factors. Furthermore, the spreading factor selection is cast as a proportional fair optimization problem used as a benchmark for the learningbased algorithm. Extensive simulations were run in a realistic environment taking into account physical phenomena in LoRaWAN such as the capture effect and interspreading factor collision, as well as nonuniform device distribution. In such a realistic setting, we evaluate the performances of the EXP3.S algorithm, an efficient variant of the EXP3 algorithm, and show its relevance against the fair centralized solution and basic heuristics.