## 3 Related Works
### Applications of Reinforcement Learning in Link Quality Estimation  
Estimating radio link quality plays an essential role in both routing and topology control. A comprehensive study of the wireless link behavior in a low-power setting is conducted in [26]. The lossy and time-varying nature of the radio links requires frequent evaluation of the link quality towards each neighbor. Hence, among different classes of machine learning algorithms, Reinforcement Learning emerges as the most suitable mathematical framework for real-time and adaptable resource optimization. [PERSON] et al. [14] conduct a survey that delves into applying bandit theory, a pivotal problem domain within Reinforcement Learning, specifically in the domain of scheduling wireless links. QL-TPC [27] models the transmission power control problem as a Markov Decision Problem (MDP). The authors apply Q-learning, which is a Temporal Difference method, to find the optimal transmission power. The states are defined by the number of retransmissions and _Clear Channel Assessments_ (CCA) attempts, and rewards are determined when receiving acknowledgments. QL-TPC employs \(\varepsilon\)-greedy as the exploration strategy, which introduces sub-optimal exploration. A basic solution for the exploration-exploitation dilemma is the famous \(\varepsilon\)-greedy algorithm, which has a rich history of being used in wireless systems [14]. Nodes choose a constant value for \(\varepsilon\), and in each slot, either select the highest rewarded arm in a greedy approach using probability 1-\(\varepsilon\) or randomly select among other arms to explore using probability. The selection of the arms in the exploration phase is purely random. \(\varepsilon\)-greedy has two major shortcomings that more advanced solutions try to overcome. First, the exploration continues worthlessly even when it is obvious which actions are better rewarded. Second, the agent naively takes random actions in the exploration phase and discards all the retrieved information. [PERSON] et al. [28] model the problem of channel and transmission power selection using adversarial bandits. In these types of bandit problems, the probability distribution of the rewards cannot be attributed to any static distribution since an adversary is opposing the agent. Nodes that incur interference with each other when selecting transmission power are modeled as adversaries. If all agents (nodes) take a strategy with vanishing regret, the joint distribution converges to an equilibrium. Hence, they employ two such strategies, namely the exponential-based weighted average and Follow The Perturbed Leader (FTPL). The main idea of the former is assigning a probability for the selection of actions that is proportional to the accumulated regret. In the latter strategy, agents add a random perturbation to the regrets and then select the action with minimum accumulated regret. [PERSON] et al. [29] employ a multi-layer perceptron (MLP) model to estimate the optimal transmission power in RPL networks with various topologies. This necessitates model training through an offline approach and a labeled dataset. [PERSON] et al. [30] formulate the problem of selecting an efficient cluster head as an evolutionary game. Each node is modeled as a self-interest agent that continuously adapts its strategy to maximize energy efficiency. This approach aims to strike a balance between the remaining energy, hop level, density, and degree of connectivity. This proposal is not integrated with the RPL protocol stack. In the context of Wireless Body Area Networks, the anatomical constraints of body movements, and the orientations measured via motion sensors can be exploited in favor of topology control. [PERSON] [31] takes such an approach and employs learning automata as a learning technique to optimize the transmission power. In the context of cellular IoT and massive MIMO networks, the problem can be approached using methods such as Channel Inversion, which adjusts the transmitted power inversely proportional to the channel gain.