DOI: 10.5281/zenodo.6394799
title: SMART - Self-adaptive Machine Learning Approach for Real-time Tuning of IEEE 802.11 PHY and MAC layers

description: <strong>Introduction</strong> Worldwide the demand for wireless access networks providing very high throughputs has been increasing exponentially, namely due to bandwidth-hungry applications such as high definition video streaming and augmented reality. In order to fulfil these requirements, the Wi-Fi standard was enriched with new amendments, such as IEEE 802.11n, IEEE 802.11ac, and recently IEEE 802.11ax (Wi-Fi 6). New parameters have been proposed for both physical (PHY) and media access control (MAC) layers, including channel bonding, short guard interval (SGI), and advanced modulation and coding schemes (MCS). However, the high variability of the signal strength in the wireless radio channel, allied to the channel asymmetry, makes the selection of optimal configurations for these parameters a challenge. Typically, these parameters are configured with a default value. For runtime optimization, some algorithms have already been proposed. Still, they were designed considering legacy IEEE 802.11 releases and static scenarios. Besides, these parameters have their trade-offs that need to be properly managed. To help dealing with this, machine learning has been recently introduced in wireless networks, providing the intelligence that networks need in order to be smart and self-adaptive. SWOP (Smart Wireless Optimization) is a cross-layer optimization approach for Wi-Fi networks extending the current Rate Adaptation (RA) approach, for instance, followed by the well-known Minstrel algorithm widely used in practice. Our approach takes advantage of Deep Reinforcement Learning (DRL) in order to learn the optimal Wi-Fi link configuration. By considering the wireless channel as the environment, the transmitter node (the agent) chooses the best link parameters (the action) in order to maximize the throughput (the reward) based on the channel metrics captured from the environment (the state). In this work we propose a simple DRL-based Wi-Fi Rate Adaptation (RA) algorithm, named Data-driven Algorithm for Rate Adaptation (DARA), which is one of the modules of Smart Wireless Optimization (SWOP) SMART aimed to run a set of wireless experiments on top of w-iLab.t testbeds provided by the Fed4FIRE+ project to directly validate our DRL model and learn a policy from the wireless experiments executed in a controlled environment. However, after facing difficulties with the scenarios we could achieve on the real testbed, we decided to train and test DARA using a trace-based simulation approach. In simulation, we could train our model in scenarios that are more complex and diverse whilst easy to configure, when compared to real testbeds. The w-iLab.t testbeds were still used to capture data traces (e.g. Signal-to-Noise Ratio, position of nodes, transmission power and link distance) that were then injected in ns-3 for validating DARA. With this work, we concluded that DARA performance is impacted when operating in scenarios with asymmetric links, which is common in the highly dynamic and unpredictable wireless environments. Furthermore, the asymmetry offset varies between scenarios and it may also change for the same scenario, as time progresses. This randomness is not addressed when solely considering the SNR as the link metric, posing a challenge in the learning phase of DARA. Despite these limitations, the results obtained show that DARA still achieves up to 14.9% higher throughput higher than Minstrel [1] and slightly lower than Ideal [2] for most of the scenarios. The results obtained will serve as a basis to support our ongoing and future research. <strong>Folder Organization</strong> The following dataset presents the results of the SMART project, organized in different folders for each Rate Adaptation Algorithm, as well as the traces that were used to obtain such results: <strong>DARA: </strong>Results obtained using our solution <strong>(Naming Convention #1, Folder Content #1)</strong> <strong>MIN: </strong>Results obtained using Minstrel-HT <strong>(Naming Convention #1, Folder Content #2)</strong> <strong>ID: </strong>Results obtained using Ideal <strong>(Naming Convention #1, Folder Content #2)</strong> <strong>TRACES: </strong>Trace files used to obtain the results present in this dataset <strong>(Naming Convention #2, Folder Content #3)</strong> <strong>Naming Convention #1 – RAA TID TP TO:</strong> Rate Adaptation Algorithm<strong> (RAA) </strong> <strong>drl </strong>– Data Driven Algorithm for Rate Adaptation <strong>min </strong>– MinstrelHTWifiManager <strong>id </strong>– IdealWifiManager Trace ID<strong> (TID)</strong> <strong>3 </strong>up to<strong> 8</strong> Transport Protocol<strong> (TP)</strong> <strong>udp </strong>– User Datagram Protocol Traffic Orientation<strong> (TO)</strong> <strong>normal </strong>– A<strong>-&gt;</strong>B <strong>reversed </strong>– B<strong>-&gt;</strong>A <strong>Naming Convention #2 – TID_TXP:</strong> Trace ID<strong> (TID) </strong> <strong>3 </strong>up to<strong> 8</strong> Transmitting Power in dBm <strong>(TXP) </strong> <strong>3, 5, 7, 9, 12 dBm </strong> <strong>Folder Content #1: </strong> <em>checkpoint_ RAA TID TP TO</em><strong> (Folder)</strong> <strong>Policy Checkpoint</strong> with which the results were obtained <strong>Flowmonitor </strong>output for the configured scenario Column 1 – <strong>Step Counter</strong> Column 2 – <strong>Reward Value</strong> Column 3 – <strong>Observation Value</strong> Column 4 – <strong>Action Value</strong> Column 1 – <strong>Simulation Time </strong>(seconds) Column 2 – <strong>Throughput </strong>(Mbit/100ms) <strong>Folder Content #2: </strong> <strong>Flowmonitor </strong>output for the configured scenario Column 1 – <strong>Simulation Time </strong>(seconds) Column 2 – <strong>Throughput </strong>(Mbit/100ms) <strong>Folder Content #3 - </strong>Source: https://zenodo.org/record/3713271#.YjjBVDXLdhE<strong>: </strong> · <em>date_time</em><strong>.cfg </strong>configuration details of the experiment · <em>date_time_NodeID</em><strong><em><sup>[1]</sup></em></strong><em>_SenderID</em><strong><em><sup>[2]</sup></em></strong><em>_ReceiverID</em><strong><em><sup>[3]</sup></em></strong><em>_FlowType</em><strong><em><sup>[4]</sup></em></strong><em>_Params</em><strong><em><sup>[5]</sup></em></strong><strong>.snr </strong>– logs of the Signal/Noise ratio (1 file per node/flow) · <em>date_time_NodeID_SenderID_ReceiverID_FlowType_Params</em><strong>.stats</strong> – logs of the packets received (1 file per node/flow) <sub>[1]</sub><sub> ID of the node Logging node</sub> <sub>[2]</sub><sub> ID of the Sender node</sub> <sub>[3]</sub><sub> ID of the Receiver node</sub> <sub>[4]</sub><sub> Flow type: Unidirectional, Bidirectional or Unidirectional with Multiple Access</sub> <sub>[5]</sub><sub> Configurable parameters: Sender/Receiver Transmission Power and Data Rate (when applicable)</sub> <strong>References</strong> 1. F. FietKau, “Minstrel_HT: New rate control module for 802.11n [LWN.net]”. Mrt-2010. 2. “ns-3: ns3::IdealWifiManager Class Reference,” Jan 2021, [Online; accessed 23. Jun. 2021]. Available: https://www.nsnam.org/docs/release/3.33/doxygen/classns3_1_1_ideal_wifi manager.html