Tutorial on Machine Learning for Spectrum Sharing in Wireless Networks

As spectrum utilization efficiency is the major bottleneck in current wireless networking, many stakeholders discuss that spectrum should be shared rather than being exclusively allocated. Shared spectrum access raises many challenges which, if not properly addressed, degrades the performance level of the co-existing networks. Coexistence scenarios may involve two or more networks: same or different types; operated by the same or different operators. The complex interactions among the coexisting networks can be addressed by the machine learning tools which by their nature embrace uncertainty and can model the complex interactions. In this tutorial, we start with the basics of coexistence of wireless networks in the unlicensed bands. Then, we focus on WiFi and LTE-U coexistence. After providing a brief overview of machine learning topics such as supervised learning, unsupervised learning, reinforcement learning, we overview five particular examples which exploit learning schemes to enable efficient spectrum sharing entailing a generic cognitive radio setting as well as LTE and WiFi coexistence scenarios. We conclude with a list of challenges and research directions. I. SPECTRUM SHARING IN WIRELESS NETWORKS Spectrum sharing is the situation where at least two users or technologies are authorized to use the same portion of the radio spectrum on a non-exclusive manner1. We overview the current state of spectrum sharing and provide a taxonomy of spectrum sharing scenarios. We can list the main challenges in providing peaceful coexistence as follows: (i) scarcity of the resources, (ii) heterogeneity of the coexisting networks, (iii) power asymmetry, and (iv) lack of coordination, communication, and cooperation among the coexisting networks. II. COEXISTENCE IN THE UNLICENSED BANDS: THE CASE OF WIFI AND LTE-U The success of IEEE 802.11 networks in the unlicensed bands, i.e., 2.4 GHz and 5 GHz, has proved the efficiency and feasibility of using spectrum in a license-exempt manner. Currently, even the cellular providers consider expanding their network’s capacity with unlicensed spectrum to cope with the increasing wireless traffic demand. More particularly, Qualcomm’s 2013 proposal  of aggregating 5GHz bands with the licensed carriers of an LTE network has paved the way for LTE unlicensed networks. However, operation in the unlicensed bands has to address the coexistence challenges. For example, WiFi networks at 2.4 GHz bands, e.g., 802.11b/g/n, have to find the best channel 1S. Forge, R. Horvitz, and C. Blackman. Perspectives on the value of shared spectrum access. Final Report for the European Commission, 2012. among three non-overlapping channels for operation in a very-dense WLAN deployment. Additionally, 2.4 GHz band accommodates also non-WiFi technologies such as Bluetooth, ZigBee, or microwave ovens which all create interference on WLANs. As for 5GHz which has many more non-overlapping channels compared to 2.4 GHz, the more severe challenge is to coexist with technologies other than 802.11n/ac/ax networks, namely unlicensed LTE networks and radars. The main coexistence mechanism of WiFi is listen-beforetalk (LBT) which is also known as carrier sense multiple access with collision avoidance (CSMA/CA). A station with a traffic to transmit has to first check whether the medium is free or not. To decide on the state of the medium, two approaches exist: carrier sense and energy detection. In carrier sensing, a WiFi node decodes the preamble of a WiFi frame that is received above some energy detection level. The node extracts the information from the PLCP header which carries information about the occupancy duration of the medium by that ongoing flow. This mechanism is also referred to as channel reservation. With this information, a WiFi node knows when to re-start sensing the medium for a transmission opportunity. Energy detection (ED) is a simpler approach in which a candidate transmitter decides that the air is free if the signal level is below a predefined ED threshold. This approach is used for detecting inter-technology signals, where the received signal is not decodable, i.e., it belongs to other technologies (or corrupted WiFi signals). Despite its simplicity, ED requires more effort on a potential transmitter as it must constantly sense the energy level in the air to detect a transmission opportunity. As LTE follows a scheduled medium access on the licensedspectrum, there is no notion or necessity of politeness or LBT in more technical terms. However, it is vital for LTE unlicensed to implement such mechanisms for coexistence with WiFi and other unlicensed LTE networks at 5 GHz bands. Currently, frequency-domain sharing is a first step only. In other words, an LTE small cell first checks the channel activities and selects a clear channel, if any. For time sharing, there are two approaches taken by two variants of LTE unlicensed: duty cycling by LTE-U and LBT by License-Assisted-Access (LAA). LTE-U which is an industry-led effort lets small cells apply duty cycling where during the OFF periods WiFi can access the medium. As this approach does not mandate LBT before turning small cell transmissions on, it may degrade WiFi performance drastically. LAA requires LBT similar to WiFi’s CSMA/CA. LAA speciCROWNCOM2017 TUTORIAL 2 fication is led by 3GPP and aims to develop a global solution in contrast to LTE-U which is only compliant to countries like US, Korea, China where LBT is not mandatory. We overview basics of these two variants and list the major issues in their peaceful coexistence with WiFi networks. III.  ON MACHINE LEARNING We provide a sparse overview of learning approaches: supervised, unsupervised, and reinforcement learning. IV. THE ROLE OF MACHINE LEARNING IN SPECTRUM SHARING AND COEXISTENCE In this part, we examine the literature using ML approaches to solve the coexistence issues as our case studies. Is the channel idle or busy?This question is at the heart of coexistence of networks in a multi-channel environment, as the first step of coexistence is to choose a channel that is clear. For cognitive radio networks, it is mandatory to detect the channel state to avoid violating the rules of secondary spectrum access. Casting this question into a binary classification problem, authors  introduce several (un)supervised learning algorithms to correctly identify the state of a channel. While supervised approaches require the real channel state information from the Primary Users, unsupervised learning such as K-means does not require any input from the PUs which is a desirable property of classification scheme in a practical setting. Which unlicensed channel to select for each LAA SBS for inter-operator coexistence?As we expect multiple LAA operators deploy their small cells independently, there is surely the question of how to select an unlicensed channel to aggregate, particularly in case there are more cells than the number of available channels. One way of channel selection is to let every LAA BS learn from its own observations via trial-and-error, Q-learning . Which unlicensed carrier to aggregate and how long to use this carrier?Q-learning framework can also be applied to an LAA setting where an LAA BS needs to select an unlicensed carrier and the transmission duration on the selected carrier . Can WiFi exploit ML for defending itself against LTEU interference?Different than the literature which develops coexistence solutions to be deployed at the LTE base stations for WiFi/LTE setting,  proposes to also equip the WiFi APs with a tool that estimates the ON-duration of an existing LTE-U network in the neighborhood. Moreover, the developed solution can estimate the remaining airtime for the WiFi AP based on the LTE’s predicted ON duration. Key idea of WiPLUS is to detect the times where LTE-U has an ongoing transmission using the data passively collected from the MAC FSM of the NIC. However, although LTE-U signal may not be detected above the ED level, it may still have a severe impact on WiFi. Thus, PHY-layer analysis solely on signal level is short of detecting the moderate interference regime. WiPLUS overcomes this challenge by combining data from MAC FSM states and ARQ missing acknowledgments. Sampled data from a testbed has a lot of noise due to imperfections of the measuring devices and the complex interactions among the coexisting systems as well as PHY and MAC layers. WiPLUS applies K-means clustering to detect outliers on the estimated LTE-U on-durations. After filtering the data points based on the signal’s frequency harmonics, WiPLUS calculates the LTEU on-time as the average of the data points, each of which corresponds to an estimate of LTE-U on-time. Can we estimate WiFi link performance by learning from real-world link capacity measurements?In a multiAP setting, an AP can select the operation channel based on the expected capacity of the existing links. The traditional way is to take the SNIR-based capacity estimate into account, i.e., Shannon’s capacity formula. However, this capacity model may sometimes fail to represent the complex interactions between PHY and MAC layers, e.g., partially-overlapping channels in case of channel bonding in new 802.11ac/ax standards. The idea of [6] is to use supervised learning as a tool to model the complex interactions among many factors such as power and PHY rate of a neighboring WiFi link implicitly rather than modelling it explicitly. V. OPEN RESEARCH DIRECTIONS Machine learning-based solutions can embrace the complexity and uncertainty prevalent in the complex scenarios, especially hybrid horizontal spectrum sharing, by learning from the observations. However, a wireless network poses peculiar challenges such as the energy limitations, real-time operation, and sometimes fast changes in the operation environment that render learning less effective. We overview such challenges and conclude with some open questions in this