## 1 Introduction  
Multi-armed bandit (MAB) algorithms are the algorithms developed to solve MAB problems [1]. The name MAB problem resulted from an analogy of a scenario in which a player selects one slot machine (also known as a one-armed bandit) among many of them sequentially to maximize the total profit at the end of the game. MAB algorithms attempt to find the best strategies in such scenarios by adequately determining the trade-off between the exploitation of prior knowledge and the exploration of new opportunity. MAB algorithms have attracted significant interest from not only academia but also the industry because they can be applied to solve many practical problems in various areas, including commercial applications such as online advertisements and financing. The epsilon-greedy algorithm (\(\epsilon\)-greedy) [1], upper confidence bound (UCB) [2], exponential-weight algorithm for exploration and exploitation (Exp3) [3], Kullback-Leibler UCB (KL-UCB) [4], and Thompson sampling [5, 6] are examples of MAB algorithms. Among the MAB algorithms, KL-UCB and Thompson sampling might be the most popular ones owing to their performance in maximizing rewards while minimizing regrets [7]. Today, the demand for wireless communications continues to increase partly because wireless communications have become essential to the daily lives of most people in this highly connected world. Furthermore, recent advances in Internet of Things (IoT) technologies [8] and their usage [9] have placed an even higher demand for wireless communications. To mitigate the ever-increasing demand for wireless communications, technologies such as edge/fog computing [10, 11, 12] are becoming important parts of communication infrastructures. In addition to cellular wireless networks such as long-term evolution (LTE) [13] and 5G [14], carrier-sensing multiple access/collision avoidance (CSMA/CA) wireless communication standards (i.e., IEEE 802.11 [15]) play important roles by providing an alternative method of relatively easy and cheap wireless communication. However, because of the lack of the central control among mostly self-installed and self-managed CSMA/CA wireless networks, they often result in low resource utilization of network resources such as bandwidth and frequency channels, particularly when there are multiple users and access points in the vicinity. Furthermore, when CSMA/CA wireless networks operate in a purely distributed manner, such as the distributed coordination function (DCF) mode [15], inadequate selections of data transmission rates can significantly affect the performance of the wireless networks, particularly when the wireless channel status is dynamically changing or users are involved with movement. To overcome the difficulties, many studies have introduced algorithms for adaptive transmission rate control such as Minster [16], collision-aware rate adaptation (CARA) [17], Q-learning-based rate control [18], and deep learning-based approaches [19]. These algorithms control the transmission rates of data frames by adaptively selecting an appropriate modulation and coding scheme (MCS) in response to the changes in the status of the wireless channel between a sender and receiver of the data frames. Furthermore, if multiple communication links are available between a sender and receiver through the use of multiple antennas and the multiple-input multiple-output (MIMO) mode, most transmission rate control algorithms also determine the number of spatial streams (e.g., \(1\times 1\) or \(2\times 2\)) together with the MCS to be used for each transmission among possible combinations to achieve better performance in the given wireless environment. Hereinafter, we use the term MCS level to designate not only the MCS but also the number of communication links to be used for a transmission. In recent years, researchers have introduced MAB algorithms to improve the performance of CSMA/CA wireless communication networks by considering the selection of the adequate MCS level as a special case of the MAB problem.