## 5 Performance Evaluation  
In this section, the performance of the MAB-based channel selection methods is evaluated by simulation. In the simulation, there are two kinds of massive IoT networks where ALOHA communication without channel sensing is adapted. IoT devices in one network transmit data using the channel selected by the MAB-based methods. IoT devices in the other network transmit data using the allocated fixed channel following a two-state Markov model [34]. The two states are ON state and OFF state. During the ON state, IoT devices operate regularly, including wake-up and sleep modes. During the wake-up mode, the IoT device transmits data using the allocated fix channel. During the sleep mode and OFF state, IoT devices keep silent without transmitting data. The state transition probability of the two-state Markov model can be expressed as:  
\[P= \begin{pmatrix}\text{ON-ON}&\text{ON-OFF}\\ \text{OFF-ON}&\text{OFF-OFF}\end{pmatrix} = \begin{pmatrix}\frac{1+\lambda_{k}}{2 }&\frac{1-\lambda_{k}}{2}\\ \frac{1-\lambda_{k}}{2}&\frac{1+\lambda_{k}}{2}\end{pmatrix} , \tag{17}\]  
where \(\lambda_{k}\) is a parameter that indicates the ease of transition of the state transitions in channel \(k\) and its range is from \(-1\) to \(+1\). This formulation means that the network continues the past state (i.e., ON-ON or OFF-OFF) with a probability of \(\frac{1+\lambda_{k}}{2}\) and transitions from the past state to a different state (i.e., ON-OFF or OFF-ON) with a probability of \(\frac{1-\lambda_{k}}{2}\). The transmission is a failure if two or more IoT devices simultaneously access the same channel. This paper verifies the effectiveness of applying the MAB algorithm such as \(\epsilon\)-greedy, UCB1-tuned, and basic TOW and MTOW to the channel selection problem in a massive heterogeneous IoT network and evaluates the network performance in terms of FSR. The common simulation settings are summarized in Table 1. Note that the value of \(\epsilon\) is set to 0.1 in the simulation. Figure 6 shows an example of channel selection based on the TOW method. In the simulation, the numbers of IoT nodes and channels are set to 10 and 15, respectively. Among the 15 channels, 12 channels are loaded by the other IoT network. The orange line in Figure 6 indicates the loaded channel and the loaded time by the IoT devices of the other IoT network. The channel access situation of the 10 IoT devices in the IoT network accessing channels based on TOW algorithms are shown at different points. Simulation results show that the IoT devices can avoid accessing the channels loaded by other IoT networks. The reason is that IoT devices can select the channel with the highest probability based on the TOW method. The communication fails when IoT device selects the channel that are loaded by other IoT networks, which reduces the probability of the selected channel. By iterative selection and the update of the probability parameter corresponding to each channel, IoT devices can select the channels without load. \begin{table}
\begin{tabular}{c c} \hline Simulation time [s] & 10,000 \\ \hline Number of nodes \(M\) & 100, 1000, 10,000 \\ \hline Number of available channels \(K\) & 15, 30, 60 \\ \hline Duty cycle & 0.01, 0.1 \\ \hline Number of load channels & 1/5, 2/5, 3/5, 4/5 of \(K\) \\ \hline Load duty cycle [\%] & 0.1, 0.5, 0.9 \\ \hline Duration of each state [s] & 100 \\ \hline \(\lambda\) & 0.8 \\ \hline Forgetting parameter \(\alpha\) & 0.95 \\ \hline \(\epsilon\) & 0.10 \\ \hline \end{tabular}
\end{table} 
Table 1: Simulation settings. Figure 6: Example of channel selection based on TOW method in the heterogeneous IoT networks. Figure 7 illustrates the relation between the FSR and the number of IoT nodes. In this simulation, the number of IoT devices varies from \(10^{2}\) to \(10^{4}\). The number of channels is set to 20. The parameter related to the state transition probability of the IoT devices in the network where IoT devices transmit data using the allocated fixed channels \(\lambda\) is set as 0.8. The duty ratio is set to 0.5.