# A Reinforcement-Learning-Based Distributed Resource Selection Algorithm for Massive IoT
## 6 Performance Evaluation  
To validate that the cognitive-IoT prototypes make the decisions to select the suitable channels to transmit adaptively, a series of experiments were conducted. A photograph of the experiment setting is shown in Figure 6. The cognitive-IoT prototypes (see Figure 7) were deployed in a 1 m \(\times\) 1 m area with high density, where they communicated with three gateways (see Figure 8) operating on different channels with a star topology, following the proposed procedure introduced in Section 5. The deployment and topology were static throughout the experiments. In addition, the bandwidth of all three available channels was 200 kHz. Each IoT node had a 1000-ms sleep interval. \(\alpha\) was chosen as 0.995 in the experiments. Table 2 summarizes the experiment parameters. Figure 8: Gateways. Figure 6: Photograph of the experiment setting. Figure 7: Cognitive-IoT prototypes. Furthermore, to validate that the cognitive-IoT prototypes select suitable channels adaptively, the externally-offered loads that were generated by the IEEE 802.15.4e IoT devices were added to evaluate the effectiveness of the cognitive-IoT prototypes in four different test scenarios, which are summarized in Table 3. The sleeps internal of all the IEEE 802.15.4e IoT devices was set as 100 ms.  
First, to evaluate that the cognitive-IoT prototypes can improve the successful frame delivery of the network by adaptively selecting the suitable channel to send the frame, the Frame Successful Ratio (FSR) (see Equation (13)) was compared with that of the evenly assigned (EA) IoT devices. Here, the EA IoT devices represent the IEEE 802.15.4e IoT devices on which we implemented the standard procedure of IEEE 802.15.4e on the same SBC with the cognitive-IoT prototype shown in Figure 5. These EA IoT devices were set up to operate statically on the assigned channels where devices were EA on each available channel. Because the EA IoT devices operated based on the standard procedure of IEEE 802.15.4e, the EA IoT devices were not be able to change their channel once their operating channel was pre-determined. Each EA IoT device periodically went to sleep to save energy, which is the same as the cognitive-IoT prototype. The sleep intervals of all the EA IoT devices were set to the same as those of the cognitive-IoT prototypes, i.e., 1000 ms.  
\[FSR=\frac{\sum_{j=1}^{M}\sum_{i=1}^{K}R_{i}^{j}}{\sum_{j=1}^{M}\sum_{i=1}^{K}N _{i}^{j}} \tag{13}\]  
where \(M\) is the number of devices and, accordingly, \(j\) is the index of the device. Meanwhile, \(K\) is the number of available channels, and \(i\) is the index of the available channel. The FSR results are shown in Figure 9. In the \(Ld=(0,0,0)\) test scenario, the FSR of the cognitive-IoT prototype was practically the same as that of the EA IoT devices. As for the other test scenarios with an added externally-offered load, i.e., \(Ld(0,2,3),Ld(0,1,4)\), and \(Ld(0,0,5)\), the FSR results of the cognitive-IoT prototypes were typically higher than those of the EA IoT devices. This was because the cognitive-IoT prototypes generally adaptively selected appropriate channels to send the data frames successfully, whereas the EA IoT devices used the assigned channels, which became congested and failed to send the data frames. It can be stated that by employing the proposed TOW-dynamics-based algorithm to select suitable channels adaptively, the cognitive-IoT prototype improved the FSR of the network. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **North** \\ \hline Available channels & 924.6 MHz (CH44), 925.8 MHz (CH50), 927.0 MHz (CH56) \\ Number of cognitive-IoT prototypes & 30 \\ Data rate (kbps) & 50 \\ Transmission power (mw) & 20 \\ Sleep mode interval of the cognitive-IoT prototype (ms) & 1000 \\ \(\alpha\) & 0.995 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Experiment Parameters.