# Decentralized Blind Spectrum Selection in Cognitive Radio Networks Considering Handoff Cost
## 3 Problem Formulation and Policies
// Main loop  For \(t=N+1\) to \(T\)  Step1: Select the set \(\mathcal{O}_{K}\) contains arms with the \(K\) highest index values: \[\hat{\theta}_{i}(t-1)+\sqrt{\frac{2\ln t}{n_{i}(t-1)}}.\] Step2: Play the arm with the minimal index value in \(\mathcal{O}_{K}\) according to \[\hat{\theta}_{i}(t-1)-\sqrt{\frac{2\ln t}{n_{i}(t-1)}}.\] Step3: Update \(n_{i}(t)\) and \(\hat{\theta}_{i}(t)\). EndFor
```  
**Algorithm 2** SL(K) policy for the user with rank \(K\). ```
1. //Define: \(n_{i}(t)\): the number of arm \(i\) is played after \(t\) slots. \(\hat{\theta}_{i}(t)\): sample mean availabilities after \(t\) slots. \(\epsilon_{t}:=\min[\frac{\beta}{T},1]\), where decay rate \(\beta\) is prior valuated according to the arm reward means \(\quad\)// Init: play each arm once  For \(t=1\) to C  Play arm \(i=t\) and let \(n_{i}(t)=1\), \(\hat{\theta}_{i}(t)=X_{i}(t)\)  EndFor
```  
**Algorithm 3** \(k\)th-UCB1 policy for the user with rank \(K\). ```
1. //Define: \(n_{i}(t)\): the number of arm \(i\) is played after \(t\) slots. \(\hat{\theta}_{i}(t)\): sample mean availabilities after \(t\) slots. \(\epsilon_{t}:=\min[\frac{\beta}{T},1]\), where decay rate \(\beta\) is prior valuated according to the arm reward means \(\quad\)// Init: play each



The above policies are derived and investigated in the scenario where there is no expense when the player switches from one arm to another. However, in CRNs, the handoff cost should be taken into consideration as illustrated in Section 2. Therefore, let

\[H^{\pi}(t)=H\!\!\sum_{i=1}^{C}\sum_{j=1}^{M}\mathbb{I}_{i,j}(t)\mathbb{I}_{i,j}(t) \tag{5}\]

be the sum of handoff cost of all users at slot \(t\), where \(\mathbb{I}_{i,j}(t)\) is the indicator if user \(j\) switches to arm \(i\) from other arms. Then, define the handoff regret as

\[HR^{\pi}(\Theta;T)=\mathbb{E}^{\pi}[\sum_{t=2}^{T}H^{\pi}(t)]. \tag{6}\]

We define the total regret as

\[R^{\pi}(\Theta;T)=R^{\pi}_{M}(\Theta;T)+HR^{\pi}(\Theta;T) \tag{7}\]

Since the inclusion of the handoff delay \(H\), \(HR^{\pi}(\Theta;T)\), and \(R^{\pi}_{M}(\Theta;T)\) are correlatively related, it is difficult to make a theoretical analysis of the total regret in a distributed multi-user case, although the authors of [15] considered this problem in a single-user case. In the next section, we examine the above policies by simulation and discuss their performance. ## 4 Simulation Results and Analysis

In this section, we present simulation results for the scheme proposed in this work. Simulations are done using Matlab, and we assume \(C=9\) channels with channel availabilities B = [0.5, 0.2, 0.8, 0.6, 0.9,.03, 0.4, 0.1, 0.7] and \(M=3\) SUs. The policy parameter configuration is the decay rate \(\beta=400\) for the \(\varrho^{\text{PRE}}\) policy and \(\beta=50\) for the \(k\)th-UCB1 policy, which is an optimal configuration according to the authors of [11], and the SL(K) policy is parameterless. The time scope is \(T=5\times 10^{4}\). Every experiment is repeated 50 times. As the regret of one user already takes the collision into consideration, the total regret of a CRN is simply the sum of all users in that CRN. Therefore, we present the regret and actions of one user in CRN where we take SU with the rank \(K=2\). Figure 3 shows the regrets and actions of these policies under fixed handoff cost \(H=0.1\). \[\sum_{t=1}^{T}S^{*}(t)=T\sum_{i\in\mathcal{O}_{M}^{*}}\Theta_{i}. \tag{2}\]0 The regrets and action histogram of \(\varrho^{\text{PRE}}\) (**a**), \(k\)th-UCB1 (**b**), and SL(K) (**c**) when \(H=0.1\). From Figure 3, we can see that the three policies all achieve the logarithm regret and the actions converge to the third arm, which is the arm with the second-best channel availability. The regret of policy \(k\)th-UCB1 (Figure 3b) is smaller than that of policy \(\varrho^{\text{PRE}}\) (Figure 3a). This is caused by two aspects. Firstly, the decay rate \(\beta\) in \(k\)th-UCB1 can be smaller than \(\varrho^{\text{PRE}}\) and does not make the policy diverge.