# Joint Spreading Factor and Channel Assignment in Multi-Operator LoRaWAN Deployments
## 6 Multi-Operator Game for Channel Selection
### Best Response Algorithm for Computing the NE  
We first propose a semi-distributed approach for computing the NE based on repeated best response dynamics. In every iteration \(t_{c}\), a player \(i\) (network operator) seeks to find its optimal channel selection as a response to \(\mathbf{x}_{c}^{-i}(t_{c}-1)\), the decisions of all the other players, by solving the following problem:  
\[\underset{x_{c}^{i}}{\text{Maximize }U_{i}^{e}(x^{i},\mathbf{x}^{-i})}\] (13a) Subject to \[\sum_{c=1}^{C}x_{c}^{i}\leq n,\ \ \forall i\in\mathcal{N}, \tag{13b}\] \[x_{c}^{i}\in\{0,1\}\ \ \forall i\in\mathcal{N},\forall c\in \mathcal{C} \tag{13c}\]  
The corresponding pseudo-code for our proposal can be seen in Algorithm 2. ```
1:Requires: Maximum tolerance \(\epsilon\gets 10^{-5}\)
2:Input: Initial channel assignment
3:Initialize:\(t_{c}=0\)
4:Do
5:\(t_{c}\)\(\leftarrow\)\(t_{c}\)+1
6:For i=1,...,\(\mathcal{N}\)
7: Solve the problem in (13) for \(i\)
8: Update channel selection
9: Compute \(\delta^{i}=||x_{c}^{i}(t_{g})-x_{c}^{i}(t_{g}-1)||\)
10:EndFor
11:While\(\exists\)\(i\) such that \(\delta^{i}\geq\epsilon\)

### Distributed Learning-Based Approach to Computing the NE

In this section, we propose using replicator dynamics to solve the problem in a completely decentralized manner. Because our game possesses a potential function, the paper in [15] attests that replicator dynamics will converge almost surely to a pure Nash equilibrium. We first solve the problem for \(n=1\), i.e., each operator only chooses one channel for transmissions. The goal is to stochastically learn the best strategy (channel choice). This is done with each operator selecting the channel randomly, with a set of probabilities \((p_{1}^{i},p_{2}^{i},...,p_{C}^{i})\). This set is maintained by each player \(i\) and it is iteratively updated until convergence. Let \(p_{c}^{i}(t)\) be the probability that network operator \(i\) picks channel \(c\). The sum of all the probabilities is equal to 1, for each network \(i\), i.e, \(\sum_{c\in C}p_{c}^{i}(t)=1\ \forall\ i\in\mathcal{N}\), \(\forall\ t\). At the beginning of the learning process, channels are selected with equal probabilities. At every time step \(t\), the network chooses a certain channel that is based on the consecutively updated probabilities and it is subsequently issued a reward. The resulting utility output is a negative real because our objective introduces a logarithmic function, and since our algorithm operates on numerical values of the tenths order. The reward is, as such, evaluated in terms of minimum cost as opposed to maximum profit. Among the different existing time-difference reinforcement learning approaches, we elect to update the probabilities, after any channel selection is made, as follows:

\[p_{c}^{i}(t+1)=\left\{ \begin{array}{ll}p_{c}^{i}(t)+\beta\mathcal{R}(1-p_{c }^{i}(t)),&\text{if}\ \ \theta_{ic}=1\\ p_{c}^{i}(t)-\beta\mathcal{R}p_{c}^{i}(t),&\text{otherwise}\end{array} \right. \tag{14}\]

In the function above, \(\theta_{ic}\) is a variable that is equal to one if network operator \(i\) chooses channel \(c\), and zero otherwise. \(\beta\) is the learning rate, which is chosen between 0 and 1, and \(\mathcal{R}\) is the reward. The reward reflects how good the channel selection was and it is computed as:

\[\mathcal{R}=1-\frac{U_{l}^{c}}{U_{max}}, \tag{15}\]

where \(U_{max}\) quantifies an upper bound for the player utility. The learning rate \(\beta\) controls both the speed with which the algorithm converges towards a preferred channel selection choice for the different networks, as well as the efficiency of the choice. A high value of \(\beta\) would incur quicker decisions, but it might miss the Nash equilibrium and lead to inefficient decisions. Ideally, we want to chose the highest value of \(\beta\) that would always converge towards an efficient Nash equilibrium. The network needs to know the packet success rate (total packet delivery ratio) in order to estimate the reward.