# Towards Collaborative and Dynamic Spectrum Sharing via Interpretation of Spectrum Access Policies
## 5 Policies
### Learner  
One of the main responsibilities of the PE is to automatically and periodically determine optimal values for the Modulation and Coding Scheme (MCS) and the Transmit Power (PTx). The inputs to this process are the current settings and the history of previously used values for these parameters. The proposed transmissions are checked by the Policy Reasoner for the purpose of checking conformance with the active policies. The policy rules are executed by the BaseVISor reasoning engine, which is part of the VIStology Inference Store component. The conceptual view of the optimization and learning algorithm while selecting proposed values of transmit power and MCS is shown in Figure 9. According to this figure, the PE maintains a representation (a table) of possible transmit states (_knobs_) and relations to other neighboring knobs. Periodically, the algorithm selects a knob and sends it to the radio to use in the next transmissions. The radio then sends feedback to the PE summarizing the performance of the radio using the selected knobs in the current environment parameters. The feedback is computed based on the dependency of the Bit Error Rate (BER) on the ratio of Energy per Bit (Eb) to the Spectral Noise Density (No), Eb/No. After receiving the feedback, the PE updates its state table accordingly. This is the learning step. Then, the algorithm selects the next knob to be sent to the radio. In order to analyze the performance of the PE, we implemented the initialization and the main loop of the algorithm in Matlab. The conceptual model of the state transitions (Figure 9) was implemented as a table. It is a \(5\times 3\) matrix, in which each row corresponds to a specific value of the transmit power, while each column corresponds to a specific MCS. There are two matrices associated with this model; we call them PTable and KTable. The entries in the PTable represent the expected quality of the transmission using the specific knobs. These values are updated by the learning procedure. The KTable, on the other hand, keeps the probabilities of success weighted by the MCS. The entries of this table were calculated by multiplying the PTable by 2, 3 and 4 for QPSK, 8-PSK and 16-QAM, respectively. To compute the feedback, we first simulated the Bit Error Rates (BER) of various modulation schemes over an additive white Gaussian noise (AWGN) channel. The input argument for this simulation was, \(E_{b}/N_{0}\), i.e., the ratio of bit energy to noise power spectral density, in dB, corresponding to the different \(E_{b}/N_{0}\) levels. The feedback calculations based on these models are shown in Figure 10.  
Figure 10: Success rate based on the BER models for different modulations. Figure 9: Modification of transmission parameters. We used a simplified reinforcement learning procedure. First, noise was estimated according to the noise model described above. Based on the noise and the power level, \(P_{tx}\), the values of \(E_{b}/N_{0}\) were computed and then used to find BER, Packet Error Rate (PER) and \(p\) (where \(N\) was set to 800):  
\[PER=1-(1-BER)^{N}\]  
This was followed by the application of the value of \(p\) to the update rule of the _PTable_:  
\[PT\text{{\it{Table}}}(row,col)=p_{cur}+(p_{new}-p_{cur})\cdot r\]  
This was then followed by updating the KTable, as described earlier, and the selection of the maximal value and then using its matrix coordinates to select Power and MCS. To date, the above learning rule was used only for a single value update. In the next version of this algorithm, we plan to propagate the updates to other neighbors of the current node. The result of simulations are shown in Figure 11.  
As we can see from this figure, in most of the steps, the algorithm selects values that guarantee the highest success rate. However, in some steps, the success rate falls very rapidly. Although it recovers in two steps, this is still a rather unexpected behavior, and such a drop is highly undesirable. The explanation of this behavior can be traced to the noise model.