Abstract-We propose a dynamic spectrum access scheme where secondary users recommend "good" channels to each other and access accordingly. We formulate the problem as an average reward based Markov decision process. Since the action space of the Markov decision process is continuous (i.e., transmission probabilities), it is difficult to find the optimal policy by simply discretizing the action space and use the policy iteration, or value iteration. Instead, we propose a new algorithm based on the Model Reference Adaptive Search method, and prove its convergence to the optimal policy. Numerical results show that the proposed algorithm achieves up to 18% performance improvement than the static channel recommendation scheme and up to 63% performance improvement than the random access scheme, and is robust to channel dynamics.
In REF , the problem of channel recommendation is described as an average reward based Markov decision process, and a model reference adaptive search method is proposed.
Adaptive channel recommendation for dynamic spectrum access
2011 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)
2011 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)
Computer Science