A dynamic programming approximation for downlink channel allocation in cognitive femtocell networks
Both femtocells and cognitive radio (CR) are envisioned as promising technologies for the NeXt Generation (xG) cellular networks. Cognitive femtocell networks (CogFem) incorporate CR technology into femtocell deployment to reduce its demand for more spectrum bands, thereby improving the spectrum utilization. In this paper, we focus on the channel allocation problem in CogFem, and formulate it as a stochastic dynamic programming (SDP) problem aiming at optimizing the long-term cumulative system throughput of individual femtocells. However, the multi-dimensional state variables resulted from complex exogenous stochastic information make the SDP problem computationally intractable using standard value iteration algorithms. To address this issue, we propose an approximate dynamic programming (ADP) algorithm in pursuit of an approximate solution to the SDP problem. The proposed ADP algorithm relies on an efficient value function approximation (VFA) architecture that we design and a stochastic gradient learning strategy to function, enabling each femtocell to learn and improve its own channel allocation policy. The algorithm is computationally attractive for large-scale downlink channel allocation problems in CogFem since its time complexity does not grow exponentially with the number of femtocells. Simulation results have shown that the proposed ADP algorithm exhibits great advantages: (1) it is feasible for online implementation with a fair rate of convergence and adaptability to both long-term and short-term network dynamics; and (2) it produces high-quality solutions fast, reaching approximately 80% of the upper bounds provided by optimal backward dynamic programming (DP) solutions to a set of deterministic counterparts of the formulated SDP problem.
channel allocation femtocell cognitive radio approximate dynamic programming