A personalized QoE-aware handover decision based on distributed reinforcement learning
Recent developments in heterogeneous mobile networks and growing demands for variety of real-time and multimedia applications have emphasized the necessity of more intelligent handover decisions. Addressing the context knowledge of mobile devices, users, applications, and networks is the subject of context-aware handoff decision as a recent effort to this aim. However, user perception has not been attended adequately in the area of context-aware handover decision making. Mobile users may have different judgments about the Quality of Service (QoS) depending on their environmental conditions, and personal and psychological characteristics. This reality has been exploited in this paper to introduce a personalized user-centric handoff decision method to decide about the time and target of handover based on User Perceived Quality (UPQ) feedbacks. The UPQ degradations are mainly for the sake of (1) exiting the coverage of the serving Point of Attachment (PoA) or (2) QoS degradation of serving access network. Using UPQ metric, the proposed method obviates the necessity of being aware about rapidly varying network QoS parameters and overcomes the complexity and overhead of gathering and managing some other context information. Moreover, considering the underlying network and geographical map, the proposed method is able to inherently exploit the trajectory information of mobile users for handover decision. UPQ degradation is not only due to the user behaviour, but also due to the behaviours of others users. As such, multi-agent reinforcement learning paradigm has been considered for target PoA selection. The employed decision algorithm is based on WoLF-PHC learning method where UPQ is used as a delayed reward for training. The proposed handoff decision has been implemented under IEEE 802.21 framework using NS2 network simulator. The results have shown better performance of the proposed method comparing to conventional methods assuming regular movement of mobile users.
qoe-aware handover distributed reinforcement learning context-aware handover user perceived quality