Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network following the binary offloading policy, such that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm under time-varying wireless channels that jointly optimizes task offloading decisions and wireless resource allocation to maximize the data processing capability of the network. In practice, this requires successively solving hard combinatorial optimization problems to address the multi-user correlation in the offloading decisions. The existing approaches are mainly based on either branch-and-bound algorithm or relaxation heuristics, which are limited by the tradeoff between optimality and efficiency. To tackle this problem, we propose in this paper a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network to generate offloading decisions. In particular, the proposed DROO framework does not require any manually labeled training data as the input, and thus completely removes the need of solving combinatorial optimization problems. Besides, it avoids the curse of dimensionality problem encountered by some existing reinforcement learning approaches and is computationally efficient in large-size networks. To further reduce the computational complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing methods. For example, the complexity is reduced from several seconds to less than 0.1 second in a 30-user network, making real-time and optimal offloading design truly viable even in a fast fading environment. Mobile-edge computing, wireless power transfer, reinforcement learning, resource allocation. L. Huang is with the College 2 I. INTRODUCTION Modern Internet of Things (IoT) technology is fundamentally constrained by limited battery lifetime and low computing power of size-constrained wireless devices (WDs), especially for computation intensive applications such as augmented reality. Thanks to the recent advance in wireless power transfer (WPT) technology, the batteries of WDs can be continuously charged over the air without the need of battery replacement [1]. Meanwhile, the device computing power can be effectively enhanced by the recent development of mobile-edge computing (MEC) technology [2]. With MEC, the WDs can offload intensive computations to nearby edge servers for reduced computation latency and energy consumption [3]. The newly emerged wireless powered MEC combines the advantages of the two aforementioned technologies, whose deployment in IoT networks is promising to solve the two fundamental performance limitations [4], [5] . In this paper, we consider a wireless powered MEC network as shown in Fig. 1 , where the access point (AP) is responsible for both transferring RF (radio frequency) energy to and receiving computation offloading from the WDs. In particular, the WDs follow a binary task offloading policy [6] , which is a commonly used offloading model for non-partitionable simple sensing tasks in IoT networks. In this case, a task is either computed locally or offloaded to the MEC server for remote computing. In a multi-user scenario, a major challenge is the joint optimization of individual computing mode selection (i.e., offloading or local computing) and transmission time allocation (on WPT and offloading) to achieve the maximum computation performance. Due to the combinatorial nature of computing mode selection, the problem is generally formulated as mixed integer programming (MIP). To tackle the MIP problem, branch-and-bound algorithms [7] and dynamic programming [8] are used to solve for the globally optimal offloading solution. However, the search spaces of both methods increase exponentially with the network size N and are computationally prohibitive for large-scale MEC networks. To reduce computational complexity, heuristic local searching methods are proposed to reduce the computational complexity. For instance, [5] proposed a coordinate descent (CD) method that searches along one binary variable at a time. A similar heuristic search method for multi-server MEC networks was studied in [9], which iteratively adjusts binary offloading decisions. Another widely adopted heuristic is through convex relaxation, e.g., by relaxing integer variables to be continuous between 0 and 1 [10] or by approximating the binary constraints with quadratic constraints [11] . Nonetheless, on one hand,
More recently, a deep reinforcement learning framework for task offloading was studied in a single-AP scenario REF .
Deep Reinforcement Learning for Online Offloading in Wireless Powered Mobile-Edge Computing Networks
ArXiv
ArXiv
Computer Science