Maximizing Profits of Allocating Limited Resources under Stochastic User Demands Nowadays, cloud brokers play an important role for allocating resources in the cloud computing market, which mediate between cloud users and service providers by buying a limited capacity from the providers and subleasing them to the users to make profits. However, the user demands are usually stochastic and the resource capacity bought from cloud providers is limited. Therefore, in order to maximize the profits, the broker needs an effective resource allocation algorithm to decide whether satisfying the demands of arriving users or not, i.e. need to allocate the resource to a valuable user. In this paper, we propose a resource allocation algorithm named QDP, which is based on reinforcement learning and dynamic programming, for the broker to maximize the profits. First, we consider all arriving usersu0027 demands at each stage as a bundle, and model the process of the broker allocating resources to all arriving users as a Markov Decision Process. We then use the Qlearning algorithm to determine how much resources will be allocated to the bundle of users arriving at the current stage. Next, we use dynamic programming to decide which cloud user will obtain the resources. Finally, we run experiments in the artificial dataset and realistic dataset respectively to evaluate our resource allocation algorithm against other typical resource allocation algorithms, and show that our algorithm can beat other algorithms, especially in the setting of the broker having extremely limited resources.

When Learning Joins Edge: RealTime Proportional Computation Offloading via Deep Reinforcement Learning Computation offloading makes sense to the interaction between users and computeintensive applications. Current researches focused on deciding locally or remotely executing an application, but ignored the specific offloading proportion of application. A full offloading cannot make the best use of client and server resources. In this paper, we propose an innovative reinforcement learning (RL) method to solve the proportional computation problem. We consider a common offloading scenario with timevariant bandwidth and heterogeneous devices, and the device generates applications constantly. For each application, the client has to choose locally or remotely executing this application, and determines the proportion to be offloaded. We formalize the problem as a longterm optimization problem, and then propose a RLbased algorithm to solve it. The basic idea is to estimate the benefit of posible decisions, of wihch the decision with the maximum benefit is selected. Instead of adopting the original Deep Q Network (DQN), we propose Advanced DQN (ADQN) by adding Priority Buffer Mechanism and Expert Buffer Mechanism, which improves the utilization of samples and overcomes the cold start problem, respectively. The experimental results show ADQNu0027s high feasibility and efficiency compared with several traditional policies, such as None Offloading Policy, Random Offloading Policy, Link Capacity Optimal Policy, and Computing Capability Optimal Policy. At last, we analyse the effect of expert buffer size and learning rate on ADQNu0027s performance.