# Resource Scheduling and Energy Cooperation in HetNet with Cross-Layer Interference Constraints

Authors: Fanrong Kong; Yan Huang
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Facing more and more severe global warming problems, renewable energy, as an alternative to traditional fossil fuels, is attracting more and more attentions due to its capability of reducing carbon emission. This paper considers two-tier HetNets with orthogonal frequency division multiple access (OFDMA), where the macro base station (MBS) is powered by power grid and small base stations (SBSs) have hybrid energy supplies. Through smart grid, SBSs can share their renewable energy with each other. We consider the problem of cross-layer interference caused by spectrum reuse, the burst of user data, and the randomness of renewable energy arrivals. Through energy cooperation, this paper investigates maximizing the time-average energy efficiency of SBSs. Based on user data queue and SBS energy queue, the optimal problem is decoupled into two subproblems by Lyapunov optimization: resource allocation subproblem and energy scheduling and energy cooperation subproblem. By solving two subproblems, the online solution to the optimization problem is obtained. Through theoretical analysis and simulations, both user data queues and energy queues have an upper bound, the network is stable, and the proposed algorithm performs better than the basic algorithm without energy cooperation.


---

## Body

## 1. Introduction

With the dramatic increase of smart devices, a high burden will be posed on the fifth-generation (5G) wireless networks, which are expected to provide user equipment (UE) with ubiquitous service. To achieve higher capacity and better coverage, more base stations (BSs) are deployed, which means higher energy consumption. As claimed in [1], energy consumption of information and communication technology (ICT) occupies 10% of world’s total energy consumption. Concerned about the environment and energy saving, operators are exploring energy harvesting technology swhich is able to harvest renewable energy source (RES) by solar panels and wind turbines.It is widely known that, in order to meet the requirements of 5G wireless networks, the heterogeneous network (HetNet), where various small cells are densely underlaid in a macro cellular network, is a promising technique to achieve more spectrum-efficient and energy-efficient communications. HetNets significantly boost the overall spectral efficiency and energy efficiency through a full spatial resource reuse. However, the severe intralayer interference among dense macro base stations (MBSs) and cross-layer interference between small cell base stations (SBSs) and MBSs are challenging the successful commercial operations of HetNets. In recent years, a great emphasis has been given on advanced techniques that take interference control, resource allocation, and self-organization into account to enhance both the spectral efficiency and energy efficiency [2].On the other hand, the rapid developments in energy harvesting technologies have made the dream of self-sustaining devices and BSs potentially possible. As such, energy harvesting is highly desirable for prolonging the battery life and improving the energy efficiency of networks, especially for HetNets with low-power SBSs [3, 4]. Reference [5] summarized recent contributions in energy harvesting wireless communications and, reference [6] presented an overview of the past and recent developments in energy harvesting wireless communications and highlighted a number of possible future research directions. Particularly, for energy supply by RES, some works considered the solely RES powered communication networks [7–9]. However, owing to the dynamics of renewable energy generation and limited capacity of energy storage, sole RES may not guarantee enough power supplies for all kinds of BSs. Hence, HetNets are more likely to adopt hybrid energy supplies with both traditional power grid and RES [3–5], and the works in [10–13] focused on resource allocation in hybrid energy powered cellular networks, where the power grid is needed to provide reliable energy to BSs.Then it is noted that, with its ongoing paradigm shift from traditional grid to smart grid, the power grid also provides new opportunities for BSs’ cost saving. Since smart grid enables both two-way information and energy flows between the grid and end users, the energy cooperation in cellular networks which allows the BSs to trade and share their unevenly harvested energy to support nonuniform wireless traffic in a cost-effective way is emerging [14]. Reference [15] studied energy group buying with load sharing, where two network operators are aggregated as a single group to make the day-ahead and real-time energy purchase and their BSs share the wireless traffic at the same time. Reference [16] proposed a hybrid energy sharing framework for cellular network, where a combination of physical power lines and energy trading with other BSs using smart grid is used. In [17], to minimize the on-grid power consumption, the BS operation and the power distribution were jointly optimized. Reference [18] presented a contemporary review of recent breakthroughs on the utilization, redistribution, trading, and planning of energy harvested in future wireless networks interoperating with smart grids. In a word, the above works in [14–17] on energy cooperation and resource allocation emphasized on energy planning and/or energy trading through smart grid, while the traffic coordination or resource allocation was performed on a large timescale.In this work, we focus on the small-timescale resource scheduling and energy cooperation in a HetNet powered by hybrid energy. Particularly, the random renewable energy arrivals, the inherent cross-layer interference constraint, and the burst of user data in the two-tier HetNet are considered. With the objective of maximizing the time-average energy efficiency of SBSs, the transmission power allocation, the subcarrier assignment, the energy scheduling, and energy cooperation among SBSs are jointly optimized. The main contributions of this paper are summarized as follows:(1)
Firstly, considering a two-tier downlink HetNet with hybrid energy supplies, we jointly optimize the resource allocation, energy scheduling, and energy cooperation with the burst of user data, the cross-layer interference constraints, and the random renewable energy arrivals.(2)
Secondly, a stochastic optimization problem that maximizes the time-average energy efficiency is modelled for tradeoff between user throughput improvement and on-grid energy consumption reduction. Leveraging Lyapunov optimization technique, an online algorithm that decides transmission power allocation, subcarrier assignment, discharging and charging of the storage devices, energy cooperation among SBSs, and on-grid energy consumption is developed after several decompositions.(3)
Thirdly, both theoretical analysis and simulation results are presented to show the stability and superiority of the proposed online resource scheduling and energy cooperation algorithm.The rest of this paper is organized as follows. In Section2, we illustrate the system model that contains network model, traffic model, data queue model, energy supply and demand model, and energy queue model. Section 3 shows the formulation of the stochastic optimization problem and its transformation based on Lyapunov optimization method. An online algorithm for transmission power allocation, subcarrier assignment, energy scheduling, and energy cooperation is developed in Section 4. Performance analysis is given in Section 5. Section 6 presents simulation results and discussions. Finally, Section 7 concludes the whole work.


## 2. System Model

### 2.1. Network Model

We consider the downlink transmission of an OFDMA two-tier HetNet system as shown in Figure1. MBS of the first tier provides basic coverage and SBSs of the second tier enhance the capacity of the whole network. In this system, MBS is powered by power grid, while SBSs are powered by hybrid energy supply containing both power grid and RES. SBSs are connected through smart grid and they can exchange renewable energy through smart grid, so that the unbalanced renewable energy sources can be effectively utilized to match with the uneven user data.Figure 1
Network structure.In the system, one MBS with M active MUEs corresponds toK SBSs, and each SBS has Sk active SUEs. Here, MUEs, SBSs, and SUEs are denoted as ℳ=1,2,⋯,M indexed by m, K=1,2,⋯,K indexed by k, and S=S1,S2,⋯,Sk indexed by sk, respectively. All SBSs only provide service to the preregistered SUEs as SBSs have a closed access. We assume that all SBSs are connected to the MBS via high speed backhaul with negligible delay. To match with the real-world scenario of suburban areas, we consider sparse deployed small cells. So we could ignore the co-tier interference between SBSs.As wireless channels and renewable energy arrivals change dynamically, the transmission between BSs and UEs could be regarded as slot-based transmission. Meanwhile, during each time slot, the wireless channels and renewable energy arrivals remain stable. We consider the problem of a fixed scheduling time containingT time slot, denoted as T=1,2,⋯,T.


### 2.2. Traffic Model

This paper considers the downlink transmission of a two-tier HetNet with orthogonal frequency division multiple access (OFDMA). In the network, the total bandwidthB is equally divided into N subcarriers, denoted as N=1,2,⋯,N. To avoid intracell interference, at time slot t, subcarrier n can be allocated to zero or one SUE in each small cell. The allocation policy of subcarrier n at time slot t is denoted as Xk,sk,nt, whose value represents that subcarrier n is allocated to SUE sk in small cell k if it equals 1 and otherwise if it equals 0. Then, we have(1)∑sk=1SkXk,sk,nt≤1,∀k,n,t.Pk,sk,nSt is used to denote the power allocated by SBS k to SUE sk on subcarrier n. Limited by hardware, each SBS k has its energy consumption upper bound, denoted as Pkmax. Depending on subcarrier allocation and power allocation, the total energy consumption Pkt of SBS k at time slot t cannot exceed the aforementioned upper bound. This restriction is denoted as(2)Pkt=∑sk=1Sk∑n=1NXk,sk,ntPk,sk,nSt+Pkc≤Pkmax,where Pkc is the static energy consumption of SBS k related to cooling and baseband processing.For wireless channel,hk,sk,nSt denotes the channel gain of SUE sk from SBS k on subcarrier n, hk,m,nSMt denotes the interference channel gain of MUE m served by MBS from small cell k on subcarrier n, and hk,sk,nMSt denotes the interference channel gain of SUE sk in small cell k from MBS on subcarrier n. Generally, the channel gain contains two parts: one is the large-scale fading component with path loss and shadowing, and the other is the small-scale fading component with multipath Rayleigh fading. Pm,nMt is used to denote the power allocated by MBS to MUE m on subcarrier n at time slot t. So the received SINR of SUE sk served by SBS k and occupying subcarrier n at time slot t is denoted as(3)SINRk,sk,nSt=Pk,sk,nSthk,sk,nStPm,nMthk,sk,nMSt+δ2,where δ2 is the background noise.Meanwhile, the achievable rate on subcarriern of SUE sk served by SBS k at time slot t is defined as(4)Rk,sk,nSt=Xk,sk,ntlog21+SINRk,sk,nSt.To reuse subcarriern of MBS to transmit information, the subcarrier n has a tolerable interference level In, meaning that the interference on subcarrier n cannot exceed the interference threshold. Then, we have(5)∑k=1K∑sk=1SkXk,sk,ntPk,sk,nSthk,m,nSMt≤In.


### 2.3. Data Queue Model

Each SBS has an independent buffer for every one of its preregistered SUEs. The arriving data is first stored in the buffer and then transmitted to the corresponding SUE. We useAk,skt to denote the arriving data for SUE sk at time slot t, where SUE sk is severed by SBS k. The arriving data of SUE sk in SBS k can be modelled as an independently and identically distributed (i.i.d) process with a mean of EAk,skt=λk,sk. At each time slot, the arriving data of SUE sk cannot exceed the maximum arriving data Ak,skmax.LetJk,skt denote the data backlog of SUE sk in SBS k at time slot t. Based on the admitted data rate and service rate of each SUE, the data backlog of SUE sk in SBS k at time slot t can be updated as(6)Jk,skt+1=Jk,skt−Rk,skt++Ak,skt,where x+=maxx,0. We first give the definition of queue stability.Definition 1.
(see [19]). When a discrete-time process Lt+1=Lt−Rt++At, it is strongly stable if it satisfies(7)limsupT⟶+∞1T∑t=0T−1ELt<∞.
Depending on rate stability theorem [20], the queue Lt is rate-stable when a≤r holds, where a=limT⟶+∞1/T∑t=0T−1At and r=limT⟶+∞1/T∑t=0T−1Rt. Moreover, when all individual queues of the network are strongly stable, a network is strongly stable.
According to the stability theorem, the queueJk,skt is stable if A¯k,sk≤R¯k,sk holds, where A¯k,sk=limT⟶+∞1/T∑t=0T−1EAk,skt and R¯k,sk=limT⟶+∞1/T∑t=0T−1ERk,skt.


### 2.4. Energy Supply and Demand Model

We useEkt to represent the renewable energy collected by SBS k at time slot t, which is based on the i.i.d stochastic process. It should be noted that there is no correlation between renewable energy generations among SBSs. In this paper, the collected renewable energy is firstly used by the SBS and then the remaining energy is stored in the storage device. At time slot t, there are two cases considering the relationship between the energy consumption Pkt of SBS k and the renewable energy arrival Ekt of SBS k.(a):
Ekt<PktAt time slott, the renewable energy collected by SBS k cannot satisfy its energy consumption. For the insufficient part, it is denoted as P˜kt=Pkt−Ekt+, which can be satisfied by three ways:(1)
Obtained from its storage device.At time slott, SBS k gets Ok,kt from its storage device for the insufficient part of renewable energy.(2)
Energy cooperation among SBSs.Due to the energy cooperation between SBSs, SBSk can use the surplus renewable energy of SBS k′, which is denoted by Ok′,kt,k′≠k. The energy taken from other SBSs has upper bounds, and the maximum energy available from SBS k′ is Ok′,kmax,e<∞.(3)
Getting from the power grid.When the renewable energy getting from its storage devices and other SBSs cannot meet the energy demand, the SBSk can get energy from the power grid. The amount of energy getting from power gird is expressed as Gkt.By the above three ways, the insufficient energy of SBSk can be satisfied, which means(8)Ok,kt+∑k′≠kOk′,kt+Gkt=P˜kt,∀t,k.(b):
Ekt≥PktAt time slott, the renewable energy collected by SBS k can satisfy or exceed its energy consumption. The renewable energy collected is first used to meet its energy consumption. For the remaining renewable energy, which is denoted as E˜kt=Ekt−Pkt+, there are two applications:(1)
Sharing with other SBSs.The renewable energy sharing from SBSk to SBS k′ at time slot t is denoted by Ok,k′t,k′≠k.(2)
Stored in its storage device.At time slott, the stored energy of SBS k is denoted as Ykt≤Ykmax, where Ykmax is the maximum storage renewable energy and Ykmax<∞. Thus, at time slot t, we have(9)Ykt+∑k′≠kOk,k′t≤E˜kt,∀t,k.


### 2.5. Energy Queue Model

At each time slott, the remaining renewable energy of the storage device of SBS k is denoted by Fkt. Since the energy changes of the storage device at each time slot are mainly the charging process and the discharging process, we can describe this as an energy queue whose updating process is(10)Fkt+1=Fkt−Ok,kt+Ykt=Fkt−P˜kt−∑k′≠kOk′,kt−Gkt+Ykt.At each time slott, the discharging of storage device can neither be more than its remaining energy Fkt nor exceed the maximum discharging limit Okmax. Therefore, we get the following constraint:(11)Ok,kt≤minFkt,Okmax,∀t,k.Similarly, at each time slott, the charging of storage device can neither exceed the remaining battery capacity E˜kt nor exceed the maximum charging limit Ykmax. Thus, we have(12)Ykt≤minE˜kt,Fkmax−Fkt,Ykmax.According to (10), by taking iterated expectation and using telescoping sums over t=0,1,…,T−1, we get(13)EFkT−EFk0=∑t=0T−1E−Ok,kt+Ykt.We further divideT on both sides of (13), where T⟶+∞. According to the stability theory, the energy queue Fkt is stable due to 0≤Fkt≤Fkmax and O¯k,k=Y¯k, where O¯k,k=limT⟶+∞1/T∑t=0T−1EOk,kt is the time-average discharging of SBS k and Y¯k=limT⟶+∞1/T∑t=0T−1EYkt is the time-average charging of SBS k.For clarity, the important notations of this paper are listed in Table1.Table 1
Notations.
NotationMeaningKNumber of SBSsM,SkNumber of MUEs in MBS and number of SUEs in each SBSBTotal bandwidth in systemk,sk,nIndices of SBS, SUE, and subcarrierXk,sk,ntBinary indication of the allocation of subcarriern to whether allocated to SUE sk in SBS k at time slot tPk,sk,nStThe power allocated by SBSk to its SUE sk on subcarrier n at time slot tPm,nMtThe power allocated by MBS to MUEm on subcarrier n at time slot thk,sk,nStThe channel gain of SUEsk from SBS k on subcarrier n at time slot thm,nMtThe channel gain of MUE from MBS on subcarriern at time slot thk,m,nSMtThe interference channel gain of MUEm from SBS k on subcarrier n at time slot thk,sk,nMStThe interference channel gain of SUEsk in SBS k from MBS on subcarrier n at time slot tSINRk,sk,nStThe received SINR of SUEsk in SBS k on subcarrier n at time slot tInThe maximum tolerable interference by MUEs when occupying subcarriernRk,sk,nStThe achievable rate of SUEsk in SBS k on subcarrier n at time slot tPktThe total transmitted power of SBSk at time slot tPkcThe static power consumption of SBSk at time slot tPkmaxThe maximum energy consumption of SBSkJk,sktThe data queue length of SUEsk in SBS k at time slot tFktThe energy queue length of SBSk at time slot tEktArriving RES of SBSk at time slot tAk,sktArriving data of SBSk at time slot tOk,ktThe supplied energy of SBSk from storage device at time slot tOk,k′tThe shared energy of SBSk to SBS k′ at time slot tGktThe supplied energy of SBSk from power grid at time slot tP˜ktThe shortage RES of SBSk at time slot tE˜ktThe extra RES of SBSk at time slot tYktThe charging power of SBSk at time slot t


## 2.1. Network Model

We consider the downlink transmission of an OFDMA two-tier HetNet system as shown in Figure1. MBS of the first tier provides basic coverage and SBSs of the second tier enhance the capacity of the whole network. In this system, MBS is powered by power grid, while SBSs are powered by hybrid energy supply containing both power grid and RES. SBSs are connected through smart grid and they can exchange renewable energy through smart grid, so that the unbalanced renewable energy sources can be effectively utilized to match with the uneven user data.Figure 1
Network structure.In the system, one MBS with M active MUEs corresponds toK SBSs, and each SBS has Sk active SUEs. Here, MUEs, SBSs, and SUEs are denoted as ℳ=1,2,⋯,M indexed by m, K=1,2,⋯,K indexed by k, and S=S1,S2,⋯,Sk indexed by sk, respectively. All SBSs only provide service to the preregistered SUEs as SBSs have a closed access. We assume that all SBSs are connected to the MBS via high speed backhaul with negligible delay. To match with the real-world scenario of suburban areas, we consider sparse deployed small cells. So we could ignore the co-tier interference between SBSs.As wireless channels and renewable energy arrivals change dynamically, the transmission between BSs and UEs could be regarded as slot-based transmission. Meanwhile, during each time slot, the wireless channels and renewable energy arrivals remain stable. We consider the problem of a fixed scheduling time containingT time slot, denoted as T=1,2,⋯,T.


## 2.2. Traffic Model

This paper considers the downlink transmission of a two-tier HetNet with orthogonal frequency division multiple access (OFDMA). In the network, the total bandwidthB is equally divided into N subcarriers, denoted as N=1,2,⋯,N. To avoid intracell interference, at time slot t, subcarrier n can be allocated to zero or one SUE in each small cell. The allocation policy of subcarrier n at time slot t is denoted as Xk,sk,nt, whose value represents that subcarrier n is allocated to SUE sk in small cell k if it equals 1 and otherwise if it equals 0. Then, we have(1)∑sk=1SkXk,sk,nt≤1,∀k,n,t.Pk,sk,nSt is used to denote the power allocated by SBS k to SUE sk on subcarrier n. Limited by hardware, each SBS k has its energy consumption upper bound, denoted as Pkmax. Depending on subcarrier allocation and power allocation, the total energy consumption Pkt of SBS k at time slot t cannot exceed the aforementioned upper bound. This restriction is denoted as(2)Pkt=∑sk=1Sk∑n=1NXk,sk,ntPk,sk,nSt+Pkc≤Pkmax,where Pkc is the static energy consumption of SBS k related to cooling and baseband processing.For wireless channel,hk,sk,nSt denotes the channel gain of SUE sk from SBS k on subcarrier n, hk,m,nSMt denotes the interference channel gain of MUE m served by MBS from small cell k on subcarrier n, and hk,sk,nMSt denotes the interference channel gain of SUE sk in small cell k from MBS on subcarrier n. Generally, the channel gain contains two parts: one is the large-scale fading component with path loss and shadowing, and the other is the small-scale fading component with multipath Rayleigh fading. Pm,nMt is used to denote the power allocated by MBS to MUE m on subcarrier n at time slot t. So the received SINR of SUE sk served by SBS k and occupying subcarrier n at time slot t is denoted as(3)SINRk,sk,nSt=Pk,sk,nSthk,sk,nStPm,nMthk,sk,nMSt+δ2,where δ2 is the background noise.Meanwhile, the achievable rate on subcarriern of SUE sk served by SBS k at time slot t is defined as(4)Rk,sk,nSt=Xk,sk,ntlog21+SINRk,sk,nSt.To reuse subcarriern of MBS to transmit information, the subcarrier n has a tolerable interference level In, meaning that the interference on subcarrier n cannot exceed the interference threshold. Then, we have(5)∑k=1K∑sk=1SkXk,sk,ntPk,sk,nSthk,m,nSMt≤In.


## 2.3. Data Queue Model

Each SBS has an independent buffer for every one of its preregistered SUEs. The arriving data is first stored in the buffer and then transmitted to the corresponding SUE. We useAk,skt to denote the arriving data for SUE sk at time slot t, where SUE sk is severed by SBS k. The arriving data of SUE sk in SBS k can be modelled as an independently and identically distributed (i.i.d) process with a mean of EAk,skt=λk,sk. At each time slot, the arriving data of SUE sk cannot exceed the maximum arriving data Ak,skmax.LetJk,skt denote the data backlog of SUE sk in SBS k at time slot t. Based on the admitted data rate and service rate of each SUE, the data backlog of SUE sk in SBS k at time slot t can be updated as(6)Jk,skt+1=Jk,skt−Rk,skt++Ak,skt,where x+=maxx,0. We first give the definition of queue stability.Definition 1.
(see [19]). When a discrete-time process Lt+1=Lt−Rt++At, it is strongly stable if it satisfies(7)limsupT⟶+∞1T∑t=0T−1ELt<∞.
Depending on rate stability theorem [20], the queue Lt is rate-stable when a≤r holds, where a=limT⟶+∞1/T∑t=0T−1At and r=limT⟶+∞1/T∑t=0T−1Rt. Moreover, when all individual queues of the network are strongly stable, a network is strongly stable.
According to the stability theorem, the queueJk,skt is stable if A¯k,sk≤R¯k,sk holds, where A¯k,sk=limT⟶+∞1/T∑t=0T−1EAk,skt and R¯k,sk=limT⟶+∞1/T∑t=0T−1ERk,skt.


## 2.4. Energy Supply and Demand Model

We useEkt to represent the renewable energy collected by SBS k at time slot t, which is based on the i.i.d stochastic process. It should be noted that there is no correlation between renewable energy generations among SBSs. In this paper, the collected renewable energy is firstly used by the SBS and then the remaining energy is stored in the storage device. At time slot t, there are two cases considering the relationship between the energy consumption Pkt of SBS k and the renewable energy arrival Ekt of SBS k.(a):
Ekt<PktAt time slott, the renewable energy collected by SBS k cannot satisfy its energy consumption. For the insufficient part, it is denoted as P˜kt=Pkt−Ekt+, which can be satisfied by three ways:(1)
Obtained from its storage device.At time slott, SBS k gets Ok,kt from its storage device for the insufficient part of renewable energy.(2)
Energy cooperation among SBSs.Due to the energy cooperation between SBSs, SBSk can use the surplus renewable energy of SBS k′, which is denoted by Ok′,kt,k′≠k. The energy taken from other SBSs has upper bounds, and the maximum energy available from SBS k′ is Ok′,kmax,e<∞.(3)
Getting from the power grid.When the renewable energy getting from its storage devices and other SBSs cannot meet the energy demand, the SBSk can get energy from the power grid. The amount of energy getting from power gird is expressed as Gkt.By the above three ways, the insufficient energy of SBSk can be satisfied, which means(8)Ok,kt+∑k′≠kOk′,kt+Gkt=P˜kt,∀t,k.(b):
Ekt≥PktAt time slott, the renewable energy collected by SBS k can satisfy or exceed its energy consumption. The renewable energy collected is first used to meet its energy consumption. For the remaining renewable energy, which is denoted as E˜kt=Ekt−Pkt+, there are two applications:(1)
Sharing with other SBSs.The renewable energy sharing from SBSk to SBS k′ at time slot t is denoted by Ok,k′t,k′≠k.(2)
Stored in its storage device.At time slott, the stored energy of SBS k is denoted as Ykt≤Ykmax, where Ykmax is the maximum storage renewable energy and Ykmax<∞. Thus, at time slot t, we have(9)Ykt+∑k′≠kOk,k′t≤E˜kt,∀t,k.


## 2.5. Energy Queue Model

At each time slott, the remaining renewable energy of the storage device of SBS k is denoted by Fkt. Since the energy changes of the storage device at each time slot are mainly the charging process and the discharging process, we can describe this as an energy queue whose updating process is(10)Fkt+1=Fkt−Ok,kt+Ykt=Fkt−P˜kt−∑k′≠kOk′,kt−Gkt+Ykt.At each time slott, the discharging of storage device can neither be more than its remaining energy Fkt nor exceed the maximum discharging limit Okmax. Therefore, we get the following constraint:(11)Ok,kt≤minFkt,Okmax,∀t,k.Similarly, at each time slott, the charging of storage device can neither exceed the remaining battery capacity E˜kt nor exceed the maximum charging limit Ykmax. Thus, we have(12)Ykt≤minE˜kt,Fkmax−Fkt,Ykmax.According to (10), by taking iterated expectation and using telescoping sums over t=0,1,…,T−1, we get(13)EFkT−EFk0=∑t=0T−1E−Ok,kt+Ykt.We further divideT on both sides of (13), where T⟶+∞. According to the stability theory, the energy queue Fkt is stable due to 0≤Fkt≤Fkmax and O¯k,k=Y¯k, where O¯k,k=limT⟶+∞1/T∑t=0T−1EOk,kt is the time-average discharging of SBS k and Y¯k=limT⟶+∞1/T∑t=0T−1EYkt is the time-average charging of SBS k.For clarity, the important notations of this paper are listed in Table1.Table 1
Notations.
NotationMeaningKNumber of SBSsM,SkNumber of MUEs in MBS and number of SUEs in each SBSBTotal bandwidth in systemk,sk,nIndices of SBS, SUE, and subcarrierXk,sk,ntBinary indication of the allocation of subcarriern to whether allocated to SUE sk in SBS k at time slot tPk,sk,nStThe power allocated by SBSk to its SUE sk on subcarrier n at time slot tPm,nMtThe power allocated by MBS to MUEm on subcarrier n at time slot thk,sk,nStThe channel gain of SUEsk from SBS k on subcarrier n at time slot thm,nMtThe channel gain of MUE from MBS on subcarriern at time slot thk,m,nSMtThe interference channel gain of MUEm from SBS k on subcarrier n at time slot thk,sk,nMStThe interference channel gain of SUEsk in SBS k from MBS on subcarrier n at time slot tSINRk,sk,nStThe received SINR of SUEsk in SBS k on subcarrier n at time slot tInThe maximum tolerable interference by MUEs when occupying subcarriernRk,sk,nStThe achievable rate of SUEsk in SBS k on subcarrier n at time slot tPktThe total transmitted power of SBSk at time slot tPkcThe static power consumption of SBSk at time slot tPkmaxThe maximum energy consumption of SBSkJk,sktThe data queue length of SUEsk in SBS k at time slot tFktThe energy queue length of SBSk at time slot tEktArriving RES of SBSk at time slot tAk,sktArriving data of SBSk at time slot tOk,ktThe supplied energy of SBSk from storage device at time slot tOk,k′tThe shared energy of SBSk to SBS k′ at time slot tGktThe supplied energy of SBSk from power grid at time slot tP˜ktThe shortage RES of SBSk at time slot tE˜ktThe extra RES of SBSk at time slot tYktThe charging power of SBSk at time slot t


## 3. Problem Formulation and Transformation

### 3.1. Problem Formulation

Taking into account the random arrivals of each SUE’s traffic data and each SBS’s renewable energy, our optimization goal is to minimize time-average on-grid energy consumption while maximizing the time-average throughput of all SUEs. At the same time, it is also needed to meet the network stability constraints, the subcarrier allocation constraints, and the cross-layer interference constraints. Then, we formulate the following optimization problem:(14)P1:maxϕ∑k=1K∑sk=1SkR¯k,sk−ψ∑k=1KG¯k,s.t.C1:∑sk=1SkXk,sk,nt≤1∀k,n,t,C2:Xk,sk,nt∈0,1,∀k,sk,n,t,C3:0≤∑sk=1Sk∑n=1NXk,sk,ntPk,sk,nSt+Pkc≤Pkmax,∀k,t,C4:∑k=1K∑sk=1SkXk,sk,ntPk,sk,nSthk,m,nSMt≤In,∀n,t,C5:QueuesJk,sktandFktare stable,∀k,sk,t,C6:Ok,kt+∑k′≠kOk′,kt+Gkt=P˜kt,∀k,t,C7:Ykt+∑k′≠kOk,k′t≤E˜kt,∀k,t,C8:Ok,kt≤minFkt,Okmax,∀k,t,C9:Ykt≤minFkmax−Fkt,Ykmax,∀k,t,C10:Ok′,kt≤Ok′max,e,E˜kt,∀k,k′,t,where(15)R¯k,sk=limT⟶+∞1T∑t=0T−1EAk,skt,and it is SUE sk’s time-average service rate, and(16)G¯k=limT⟶+∞1T∑t=0T−1EGkt,and it is SBS k’s time-average on-grid energy consumption.We useι1 to denote the optimal objective value of P1. In P1, the control variables Xk,sk,nt, Pk,sk,nSt, Ok,kt, and Ok′,kt denote the subcarrier assignment, transmission power control on each subcarrier, SBS’s discharge control, and energy sharing, respectively. ϕ and ψ are nonnegative constants, which are used to balance the throughput and on-grid energy consumption. C1 indicates that one subcarrier can be allocated to at most one SUE in one SBS. C2 indicates an index of subcarrier allocation. C3 is the transmission power consumption constraint of SBSs. Constraint C4 is the tolerable cross-layer interference for each subcarrier occupied by SUEs. C5 is the stability constraint of data and energy queues. C6 is the compensation component of the insufficient renewable energy. C7 is the allocation of extra renewable energy for each SBS. C8 and C9 are discharging and charging constraints of each SBS, respectively. C10 is the renewable energy sharing constraint.


### 3.2. Problem Transformation

It is observed that, in P1, the decision variablesXk,sk,nt, Pk,sk,nSt, Ok,kt, and Ok′,kt are coupled together. We decompose the optimization problem P1 into joint resource allocation and energy scheduling by the Lyapunov optimization method in each time slot. The design principle of Lyapunov optimization method is to minimize the upper bound of Lyapunov drift minus objective function greedily without knowing the profile of stochastic events.Ft=Fkt and Jt=Jk,skt are used to represent the energy queue and the data queue, respectively. Then, we use Qt=Ft,Jt to denote all queues in the network system. At each slot t, we define the Lyapunov function as(17)ℒQt=12∑k=1k∑sk=1SkJk,skt2+12∑k=1kFkt−θ2,where θ is a perturbation factor, which ensures that there is enough energy in the energy queue. Without loss of generality, we assume that all queues are empty when t=0 such that LQt=0.The Lyapunov functionLQt is a scalar measure of network congestion. Intuitively, if LQt is small, then all queues are small; and if LQt is large, then at least one queue is large. Thus, by minimizing the Lyapunov drift (i.e., by minimizing a difference in the Lyapunov function from one time slot to the next), the data queues and energy queues can be stabilized.By using expression in (17), the Lyapunov drift can be written as(18)ΔℒQt=EℒQt+1−ℒQtQt,and, to solve optimization problem P1, we now use the drift-plus-penalty minimization method [21], where a control policy that solves P1 is obtained by minimizing the upper bound of the following drift-plus-penalty term:(19)ΔVt=ΔℒQt−VEϕ∑k=1K∑sk=1SkRk,skt−ψ∑k=1KGktΘt︸Penalty term,where V is a nonnegative control variable that represents the relative importance of minimizing the queue length and maximizing the objective function. According to [21], when V is sufficiently large, the optimization algorithm approaches the optimal policy.The upper bound of the drift-plus-penalty can be derived as follows.Lemma 1.
The upper bound of drift-plus-penalty in (19) is(20)ΔℒQt−VEϕ∑k=1K∑sk=1SkRk,skt−ψ∑k=1KGktQt≤Ξ−∑k=1K∑sk=1SkEVϕ+Jk,sktRk,sktQt+∑k=1K∑sk=1SkEJk,sktDk,skmaxQt+∑k=1KEVψGkt−Fkt−θP˜kt−∑k′≠kOk′,kt−Gkt−YktQt,=Ξ−∑k=1KE∑sk=1SkVϕ+Jk,sktRk,skt+Fkt−θPkt∗1Pkt≥EktQt+∑k=1K∑sk=1SkEJk,sktDk,skmaxQt+∑k=1KEFkt−θ+VψGkt∗1Pkt≥EktQt+∑k=1KEFkt−θEkt∗1Pkt≥EktQt+∑k=1KEFkt−θ∑k′≠kOk′,kt∗1Pkt≥EktQt+∑k=1KEFkt−θYkt∗1Pkt<EktQt,where Ξ is a positive constant that satisfies the following inequality constraint for all time slots:(21)Ξ≥12∑k=1K∑sk=1SkDk,skmax2+∑k=1K∑sk=1SkRk,sk2t+∑k=1KOkmax2.
he proof of Lemma1 is shown in Appendix A.
By Lemma1, we can transform the optimization problem P1 into minimizing the right-side term of (20) at each time slot t. Thus, we will decompose the optimization problem P1 and present an online dynamic control algorithm for the resource allocation and the energy scheduling and energy cooperation, respectively.


## 3.1. Problem Formulation

Taking into account the random arrivals of each SUE’s traffic data and each SBS’s renewable energy, our optimization goal is to minimize time-average on-grid energy consumption while maximizing the time-average throughput of all SUEs. At the same time, it is also needed to meet the network stability constraints, the subcarrier allocation constraints, and the cross-layer interference constraints. Then, we formulate the following optimization problem:(14)P1:maxϕ∑k=1K∑sk=1SkR¯k,sk−ψ∑k=1KG¯k,s.t.C1:∑sk=1SkXk,sk,nt≤1∀k,n,t,C2:Xk,sk,nt∈0,1,∀k,sk,n,t,C3:0≤∑sk=1Sk∑n=1NXk,sk,ntPk,sk,nSt+Pkc≤Pkmax,∀k,t,C4:∑k=1K∑sk=1SkXk,sk,ntPk,sk,nSthk,m,nSMt≤In,∀n,t,C5:QueuesJk,sktandFktare stable,∀k,sk,t,C6:Ok,kt+∑k′≠kOk′,kt+Gkt=P˜kt,∀k,t,C7:Ykt+∑k′≠kOk,k′t≤E˜kt,∀k,t,C8:Ok,kt≤minFkt,Okmax,∀k,t,C9:Ykt≤minFkmax−Fkt,Ykmax,∀k,t,C10:Ok′,kt≤Ok′max,e,E˜kt,∀k,k′,t,where(15)R¯k,sk=limT⟶+∞1T∑t=0T−1EAk,skt,and it is SUE sk’s time-average service rate, and(16)G¯k=limT⟶+∞1T∑t=0T−1EGkt,and it is SBS k’s time-average on-grid energy consumption.We useι1 to denote the optimal objective value of P1. In P1, the control variables Xk,sk,nt, Pk,sk,nSt, Ok,kt, and Ok′,kt denote the subcarrier assignment, transmission power control on each subcarrier, SBS’s discharge control, and energy sharing, respectively. ϕ and ψ are nonnegative constants, which are used to balance the throughput and on-grid energy consumption. C1 indicates that one subcarrier can be allocated to at most one SUE in one SBS. C2 indicates an index of subcarrier allocation. C3 is the transmission power consumption constraint of SBSs. Constraint C4 is the tolerable cross-layer interference for each subcarrier occupied by SUEs. C5 is the stability constraint of data and energy queues. C6 is the compensation component of the insufficient renewable energy. C7 is the allocation of extra renewable energy for each SBS. C8 and C9 are discharging and charging constraints of each SBS, respectively. C10 is the renewable energy sharing constraint.


## 3.2. Problem Transformation

It is observed that, in P1, the decision variablesXk,sk,nt, Pk,sk,nSt, Ok,kt, and Ok′,kt are coupled together. We decompose the optimization problem P1 into joint resource allocation and energy scheduling by the Lyapunov optimization method in each time slot. The design principle of Lyapunov optimization method is to minimize the upper bound of Lyapunov drift minus objective function greedily without knowing the profile of stochastic events.Ft=Fkt and Jt=Jk,skt are used to represent the energy queue and the data queue, respectively. Then, we use Qt=Ft,Jt to denote all queues in the network system. At each slot t, we define the Lyapunov function as(17)ℒQt=12∑k=1k∑sk=1SkJk,skt2+12∑k=1kFkt−θ2,where θ is a perturbation factor, which ensures that there is enough energy in the energy queue. Without loss of generality, we assume that all queues are empty when t=0 such that LQt=0.The Lyapunov functionLQt is a scalar measure of network congestion. Intuitively, if LQt is small, then all queues are small; and if LQt is large, then at least one queue is large. Thus, by minimizing the Lyapunov drift (i.e., by minimizing a difference in the Lyapunov function from one time slot to the next), the data queues and energy queues can be stabilized.By using expression in (17), the Lyapunov drift can be written as(18)ΔℒQt=EℒQt+1−ℒQtQt,and, to solve optimization problem P1, we now use the drift-plus-penalty minimization method [21], where a control policy that solves P1 is obtained by minimizing the upper bound of the following drift-plus-penalty term:(19)ΔVt=ΔℒQt−VEϕ∑k=1K∑sk=1SkRk,skt−ψ∑k=1KGktΘt︸Penalty term,where V is a nonnegative control variable that represents the relative importance of minimizing the queue length and maximizing the objective function. According to [21], when V is sufficiently large, the optimization algorithm approaches the optimal policy.The upper bound of the drift-plus-penalty can be derived as follows.Lemma 1.
The upper bound of drift-plus-penalty in (19) is(20)ΔℒQt−VEϕ∑k=1K∑sk=1SkRk,skt−ψ∑k=1KGktQt≤Ξ−∑k=1K∑sk=1SkEVϕ+Jk,sktRk,sktQt+∑k=1K∑sk=1SkEJk,sktDk,skmaxQt+∑k=1KEVψGkt−Fkt−θP˜kt−∑k′≠kOk′,kt−Gkt−YktQt,=Ξ−∑k=1KE∑sk=1SkVϕ+Jk,sktRk,skt+Fkt−θPkt∗1Pkt≥EktQt+∑k=1K∑sk=1SkEJk,sktDk,skmaxQt+∑k=1KEFkt−θ+VψGkt∗1Pkt≥EktQt+∑k=1KEFkt−θEkt∗1Pkt≥EktQt+∑k=1KEFkt−θ∑k′≠kOk′,kt∗1Pkt≥EktQt+∑k=1KEFkt−θYkt∗1Pkt<EktQt,where Ξ is a positive constant that satisfies the following inequality constraint for all time slots:(21)Ξ≥12∑k=1K∑sk=1SkDk,skmax2+∑k=1K∑sk=1SkRk,sk2t+∑k=1KOkmax2.
he proof of Lemma1 is shown in Appendix A.
By Lemma1, we can transform the optimization problem P1 into minimizing the right-side term of (20) at each time slot t. Thus, we will decompose the optimization problem P1 and present an online dynamic control algorithm for the resource allocation and the energy scheduling and energy cooperation, respectively.


## 4. Online Solution

The proposed problem is solved online based on two subproblems. The resource allocation subproblem determines the subcarrier assignment and the transmission power allocation on each subcarrier. The energy scheduling and energy cooperation subproblem determines the discharging and charging, the energy cooperation, and on-grid power consumption of each SBS. Solving the above two subproblems in each time slot can finally determine the online resource allocation scheme and the energy scheduling and the energy cooperation scheme, so as to maximize the time-average energy efficiency of SBSs.


### 4.1. Resource Allocation Subproblem

The resource allocation subproblem jointly considers transmission power control and subcarrier allocation. At the beginning of each time slot, we can solve the resource allocation subproblem if we get the subcarrier status and queue information. The resource allocation subproblem is given below:(22)P2:maxXk,sk,nt,Pk,sk,nSt∑k=1K∑sk=1SkVϕ+Jk,sktRk,skt+∑k=1KFkt−θPkt,s.t.C1,C2,C3,C4.Problem P2 is an NP-hard mixed-integer programming problem. To solve this problem, we extend the binary variableXk,sk,nt to the continuous variable X^k,sk,nt∈0,1. At the same time, we introduce a new variable P^k,sk,nSt=X^k,sk,ntPk,sk,nSt. Ignoring the static power consumption, problem P2 can be rewritten as(23)P3:maxX^k,sk,nt,P^k,sk,nSt∑k=1K∑sk=1SkVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2+Fkt−θ∑k=1K∑sk=1Sk∑n=1NP^k,sk,nSts.t.C^1:∑sk=1SkX^k,sk,nt≤1,∀k,n,tC^2:X^k,sk,nt∈0,1,∀k,sk,n,tC^3:∑sk=1Sk∑n=1NP^k,sk,nSt+Pkc≤Pkmax,∀k,tC^4:∑k=1K∑sk=1SkP^k,sk,nSthk,m,nSMt≤In,∀n,tC^5:P^k,sk,nSt≥0,∀k,sk,n,t.Since the optimization problem P3 is a convex optimization problem, it can be solved in polynomial time. Here, we consider using the Lagrange dual method to solve problem P3. The Lagrange function is(24)LX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑k=1K∑sk=1Sk∑n=1NVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−∑k=1Kαk∑sk=1Sk∑n=1NP^k,sk,nSt+Pkc−Pkmax−∑n=1Nβn∑k=1K∑sk=1SkP^k,sk,nSthk,m,nSMt−In+∑k=1K∑n=1Nζk,n1−∑sk=1SkX^k,sk,nt+Fkt−θ∑k=1K∑sk=1Sk∑n=1NP^k,sk,nSt,where α=αk,β=βn,ζ=ζk,n are Lagrange multipliers for constraints C^3,C^4,C^1, respectively. Further, we have the dual function as follows:(25)Dα,β,ζ=maxX^k,sk,nt,P^k,sk,nStLX^k,sk,nt,P^k,sk,nSt,α,β,ζ.The dual problem is given as(26)minα,β,ζDα,β,ζs.t.α,β,ζ≥0,C^2,C^5.The dual problem (26) can be further decomposed into K×N independent subproblems, making each SBS independently decide on each subcarrier. Therefore, problem (26) is equivalent to(27)LX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑k=1K∑n=1NLk,ntX^k,sk,nt,P^k,sk,nSt,α,β,ζ+∑k=1KαkPkmax−Pkc+∑n=1NβnIn,s.t.C^2,C^5,where(28)Lk,ntX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑sk=1SkVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−∑sk=1Skαk−Fkt+θ+βnhk,m,nSMtP^k,sk,nSt+ζk,n1−∑sk=1SkX^k,sk,nt.For the solution ofLk,nt, we first introduce the Lagrange multiplier τk,sk,n for the constraint C^5. Then, we find the partial differential function of Lk,nt with respect to P^k,sk,nSt. Then, we have(29)∂Lk,n−τk,sk,nP^k,sk,nSt∂P^k,sk,nSt=X^k,sk,ntVϕ+Jk,sktln2×hk,sk,nStX^k,sk,ntZk,sk,nt+P^k,sk,nSth^k,sk,nSt−αk+Fkt−θ−βnhk,m,nSMt−τk,sk,n,1,where Zk,sk,nt=Pm,nMthk,sk,nMSt+δ2.In order to obtain the optimal transmission power control for the resource allocation subproblem, the above problem (27) must satisfy its Karush-Kuhn-Tucker (KKT) conditions, which are denoted as(30)∂Lk,n−τk,sk,nP^k,sk,nSt∂P^k,sk,nSt=0,τk,sk,n,P^k,sk,nSt=0,P^k,sk,nSt≥0,τk,sk,n≥0.Therefore, the optimalP^k,sk,nS∗t can be obtained by setting the partial differential function to be 0. Then, the specific optimal transmission power allocation is as follows:(31)P^k,sk,nS∗t=Vϕ+Jk,sktln21αk−Fkt+θ+βnhk,m,nSMt−Zk,sk,nthk,sk,nSt+×X^k,sk,nt.We use the subgradient method to update the Lagrange multipliersαk and βn, which are updated as(32)αkε+1=αkε−ρ1εPkmax−Pkc−∑sk=1Sk∑n=1NP^k,sk,nS∗t+,(33)βnε+1=βnε−ρ2εIn−∑k=1K∑sk=1SkP^k,sk,nS∗thk,m,nSMt+,where ρ1ε and ρ2ε are the step sizes of iteration ε.Based on the above optimal transmission power allocation results, we can find the optimal subcarrier assignment. Decomposing the dual problem (27) into Sk subproblems, for each subproblem, we have(34)Lk,nP=∑sk=1SkLk,sk,nP,where(35)Lk,sk,nP=Vϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nS∗thk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−αkP^k,sk,nS∗t+Fkt−θP^k,sk,nS∗t−βnP^k,sk,nS∗thk,m,nSMt−ζk,nX^k,sk,nt.Subcarriers will be assigned to SUEs with the largestLk,sk,nP, which means(36)nsk∗=argmaxskLk,sk,nP,where Lk,ns∗,nP>0.Therefore, the subcarrier assignment decision is as follows:(37)Xk,sk,n∗t=1,ifsk=nsk∗,0,otherwise.Besides, we also use the subgradient method to update the Lagrange multiplierζk,n, which is updated as(38)ζk,nε+1=ζk,nε−ρ3ε1−∑sk=1SkX^k,sk,n∗t+,where ρ3ε is the step size of iteration ε.


### 4.2. Energy Scheduling and Energy Cooperation Subproblem

The energy scheduling and energy cooperation subproblem jointly considers the control of SBS’s discharging, charging, energy cooperation, and on-grid energy consumption. At each time slott, SBS k solves the following problem based on local information for energy scheduling and energy cooperation:(39)P4:minYkt,Ok,kt,Ok,k′t∑k=1KFkt−θ+VψGkt+∑k=1KFkt−θYkt+∑k=1KFkt−θ∑k′≠kOk′,kts.t.C7,C8,C9,C10,C11:Ok,kt+∑k′≠kOk′,kt≤P˜kt.According to constraint (8) and ∑k=1K∑k′≠kOk′,kt=∑k=1K∑k′≠kOk,k′t, P4 can be further written as(40)P5:minYkt,Ok,kt,Ok,k′t∑k=1KFkt−θ+VψP˜kt−∑k=1KFkt−θ+VψOk,kt+∑k=1KFkt−θYkt−Vψ∑k=1K∑k′≠kOk,k′ts.t.C7,C8,C9,C10,C11.The optimal solution to the above problem isYk∗t, Ok,k∗t, and Ok,k′∗t. The energy Gk∗t of SBS k getting from the grid can be expressed as(41)Gk∗t=P˜kt−Ok,k∗t−∑k′≠kOk′,kt+.Since P5 is a linear programming problem, it can be solved directly by linear programming tools.


### 4.3. Online Algorithm

As presented in the above subsections, the formulated time-average delay-aware and interference-aware energy efficiency problem is solved distributively through several decompositions. Firstly, we use Lyapunov optimization method to decompose the optimal problem into two subproblems at each time slot, that is, resource allocation subproblem and energy scheduling and energy cooperation subproblem. Secondly, the resource allocation subproblem is solved through Lagrange dual decomposition. Then, the dual problem is decomposed intoK×N independent subproblems for optimal power allocation and Sk independent subproblems for optimal subcarrier assignment. Meanwhile the energy scheduling and energy cooperation subproblem is decomposed into K independent subproblems for the optimal discharging, charging, energy cooperation, and on-grid energy consumption.Based on the above solution, we further present the distributed algorithm for joint resource scheduling and energy cooperation in interference-constrained HetNets powered by hybrid energy, which is shown in Algorithm1.Algorithm 1: Resource scheduling and energy cooperation algorithm for HetNets with cross-layer interference constraints.
Require: at time slot t, get the queue states Jt, Ft, and the channel gain Ht=hk,sk,nStOutput: transmission power allocation and subcarrier assignment, energy scheduling and energy cooperationStage 1: resource allocationInitialize Lagrange multipliersα0, β0andζ0, set ν=0Repeatν=ν+1For each SBS k∈KFor each SUE s∈SFor each subcarrier n∈NAccording to (31), calculate Pk,sk,nStAccording to (35), calculate Lk,sk,ntAccording to (37), calculate Xk,sk,ntEnd ForEnd ForEnd ForAccording to (32) update αAccording to (33) update βAccording to (38) updateζUntilν≤νmax and or the convergence of the multipliers.Stage 2:Energy Scheduling and Energy CooperationSolve P5 using linear programming tools, getYk∗t,Ok,k∗t,Ok,k′∗tBased on (41), calculate Gk∗tUpdate all queues


## 4.1. Resource Allocation Subproblem

The resource allocation subproblem jointly considers transmission power control and subcarrier allocation. At the beginning of each time slot, we can solve the resource allocation subproblem if we get the subcarrier status and queue information. The resource allocation subproblem is given below:(22)P2:maxXk,sk,nt,Pk,sk,nSt∑k=1K∑sk=1SkVϕ+Jk,sktRk,skt+∑k=1KFkt−θPkt,s.t.C1,C2,C3,C4.Problem P2 is an NP-hard mixed-integer programming problem. To solve this problem, we extend the binary variableXk,sk,nt to the continuous variable X^k,sk,nt∈0,1. At the same time, we introduce a new variable P^k,sk,nSt=X^k,sk,ntPk,sk,nSt. Ignoring the static power consumption, problem P2 can be rewritten as(23)P3:maxX^k,sk,nt,P^k,sk,nSt∑k=1K∑sk=1SkVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2+Fkt−θ∑k=1K∑sk=1Sk∑n=1NP^k,sk,nSts.t.C^1:∑sk=1SkX^k,sk,nt≤1,∀k,n,tC^2:X^k,sk,nt∈0,1,∀k,sk,n,tC^3:∑sk=1Sk∑n=1NP^k,sk,nSt+Pkc≤Pkmax,∀k,tC^4:∑k=1K∑sk=1SkP^k,sk,nSthk,m,nSMt≤In,∀n,tC^5:P^k,sk,nSt≥0,∀k,sk,n,t.Since the optimization problem P3 is a convex optimization problem, it can be solved in polynomial time. Here, we consider using the Lagrange dual method to solve problem P3. The Lagrange function is(24)LX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑k=1K∑sk=1Sk∑n=1NVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−∑k=1Kαk∑sk=1Sk∑n=1NP^k,sk,nSt+Pkc−Pkmax−∑n=1Nβn∑k=1K∑sk=1SkP^k,sk,nSthk,m,nSMt−In+∑k=1K∑n=1Nζk,n1−∑sk=1SkX^k,sk,nt+Fkt−θ∑k=1K∑sk=1Sk∑n=1NP^k,sk,nSt,where α=αk,β=βn,ζ=ζk,n are Lagrange multipliers for constraints C^3,C^4,C^1, respectively. Further, we have the dual function as follows:(25)Dα,β,ζ=maxX^k,sk,nt,P^k,sk,nStLX^k,sk,nt,P^k,sk,nSt,α,β,ζ.The dual problem is given as(26)minα,β,ζDα,β,ζs.t.α,β,ζ≥0,C^2,C^5.The dual problem (26) can be further decomposed into K×N independent subproblems, making each SBS independently decide on each subcarrier. Therefore, problem (26) is equivalent to(27)LX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑k=1K∑n=1NLk,ntX^k,sk,nt,P^k,sk,nSt,α,β,ζ+∑k=1KαkPkmax−Pkc+∑n=1NβnIn,s.t.C^2,C^5,where(28)Lk,ntX^k,sk,nt,P^k,sk,nSt,α,β,ζ=∑sk=1SkVϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nSthk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−∑sk=1Skαk−Fkt+θ+βnhk,m,nSMtP^k,sk,nSt+ζk,n1−∑sk=1SkX^k,sk,nt.For the solution ofLk,nt, we first introduce the Lagrange multiplier τk,sk,n for the constraint C^5. Then, we find the partial differential function of Lk,nt with respect to P^k,sk,nSt. Then, we have(29)∂Lk,n−τk,sk,nP^k,sk,nSt∂P^k,sk,nSt=X^k,sk,ntVϕ+Jk,sktln2×hk,sk,nStX^k,sk,ntZk,sk,nt+P^k,sk,nSth^k,sk,nSt−αk+Fkt−θ−βnhk,m,nSMt−τk,sk,n,1,where Zk,sk,nt=Pm,nMthk,sk,nMSt+δ2.In order to obtain the optimal transmission power control for the resource allocation subproblem, the above problem (27) must satisfy its Karush-Kuhn-Tucker (KKT) conditions, which are denoted as(30)∂Lk,n−τk,sk,nP^k,sk,nSt∂P^k,sk,nSt=0,τk,sk,n,P^k,sk,nSt=0,P^k,sk,nSt≥0,τk,sk,n≥0.Therefore, the optimalP^k,sk,nS∗t can be obtained by setting the partial differential function to be 0. Then, the specific optimal transmission power allocation is as follows:(31)P^k,sk,nS∗t=Vϕ+Jk,sktln21αk−Fkt+θ+βnhk,m,nSMt−Zk,sk,nthk,sk,nSt+×X^k,sk,nt.We use the subgradient method to update the Lagrange multipliersαk and βn, which are updated as(32)αkε+1=αkε−ρ1εPkmax−Pkc−∑sk=1Sk∑n=1NP^k,sk,nS∗t+,(33)βnε+1=βnε−ρ2εIn−∑k=1K∑sk=1SkP^k,sk,nS∗thk,m,nSMt+,where ρ1ε and ρ2ε are the step sizes of iteration ε.Based on the above optimal transmission power allocation results, we can find the optimal subcarrier assignment. Decomposing the dual problem (27) into Sk subproblems, for each subproblem, we have(34)Lk,nP=∑sk=1SkLk,sk,nP,where(35)Lk,sk,nP=Vϕ+Jk,sktX^k,sk,nt×log21+P^k,sk,nS∗thk,sk,nStX^k,sk,ntPm,nMthk,sk,nMSt+δ2−αkP^k,sk,nS∗t+Fkt−θP^k,sk,nS∗t−βnP^k,sk,nS∗thk,m,nSMt−ζk,nX^k,sk,nt.Subcarriers will be assigned to SUEs with the largestLk,sk,nP, which means(36)nsk∗=argmaxskLk,sk,nP,where Lk,ns∗,nP>0.Therefore, the subcarrier assignment decision is as follows:(37)Xk,sk,n∗t=1,ifsk=nsk∗,0,otherwise.Besides, we also use the subgradient method to update the Lagrange multiplierζk,n, which is updated as(38)ζk,nε+1=ζk,nε−ρ3ε1−∑sk=1SkX^k,sk,n∗t+,where ρ3ε is the step size of iteration ε.


## 4.2. Energy Scheduling and Energy Cooperation Subproblem

The energy scheduling and energy cooperation subproblem jointly considers the control of SBS’s discharging, charging, energy cooperation, and on-grid energy consumption. At each time slott, SBS k solves the following problem based on local information for energy scheduling and energy cooperation:(39)P4:minYkt,Ok,kt,Ok,k′t∑k=1KFkt−θ+VψGkt+∑k=1KFkt−θYkt+∑k=1KFkt−θ∑k′≠kOk′,kts.t.C7,C8,C9,C10,C11:Ok,kt+∑k′≠kOk′,kt≤P˜kt.According to constraint (8) and ∑k=1K∑k′≠kOk′,kt=∑k=1K∑k′≠kOk,k′t, P4 can be further written as(40)P5:minYkt,Ok,kt,Ok,k′t∑k=1KFkt−θ+VψP˜kt−∑k=1KFkt−θ+VψOk,kt+∑k=1KFkt−θYkt−Vψ∑k=1K∑k′≠kOk,k′ts.t.C7,C8,C9,C10,C11.The optimal solution to the above problem isYk∗t, Ok,k∗t, and Ok,k′∗t. The energy Gk∗t of SBS k getting from the grid can be expressed as(41)Gk∗t=P˜kt−Ok,k∗t−∑k′≠kOk′,kt+.Since P5 is a linear programming problem, it can be solved directly by linear programming tools.


## 4.3. Online Algorithm

As presented in the above subsections, the formulated time-average delay-aware and interference-aware energy efficiency problem is solved distributively through several decompositions. Firstly, we use Lyapunov optimization method to decompose the optimal problem into two subproblems at each time slot, that is, resource allocation subproblem and energy scheduling and energy cooperation subproblem. Secondly, the resource allocation subproblem is solved through Lagrange dual decomposition. Then, the dual problem is decomposed intoK×N independent subproblems for optimal power allocation and Sk independent subproblems for optimal subcarrier assignment. Meanwhile the energy scheduling and energy cooperation subproblem is decomposed into K independent subproblems for the optimal discharging, charging, energy cooperation, and on-grid energy consumption.Based on the above solution, we further present the distributed algorithm for joint resource scheduling and energy cooperation in interference-constrained HetNets powered by hybrid energy, which is shown in Algorithm1.Algorithm 1: Resource scheduling and energy cooperation algorithm for HetNets with cross-layer interference constraints.
Require: at time slot t, get the queue states Jt, Ft, and the channel gain Ht=hk,sk,nStOutput: transmission power allocation and subcarrier assignment, energy scheduling and energy cooperationStage 1: resource allocationInitialize Lagrange multipliersα0, β0andζ0, set ν=0Repeatν=ν+1For each SBS k∈KFor each SUE s∈SFor each subcarrier n∈NAccording to (31), calculate Pk,sk,nStAccording to (35), calculate Lk,sk,ntAccording to (37), calculate Xk,sk,ntEnd ForEnd ForEnd ForAccording to (32) update αAccording to (33) update βAccording to (38) updateζUntilν≤νmax and or the convergence of the multipliers.Stage 2:Energy Scheduling and Energy CooperationSolve P5 using linear programming tools, getYk∗t,Ok,k∗t,Ok,k′∗tBased on (41), calculate Gk∗tUpdate all queues


## 5. Performance Analysis

Theorem 1.
Whenθ=^Vψ+Okmax and(42)0<V≤Fkmax−Ykmax−Okmaxψ,hold, the following relationships have been established:(i)
IfFkt>Fkmax−Ykmax, Yk∗t=0.(ii)
IfFkt<Okmax, Ok,k∗t=0.Proof.
See AppendixB.Theorem 2.
Whenθ=^Vψ+Okmax and(43)0<V≤Fkmax−Ykmax−Okmaxψ,are satisfied, the renewable energy collected by the storage device satisfies Fkt∈0,Fkmax at any time slot.Proof.
See AppendixC.Lemma 2.
Assume that the coverage range of the network isΛ. As the data arriving process Jk,skt, the channel gain Ht, and the renewable energy generation Ekt all follow i.i.d processes, a random control strategy ω is implemented. Then,(44)Eϕ∑k=1K∑sk=1SkRk,skωt−Eψ∑k=1KGkωt=ι2,EOk,kωt=EFkωt,ERk,skωt≥EAk,skt+ς,where ς is a finite positive number. When the constraint of P1 is satisfied, it can be proved by Carathéodory’s theorem. Please refer to the specific proof in [22, 23].Theorem 3.
(a) The upper bound of the time-average data queue is(45)limT⟶∞sup1T∑t=0T−1∑k=1K∑sk=1SkJk,skt≤Ξς.
(b) The time-average energy efficiency of the proposed algorithm satisfies(46)limT⟶∞sup1T∑t=0T−1ϕ∑k=1K∑sk=1SkAk,sk−ψ∑k=1KGk≥ι1−ΞV.Proof.
See AppendixD.


## 6. Simulation Results

In this section, the simulation results of the proposed algorithm are given. We consider a hybrid network that consists of one MBS andK=3 SBSs, where the coverage radius of MBS is 500m and the coverage radius of SBS is 30m. The MBS has 10 MUEs and each SBS has 2 SUEs. In this paper, we consider wireless channels with small-scale Rayleigh fading component and a large-scale path loss. Every SUE has a buffer to its related SBS, where the buffer size Jk,skmax=500. At each time slot, SBS k receives burst data of its corresponding SUE sk which follows a Poisson distribution with average rate λk,sk=15. SBS k receives renewable energy which follows a Poisson distribution with average rate λk=5.5J/s [9, 24]. The other parameters are listed in Table 2.Table 2
Simulation parameters.
ParameterValueTotal bandwidthB1 MHzAdditive white Gaussian noise power spectral densityN0−174 dBmNumber of subcarriersN10Noise constantσk2B/NN0Number of subcarriersN10Transmitted power of MBS on each subcarrierPm,nM1 WMaximum energy consumption of SBSPkmax5 WStatic power of SBSPkc4.8 WMaximum interference temperatureIn2×10−6Path loss between SBS and SUE37+30logdPath loss between MBS and MUE34+40logdDelay constraintρk,skmax50Capacity of batteryCkmax60 JMaximum charge rateFkmax1.5∗PkmaxMaximum discharge rateOkmaxPkmaxFirstly, we demonstrate the stability of network. We chooseT=1000,V=10,ϕ=4,ψ=0.5; the trend of each queue is shown in Figure 2. As the same types of queues follow a similar trend, we only show the trends of SUE1 of SBS1, SUE2 of SBS2, and SUE3 of SBS3 in the data queue J and show the trends of SBS1, SBS2, and SBS3 in the energy queue F. From Figure 2, all data queues are strictly less than the maximum data buffer size, which is in line with Theorem 3 (a); and all the energy queues do not exceed the capacity of the storage devices, which is in line with Theorem 2.Figure 2
Stability of data queues and energy queues.
(a)(b)WhenT=1000,V=10,ϕ=4,ψ=0.5,λk,sk=5, we give the relationship between the long-term admitted rate A¯k,sk and the service rate R¯k,sk for all SUEs. As shown in Figure 3, each SUE’s admitted rate is less than the service rate, which means that the actual queues are stable.Figure 3
Relationship between admitted rate and service rate of SUEs.Figure4 shows the trend of time-average objective value with parameter ψ when V and ϕ are fixed. For each ψ, we take T=1000. With the increase of ψ, which means the emphasis on time-average on-grid energy consumption, the time-average objective values decrease. On the other hand, Figure 5 shows the trend of time-average objective value with parameter ϕ when V and ψ are fixed. For each ϕ, we take T=1000. As ϕ increases, which means that the time-average service rate weighs more, the time-average objective value increases. The results of both Figures 4 and 5 agree with Theorem 3 (b).Figure 4
Time-average objective value versusψ.Figure 5
Time-average objective value versusϕ.Figure6 compares the proposed algorithm with a basic algorithm. To highlight the proposed joint resource allocation and energy cooperation algorithm, in the basic algorithm, all SBSs are supplied by hybrid energy and perform energy scheduling noncooperatively. Besides, the resource allocation in the basic algorithm is optimized with the proposed algorithm in this work. It is shown that, at V=10, the value of the objective function of the proposed algorithm is about 7% higher than that of the basic algorithm without energy cooperation, which demonstrates that the proposed algorithm has a better time-average energy efficiency than that of the basic algorithm. Moreover, as shown in Figures 4 and 5, the tradeoff factors ϕ and ψ also greatly affect the value of the objective function.Figure 6
Performance comparison between the proposed algorithm and the basic algorithm.


## 7. Conclusion

In this paper, we consider the resource scheduling and energy cooperation for a two-tier downlink HetNet, which is powered by hybrid energy supplies. By jointly adjusting the transmission power allocation, the subcarrier assignment, the discharging and charging, the energy cooperation, and the on-grid energy consumption, we can maximize the time-average energy efficiency of SBSs in an online manner. Compared with the other works, this paper considers the burst of user data, the cross-layer interference from SBSs to MBS, and the scheduling of random renewable energy sources. Through theoretical analysis and simulations, we prove the stability and superiority of the proposed algorithm.



---

*Source: 4717195-2020-08-28.xml*