# Robust Nash Dynamic Game Strategy for User Cooperation Energy Efficiency in Wireless Cellular Networks

Authors: Shuhuan Wen; Baozhu Hu; Ahmad B. Rad; Xinbin Li; Huibin Lu; Jianhua Zhang
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

Recently, there is an emerging trend of addressing “energy efficiency” aspect of wireless communications. It has been shown that cooperating users relay each other's information to improve data rates. The energy is limited in the wireless cellular network, but the mobile users refuse to relay. This paper presents an approach that encourages user cooperation in order to improve the energy efficiency. The game theory is an efficient method to solve such conflicts. We present a cellular framework in which two mobile users, who desire to communicate with a common base station, may cooperate via decode-and-forward relaying. In the case of imperfect information assumption, cooperative Nash dynamic game is used between the two users' cooperation to tackle the decision making problems: whether to cooperate and how to cooperate in wireless networks. The scheme based on “cooperative game theory” can achieve general pareto-optimal performance for cooperative games, and thus, maximize the entire system payoff while maintaining fairness.


---

## Body

## 1. Introduction

The continuously growing demand for ubiquitous wireless access leads to the rapid development of the wireless cellular networks during the last decade. Such tremendous growth in wireless industry has made it become one of the leading sources of the world energy consumption, and it is expected to grow dramatically in the future. The electricity bill has become a significant portion of the operational expenditure of operators [1]. Rapidly rising energy costs and increasingly rigid environmental standards have led to an emerging trend of addressing “energy efficiency” aspect of wireless communication technologies.In a typical wireless cellular network, user cooperation can increase the users’ data rates [2–5]. It has been shown that cooperating users relay each other's information to improve the data rates [6]. One user's rate increases only when another user expends relay energy on its behalf. The energy is limited in the wireless cellular network but mobile users refuse to relay, which renders user cooperation (ostensibly) unappealing. But it is very important that user cooperation is advantageous from the perspective of energy efficiency. We need an approach that encourages user cooperation in order to improve their energy efficiency. The game theory is an efficient method to solve such conflicting problems [7]. It was introduced by von Neumann and Morgenstern [8]. In [9], a cooperative Nash game is used between the two users’ cooperation to improve the bits-per-energy efficiency, but it assumed global channel state information, which is often difficult to obtain in practice. Each player is given a utility function (the reward or penalty it receives) of its own strategy and the strategies are played by all the other players (or a subset of them). In the general approach, the game and the strategies are discrete, and therefore, matrices with strategies and payoffs (the rewards or penalties) may be assembled. The players should make decision when each player has not a small number of strategies. To tackle the decision-making problems on whether to cooperate and how to cooperate in wireless networks, a differential game was introduced by Isaacs [10]. Many research works based on game theory have been published [11]. Cooperative games hold out the promise of socially optimal and group efficient solutions to problems involving strategic actions. Formulation of the optimal player’s behavior is a fundamental element in this theory. The schemes based on “cooperative game theory” can achieve the general pareto-optimal performance for cooperative games and, thus, maximize the entire system payoff while maintaining fairness [12].Generally, when solving a game theoretically, one also assumes that one agent (or player) has access to the states of the other players involved in the game at all times as well as their cost functions [13, 14]. This is called the perfect information assumption. In fact, it is impossible to realize that each player knows the decision making of all the others. Little attention has been paid to how a player may learn to play the game. So in this paper, we adopt dynamic game, which investigates how decision making takes place over time [15]. Robust Nash Equilibria is derived based on the Robust Maximum Principle [16]. This extension of the traditional Maximun Principle [17], allows us to deal with systems that can be interpreted, to be presented as a family of ordinary differential equations, each of which corresponds to a different scenario or possible dynamics. But there are always uncertainties in the game. The main drawback of this last approach ignores uncertainties. Therefore, in the situation when there are any unmeasured uncertainties, another design concept must be developed.Generally, imperfect channel state information has significant impacts on the performance of not only cellular networks, but also wireless networks. Many works also assume global channel state information, which is often difficult to obtain in practice. Moreover, since the channel state information in cellular networks is often limited, we introduce the possibility of channel prediction. We study an efficient solution, which is AR-model-based prediction method, to take the prediction for the channel information states. AR-model-based prediction method has good suitability and robustness that utilizes fast tracking of channel coefficients, which demonstrates for realistic channel conditions.


## 2. System Models

### 2.1. Transmission Model

In cellular networks, we consider that two mobile users communicate with a common base-station (BS). We assume a simple uplink scenario in Figure1. User 1 transmits its data to the BS during the odd time blocks, while user 2 transmits its data during the even time blocks. The idle user can choose to act as a relay for the active user in order to help to increase the active users’ rate. The received signals at the relay and destination then [9]
(2.1)y1=hu21(k)x2+ξ1(k)(evenk),y2=hu12(k)x1+ξ2(k)(oddk),y3=hus1(k)x1+hus2(k)x2+ξ3(k),
where x1, x2 are transmission or relay symbols, and ξ is additive white Gaussian noise.Figure 1
Transmission schedule. During odd time blocks, user 1 acts as source and user 2 acts as relay, and vice versa for even time blocks.


### 2.2. Channel Prediction Model

We consider a simple uplink transmission scenario from users to the BS in the cellular networks, as depicted in Figure1. We assume that the BSs map a set of messages onto a set of sequences, which are precoded and transmitted symbol-wise over the channel. The transmission in each single channel access at sample time slot k can be stated as
(2.2)y(k)=H(k)x(k)+z(k),
where y(k) is the received signals, H(k) denotes the time-varying channel cofficient, x(k) is the symbols connected to the messages to be transmitted in channel access k, and z(k) is the additive noise at the receiver side, also assumed to be zero-mean Gaussian.In the AR-model-based algorithm, the prediction of the channel state in time slotk, the value of channel state H(k) is given by [18]
(2.3)H^(k)=∑j=1Jdj(k)H(k-j)+e(k),
where J is the AR-model order, and H(k-1)⋯H(k-j)⋯H(k-J) are J previous channel samples, and e(k) is a complex white Gaussian noise signal. The AR-model-based prediction methods are also often referred to as linear predictors (LPs). The knowledge of the identifying is needed for the computation of the AR coefficients dj(k) in (2.3). Because the channel coefficients are unknown and slowly time-variant in practice, it has to be estimated from noise-corrupted channel observations. Efficient adaptive filtering techniques, such as recursive least squares estimate (RLSE), were used to track the changes of predictor coefficients dj(k) in (2.3).The prediction error, which is defined as the difference between the predicted valueH^(k) and the actual value H(k), is employed as a performance measure [19–21]. The prediction error can be normalized with the respect to the root mean squared value (RMS) of the considered fading process [22]. The prediction error analysis of M consecutive predictions can be done by using the variance defined as follows:
(2.4)γ2(k)=1M∑m=0M-1(H(k-m)-H^(k-m))2.In this paper, we use RLSE due to its superior performance in accommodating time-varying system conditions, which results in an efficient method for time-varying radio channels [23].


### 2.3. Bits-per-Energy Efficiency

One common method to measure energy efficiency is to use bits per Joule [24]. Strict maximization of energy efficiency, then, is not typically useful in practice when users have target rates to meet or delay constraints to satisfy [9]. However, the channel prediction in Section 3 is used to sidestep this obstacle. The total energy expenditure is proportional to the sum of the power allocations in the previous time blocks. So, the total bits-per-energy efficiency for each user is proportional to [9]
(2.5)ei(pr1,pr2)=Eh[ri(prj(h))]1+Eh[pri(h)],
for i≠j, where pr1, pr2 are the power allocated by user1 and user2 while acting as the relay. ri(·) is the achievable rate, whose expression follows [9]
(2.6)ri(k)=min{max{I(xi;xj∣xj),I(xi;x3)},I(xi,xj;y3)},
where xi,yi are jointly Gaussian, and the mutual information terms depend on the channel gains and the source and relay powers at time slot k.


## 2.1. Transmission Model

In cellular networks, we consider that two mobile users communicate with a common base-station (BS). We assume a simple uplink scenario in Figure1. User 1 transmits its data to the BS during the odd time blocks, while user 2 transmits its data during the even time blocks. The idle user can choose to act as a relay for the active user in order to help to increase the active users’ rate. The received signals at the relay and destination then [9]
(2.1)y1=hu21(k)x2+ξ1(k)(evenk),y2=hu12(k)x1+ξ2(k)(oddk),y3=hus1(k)x1+hus2(k)x2+ξ3(k),
where x1, x2 are transmission or relay symbols, and ξ is additive white Gaussian noise.Figure 1
Transmission schedule. During odd time blocks, user 1 acts as source and user 2 acts as relay, and vice versa for even time blocks.


## 2.2. Channel Prediction Model

We consider a simple uplink transmission scenario from users to the BS in the cellular networks, as depicted in Figure1. We assume that the BSs map a set of messages onto a set of sequences, which are precoded and transmitted symbol-wise over the channel. The transmission in each single channel access at sample time slot k can be stated as
(2.2)y(k)=H(k)x(k)+z(k),
where y(k) is the received signals, H(k) denotes the time-varying channel cofficient, x(k) is the symbols connected to the messages to be transmitted in channel access k, and z(k) is the additive noise at the receiver side, also assumed to be zero-mean Gaussian.In the AR-model-based algorithm, the prediction of the channel state in time slotk, the value of channel state H(k) is given by [18]
(2.3)H^(k)=∑j=1Jdj(k)H(k-j)+e(k),
where J is the AR-model order, and H(k-1)⋯H(k-j)⋯H(k-J) are J previous channel samples, and e(k) is a complex white Gaussian noise signal. The AR-model-based prediction methods are also often referred to as linear predictors (LPs). The knowledge of the identifying is needed for the computation of the AR coefficients dj(k) in (2.3). Because the channel coefficients are unknown and slowly time-variant in practice, it has to be estimated from noise-corrupted channel observations. Efficient adaptive filtering techniques, such as recursive least squares estimate (RLSE), were used to track the changes of predictor coefficients dj(k) in (2.3).The prediction error, which is defined as the difference between the predicted valueH^(k) and the actual value H(k), is employed as a performance measure [19–21]. The prediction error can be normalized with the respect to the root mean squared value (RMS) of the considered fading process [22]. The prediction error analysis of M consecutive predictions can be done by using the variance defined as follows:
(2.4)γ2(k)=1M∑m=0M-1(H(k-m)-H^(k-m))2.In this paper, we use RLSE due to its superior performance in accommodating time-varying system conditions, which results in an efficient method for time-varying radio channels [23].


## 2.3. Bits-per-Energy Efficiency

One common method to measure energy efficiency is to use bits per Joule [24]. Strict maximization of energy efficiency, then, is not typically useful in practice when users have target rates to meet or delay constraints to satisfy [9]. However, the channel prediction in Section 3 is used to sidestep this obstacle. The total energy expenditure is proportional to the sum of the power allocations in the previous time blocks. So, the total bits-per-energy efficiency for each user is proportional to [9]
(2.5)ei(pr1,pr2)=Eh[ri(prj(h))]1+Eh[pri(h)],
for i≠j, where pr1, pr2 are the power allocated by user1 and user2 while acting as the relay. ri(·) is the achievable rate, whose expression follows [9]
(2.6)ri(k)=min{max{I(xi;xj∣xj),I(xi;x3)},I(xi,xj;y3)},
where xi,yi are jointly Gaussian, and the mutual information terms depend on the channel gains and the source and relay powers at time slot k.


## 3. Theoretic Formulation of Discrete Dynamic Game

A game consists of three objects: a set of players, a set of strategies which those players can enact, and a set of cost functions denoting the payoff each player derives from the strategies enacted [9]. We assume our scenario as a game in which each mobile user is a player whose strategies are the relay power allocations pri and whose cost functions are the bits-per-energy efficiencies eipri.


### 3.1. Noncooperative Games

In order to compare the energy efficiency of noncooperative and cooperative users, the energy efficiency expression of noncooperative users is given. The classic solution concept for a noncooperative game is the Nash equilibrium (NE). The payoff of every noncooperative user is [9]
(3.1)eiNE=Eh[log2(1+|hi3|2)],
where E(·) is the expectation.The noncooperative game is a strategy that the selfish users simply select their strategies and never interact again. They improve their payoff by lowing their relay power. While in fact, users can obtain higher payoff by cooperation.


### 3.2. Nash Equilibrium Dynamic Game

Recently, the game theory of user cooperation has been studied in the literature. However neither of these works considers the energy efficiency. In [9], the paper studies energy efficiency of user cooperation, but it ignores the process of decision making. The results of these works assume perfect situation in which each player knows the decision making of all the others. In fact, it is impossible to implement.In this paper we consider the linear discrete-time dynamic game model with uncertainties:(3.2)x(k+1)=Ax(k)+bu(k)+Dg(x,k),
where A, B are the known game constant matrices, x(k) is the state vector of the game, u(k) is the control strategies of each user, D is the constant matrix, and g(x,k) is the unknown noises or disturbances part. Each vector belongs to an appropriate dimensional Euclidean space.If the game under study is (as it is for this paper) between only two players, the system dynamics (3.2) may be written as
(3.3)xi(k+1)=Axi(k)+bui(k)+Dgi(x,k),
where i=1,2 denotes the two users.With this linear discrete-time system, we associate a quadratic performance index(3.4)J(u1(k),u2(k))=∑k=1K{|xi(k+1)|Q(k+1)2+|ui(k)|2}=∥xi∥Q2+∥ui∥2,
where Q(k+1)≥0, |·|Q denotes an appropriate Euclidean (semi-) norm weighted by a nonnegative definite matrix Q, ∥·∥ denotes a corresponding l2 norm.Then we give the Nash Equilibrium strategies of the two users. The payoff, now represented asL(u1(k),u2(k)), is given in the form
(3.5)L(u1(k),u2(k))=J(u1(k),u2(k))+|xi|Q2.In this case it is also assumed that the users who use strategyu1(k),u2(k) want to maximize the payoff L(·). Therefore, the objective of the game is to find a saddle-point solution u1*(k),u2*(k) to satisfy the Nash-equilibrium condition
(3.6)L(u1*(k),u2(k))≤L(u1*(k),u2*(k))≤L(u1(k),u2*(k)),u1*(k),u2*(k) is known as a feedback saddle-point solution, then u1*(k),u2*(k) is characterized as the point on the Pareto boundary that satisfies (3.6), symmetry between players, and independence to irrelevant alternatives. The control strategies of the users in a cooperative game are their relay power allocations and the cost functions are their bits-per-energy efficiencies.


## 3.1. Noncooperative Games

In order to compare the energy efficiency of noncooperative and cooperative users, the energy efficiency expression of noncooperative users is given. The classic solution concept for a noncooperative game is the Nash equilibrium (NE). The payoff of every noncooperative user is [9]
(3.1)eiNE=Eh[log2(1+|hi3|2)],
where E(·) is the expectation.The noncooperative game is a strategy that the selfish users simply select their strategies and never interact again. They improve their payoff by lowing their relay power. While in fact, users can obtain higher payoff by cooperation.


## 3.2. Nash Equilibrium Dynamic Game

Recently, the game theory of user cooperation has been studied in the literature. However neither of these works considers the energy efficiency. In [9], the paper studies energy efficiency of user cooperation, but it ignores the process of decision making. The results of these works assume perfect situation in which each player knows the decision making of all the others. In fact, it is impossible to implement.In this paper we consider the linear discrete-time dynamic game model with uncertainties:(3.2)x(k+1)=Ax(k)+bu(k)+Dg(x,k),
where A, B are the known game constant matrices, x(k) is the state vector of the game, u(k) is the control strategies of each user, D is the constant matrix, and g(x,k) is the unknown noises or disturbances part. Each vector belongs to an appropriate dimensional Euclidean space.If the game under study is (as it is for this paper) between only two players, the system dynamics (3.2) may be written as
(3.3)xi(k+1)=Axi(k)+bui(k)+Dgi(x,k),
where i=1,2 denotes the two users.With this linear discrete-time system, we associate a quadratic performance index(3.4)J(u1(k),u2(k))=∑k=1K{|xi(k+1)|Q(k+1)2+|ui(k)|2}=∥xi∥Q2+∥ui∥2,
where Q(k+1)≥0, |·|Q denotes an appropriate Euclidean (semi-) norm weighted by a nonnegative definite matrix Q, ∥·∥ denotes a corresponding l2 norm.Then we give the Nash Equilibrium strategies of the two users. The payoff, now represented asL(u1(k),u2(k)), is given in the form
(3.5)L(u1(k),u2(k))=J(u1(k),u2(k))+|xi|Q2.In this case it is also assumed that the users who use strategyu1(k),u2(k) want to maximize the payoff L(·). Therefore, the objective of the game is to find a saddle-point solution u1*(k),u2*(k) to satisfy the Nash-equilibrium condition
(3.6)L(u1*(k),u2(k))≤L(u1*(k),u2*(k))≤L(u1(k),u2*(k)),u1*(k),u2*(k) is known as a feedback saddle-point solution, then u1*(k),u2*(k) is characterized as the point on the Pareto boundary that satisfies (3.6), symmetry between players, and independence to irrelevant alternatives. The control strategies of the users in a cooperative game are their relay power allocations and the cost functions are their bits-per-energy efficiencies.


## 4. Simulation Results and Discussions

In this section, we use computer simulations to evaluate the performance of the proposed scheme and the influence of uncertainties. The results of numerical experiments are shown to see the payoff of the cooperation users. The energy efficiency of every user increases when the proposed scheme is used. We compare the proposed scheme with the noncooperation game scheme, which is a single-stage noncooperative game. The solution to a game depends on the particular game theoretic frame work employed. We consider a simple case that the selfish players select strategies in order to maximize individual energy efficiency in spite of the payoff of other players. The players never interact again. Then another user cooperation scheme which does not use dynamic game and assumes perfect channel state information [9] is compared with the proposed scheme.Figure2 shows the coefficients in the AR-model-based channel prediction algorithm of one wireless channel, d1 and d2, which are defined in (2.3). As we can see from the figure, the AR-model coefficients vary with simulation time due to the time-varying wireless channel. The recursive least square estimate (RLSE) algorithm presented in Section 3 can accommodate the time-varying wireless channel conditions to make accurate channel predictions.Figure 2
The coefficients in the AR-model-based channel prediction algorithm.The performance of the channel prediction algorithm can be measured by using the error variance, which is defined in (2.4). Figure 3 shows the error variance in the AR-model-based channel prediction algorithm. In Figure 3, we can observe that, when the simulation time increases, the error variance decreases, meaning that the channel prediction becomes more accurate. In addition, we can see that the variance drops below 1 quickly in the simulation, which means that the prediction algorithm is sufficiently accurate.Figure 3
The error variance in the AR-model-based channel prediction algorithm.Because there is time-varying external disturbance and uncertainties in the cellular network, the cooperation of users should consider the effect of them for energy efficiency. It is assumed that the external disturbance is given bywd(k+1)=0.95wd(k) with wd(0)=1. Figure 4 shows the plot of the time-varying external disturbance. It is also assumed that there is an unknown uncertainty which is time-varying in the range |vun(k)|≤1. The time-varying uncertainty vun(k) is randomly generated as shown in Figure 5.Figure 4
Time-varying external disturbance.Figure 5
Time-varying external disturbance.Figure6 depicts the variation bit per Joule efficiency of the user cooperation in the uplink scenario. The result indicates that the cooperation of the users is helpful to increase the energy efficiency. We set the expected interuser gains to set user 2’s expected gain E∥h232∥=10 dB and make E∥h132∥ dB vary between −5 dB and 20 dB in order to compare with [9]. This range allows us to see the cooperation performance of the proposed scheme when users’ channel conditions are asymmetric, which gives us a reasonable result. Figure 6 shows that each user's bit per Joule efficiency by using the proposed scheme is the largest, and the [9] is the second. The bit per Joule efficiency of the users without cooperation is the smallest. This shows it can benefit from the cooperation for selfish users by using the proposed scheme.Figure 6
The energy efficiency of different schemes.


## 5. Conclusion

In wireless cellular networks, it is very important to increase the energy efficiency of the radio access networks to meet the challenges raised by the high demands of traffic and energy consumption. In this paper, we proposed a dynamic game based on the control-theoretical approach to study the energy efficiency issues in cooperative wireless cellular networks with cellular communications. In addition, we use a predictor in our control-theoretical approach to supervise the data of imperfect channel state information. Simulation results have been presented to show the effectiveness of the proposed scheme.Mutiuser cooperation case is considered in the future work. We also consider heterogeneous cellular networks with femtocells, which is another promising technique to increase the indoor network coverage and capacity for the growing demands of cellular services, and it has been integrated in current and future radio access networks. In addition, intercell interference will be considered in our control-theoretical framework.



---

*Source: 636159-2012-11-19.xml*