# Sensor Communication Rate Control Scheme Based on Inference Game Approach

Authors: Sungwook Kim
Journal: International Journal of Distributed Sensor Networks
 (2015)

---

## Abstract

In real-life situations, decisions must be made even when limited or uncertain information is available. Therefore, the payoff of an action is not clearly known when the decision is made. Recently, game theory has become a powerful tool for analyzing the interactions between decision makers in many domains. However, the traditional game theory approach assumes that a player belief about the payoff of a strategy taken is accurate. To address this problem, we introduce a new kind of game, called an inference game, and study how degrees of uncertainty of belief about payoffs impact the outcomes of real-world games. To approximate an optimal decision, our proposed inference game model can clarify how to better manage ambiguous information. In this study, we apply our inference game model to the sensor communication paradigm and confirm that our approach achieves better performance than other existing sensor communication schemes in widely diverse Internet of Things (IoT) environments.


---

## Body

## 1. Introduction

The rapid development of Internet of Things (IoT) technology makes it possible to connect various smart objects together through the Internet and to provide more data interoperability methods for application purposes. Recent research shows an increase in the number of potential applications of IoT in information-intensive industrial sectors. In various scenarios, IoT can be realized with the help of sensor communication, which provides ubiquitous networking to connect devices, so that they can communicate with each other to make collaborative decisions with limited, or without any, human intervention. Recently, the sensor communication paradigm has been considered as a new type of communication, empowering full mechanical automation that has the potential to change our life styles [1, 2].However, enabling sensor communication in IoT is not straightforward. One major issue is how multiple machine-type devices should be connected in dynamic network situations. In addition, to achieve successful sensor communications, Quality-of-Service (QoS) provisioning is another important requirement. For machine devices, some applications require deterministic and hard timing constraints, and disasters occur when these are violated. For other applications, statistical and soft timing constraints are acceptable. Thus, one of the most challenging tasks is how to effectively multiplex massive accesses with enormously diverse QoS characteristics [3]. Existing mechanisms do not adaptively tackle this QoS issue when services in IoT are performed. Until now, it is a complex and difficult work in a dynamically changing IoT environment [4, 5].For IoT multimedia services, decisions that influence QoS are related to the packet rate control for application traffic. Based on real-time feedback, each machine device can adapt its behavior and make QoS decisions strategically to maximize its payoffs [6, 7]. This strategic interaction among machine devices can be formally modeled as a decision-making mechanism. It is regarded as a process that results in the selection of a course of action from several alternatives. However, in real-world IoT operations, control decisions have to be made with only limited information. To address this issue, it is necessary to develop an effective control decision mechanism that works in situations involving uncertainty, which is caused by time pressure, lack of data, unknown factors, randomness outcome of certain attributes, and so forth [8–11].Game theory is a study of strategic decision making. Specifically, it is the study of mathematical models of conflict and cooperation between intelligent rational decision makers. As a powerful tool for modeling strategic interactions, game theory is widely used in any fields such as biology, economics, political science, management science, and computer science and telecommunications [12–14]. However, the fundamental assumption of classical game theory is that the consequence or payoff of a strategy profile is determinate or precise [15, 16]. However, this assumption seems implausible and unreasonable under the real-world environment.In view of realistic situations, game players may not be able to exactly expect their precise payoffs of strategy profiles. Due to limited information, players in real-life games have to make decisions under uncertainty. In canonical opinion, “uncertainty” is referred to as a kind of ambiguity that describes situations where decision makers cannot determine a precise probability distribution over the possible consequences of an action [10]. Therefore, in games under uncertainty, the players could only assign a set of possible payoffs, rather than a precise payoff, and have an imprecise probability distribution over this set [8, 10]. To model this situation with indeterminate payoffs, some researchers have tried to apply some original ideas taken from decision theory to game models. However, this kind of work still assumes that the consequences in a game are accurate; it cannot adequately handle the problem concerning uncertain consequences and attitudes of players [10].Motivated by the above discussion, we propose a new game model that can deal with uncertain situations. By employing the rule of inferences, we can allow a player belief concerning the possible payoffs and determine a preference ordering over actions with respect to expected payoffs. Therefore, the proposed game model can relax the rather stringent assumption of traditional game models. Based on our uncertainty-control game model, we develop a new packet transmission rate control scheme for sensor communication. In interactive situations involving uncertainty, machine devices in the proposed scheme can respond to current IoT system conditions for adaptive management. Therefore, they properly select the most adaptable strategy for packet transmissions while ensuring QoS for sensor communication. The distinct feature of our proposed scheme is a more realistic game-based approach with the limited information. Therefore, we can achieve a well-balanced system performance between contradictory performance requirements.Recently, several sensor communication schemes have been presented for complex QoS provisioning IoT systems. TheDistributed Rate and Admission Control(DRAC) scheme in [4] is proposed to concentrate on the QoS management in sensor networks. The DRAC scheme is integrated with a game theory analysis module to model the competition of radio bandwidth among machine devices. TheClass Based Priority Scheduling(CBPS) scheme [5] is a scheduling algorithm for the communication in IoT systems. The CBPS scheme can classify and prioritize all sensor communication flows based on their QoS requirements. All the earlier work in [4, 5] has attracted a lot of attention and introduced unique challenges. Compared to the DRAC and CBPS schemes, the proposed scheme attains better performance for sensor communication in IoT systems.The rest of this paper is organized as follows. Section2 gives a formal definition of the inference game model and discusses how to set a proper preference ordering over expected payoffs. In addition, Section 2 explains in detail the main steps of proposed sensor communication algorithm. In Section 3, we present our simulation model and discuss experiments and results. Finally, Section 4 summarizes the paper and points out some possibilities of the future work.


## 2. Proposed Sensor Communication Algorithm

In this section, we develop a new game model with vagueness payoffs, called an inference game. To get the outcome of game involving uncertainty, an inference process is adopted according to the degrees of beliefs about payoffs. Compared to the traditional game model, we explain why our approach yields the effective performance for the highly dynamic IoT system nature.


### 2.1. Inference Game Model and Inference Process

To model strategic interactive situations involving uncertainty, we develop a new inference game, which is constructed based on the assumption of a player belief regarding the uncertain payoffs. Therefore, an imprecise probability distribution over the set of the possible payoffs is assigned based on the player belief. This means that the game players are not sure about the payoffs of each strategy but assign a set of possible payoffs to each strategy profile. To effectively expect the possible payoffs, we apply some original ideas taken from the Bayesian inference process. For the modeling of uncertainty, our approach has become a key challenge in the real-world decision problems. In this section, we define our inference game model (G) as follows.Definition 1.
An inference game model constitutes a 5-tupleG = (N, S, ξ, F
s, δ
s), where (i)
Nis a set of game players,
(ii)
S
=
s
1
,
s
2
,
…
,
s
n is a nonempty finite set of all pure strategies of players,
(iii)
ξ
=
u
1
,
u
2
,
…
,
u
m is the utility set of all consequent payoffs of each strategy; it is defined as discrete satisfaction levels of players where u
i
,
1
≤
i
≤
m
∈
R; there are m level satisfactions,
(iv)
F
s
=
{
F
s
(
u
)
∣
s
∈
S
,
u
∈
ξ
}  is a probability assignment function, which maps the strategy choice s over the strategy set S onto a consequence u over the strategy set ξ, where F
s
:
F
s
u
→
[
0,1
] and ∑
u
∈
ξ
F
s
(
u
)
=
1,
(v)
δ
s is the uncertainty degree of consequence, which could be caused by strategy choice s  (
0
≤
δ
s
≤
1
).During the inference game process, a strategy s
k
,
1
≤
k
≤
n can cause a consequence u
i
,
1
≤
i
≤
m that is specified by the mapping probability function F
s
k. According to the consequent payoffs,Expected Payoff Interval for the strategy s
k (E
P
I
s
k) is defined as follows:(1)
E
P
I
s
k
=
U
m
i
n
s
k
,
U
m
a
x
s
k
,
s
.
t
.
,
U
m
i
n
s
k
=
m
i
n
u
i
∈
ξ
F
s
k
u
i
×
u
i
,
U
m
a
x
s
k
=
m
a
x
u
i
∈
ξ
F
s
k
u
i
×
u
i
.Under uncertain situations, F
s function is essential for the decision making. In this paper, F
s represents the player belief for outcomes of each strategy. To dynamically adapt the current situation, F
s is updated as new observations become available. Therefore, it is necessary to adopt a scientific inference method in order to adaptively modify F
s.Bayesian inference is a method of statistical inference to provide a logical, quantitative decision. Based on the Bayes theorem and Bayesian probability rules, Bayesian inference summarizes all uncertainty by a “posterior” distribution and gives a “posterior” belief, which may be used as the basis for inferential decisions. Therefore, the concept of Bayesian inference can be used to provide solutions to predict future values based on historical data [13, 14, 17]. In this work, each player predicts each strategy reliability by using Bayesian inference and makes a decision for the next round game strategy. During the inference game round, the player has a chance to reconsider the current strategy with incoming information and reacts to maximize the expected payoff. According to the Bayes theorem and updating rule, the Bayesian inference formula can be expressed as follows [13, 14, 17]:(2)
P
t
H
∣
e
=
P
t
e
∣
H
×
P
t
H
P
t
e
,where P
t
(
H
∣
e
) is the posterior distribution of hypothesis H  under the evidence e and t represents tth round of game process. P
t
(
H
) and P
t
(
e
) are the prior probability of hypothesis H and evidence e, respectively. In the proposed inference game, we define n hypotheses for n payoff levels and m events for m  strategies; they are represented as follows:(3)
H
=
H
1
=
u
1
payoff  is  obtained,
payoff  is  oed
⋮
H
n
=
u
n
payoff  is  obtained
,
e
=
e
1
=
strategy
s
1
is  selected,
payoff  is  oed
⋮
e
m
=
strategy
s
m
is  selected.At each strategy, there are n mapping hypotheses about the payoff distribution; these hypotheses mean the satisfaction degrees about the selected specific strategy.At first, a player does not know the payoff propensity of each strategy but can learn it based on the Bayesian model. In the proposed scheme,P
t
(
e
j
,
1
≤
j
≤
m
) represents the percentage of strategy s
j (i.e., event e
j)’s selection; it is measured by the number of s
j’s selections divided by the total number of all strategy selections. P
t
(
H
l
,
1
≤
l
≤
n
) represents the occurrence ratio of hypothesis H
l; it is measured by the occurrence number of H
l divided by the total number of all hypotheses occurrences (t
n). P
t
(
e
j
∣
H
l
) is the event conditional probability, given the H
l selection; it can be computed as follows:(4)
P
t
e
j
∣
H
l
=
P
t
H
l
,
e
j
P
t
H
l
,
s
.
t
.
,
P
t
H
l
,
e
j
=
h
_
e
l
j
t
n
,where h
_
e
l
j is the number of strategy e
j’s selections when the H
l hypothesis occurs. Therefore, after each interaction, the player dynamically updates its corresponding event conditional probability P
t
(
e
j
∣
H
l
). Finally, the posterior probability P
t
(
H
l
∣
e
j
), which is the occurring probability of hypothesis H
l under the strategy e
j selection circumstance, can be obtained as follows: (5)
P
t
H
l
∣
e
j
=
P
t
e
j
∣
H
l
×
P
t
H
l
P
t
e
j
.Once getting the P
t
(
H
l
∣
e
j
)probability, the player can compute the probability assignment function for the (
t
+
1
)th round strategy selection (F
s
j
t
+
1
u
l). It is given by (6)
F
s
j
t
+
1
u
l
=
F
e
j
t
+
1
H
l
=
P
t
H
l
∣
e
j
,
s.t.,
s
j
∈
S
,
u
l
∈
ξ
,where e
j is the event that the strategy s
j is selected and H
l is the hypothesis that the u
l payoff is obtained (s
j
=
e
j and u
l
=
H
l). According to (6), each player can update his F
s
(
u
) values in an iterative feedback manner.


### 2.2. Uncertainty Degree and Inference Equilibrium

To accurately estimate the expected payoff, we define theuncertainty degree of each strategy. Based on E
P
I
s, we define theuncertainty degree of a specific strategy s
k (δ(s
k)) as follows: (7)
δ
s
k
=
U
m
a
x
s
k
-
U
m
i
n
s
k
max
s
∈
S
⁡
U
m
a
x
s
-
U
m
i
n
s
s
.
t
.
,
0
≤
δ
s
k
≤
1
.In order to make adaptive decisions, we need a preference ordering for strategies. To estimate a strategy preference, theexpected payoff for the strategy s
k (E
_
P
s
k) is defined according to the E
P
I
s
k anduncertainty degree (δ(s
k)):(8)
E
P
s
k
=
U
m
i
n
s
k
+
1
-
δ
s
k
×
U
m
a
x
s
k
-
U
m
i
n
s
k
.At each strategy selection time, players select their strategy to maximize E
_
P
s
k (i.e., max
s
∈
S
⁡
E
_
P
s). According to E
_
P
·, each player can compute the selection probability for the strategy s
k at the (
t
+
1
)th round (
P
t
+
1
(
s
k
)
). It is given by(9)
P
t
+
1
s
k
=
P
t
+
1
e
k
=
E
_
P
s
k
∑
s
j
∈
S
E
_
P
s
j
.
P
t
+
1
s
k represents the preference of strategy s
k at the (
t
+
1
)th game round. Therefore, based on the observation of the strategies’ pastexpected payoffs, players can update each strategy preference. With this information, the player can make a better decision for the next strategy selection.As a solution concept of inference game, we introduce theinference equilibrium(IE), which is more general than the Nash equilibrium. To define the IE, we introduce the concept ofuncertainty regret(UR); it is a method of comparing alternatives due to Savage [18]. In our approach, we first obtain theexpected payoff  for each strategy and then calculate the UR for each alternative. If there are two strategies (i.e., s
k, s
j
∈
S), the UR of strategy s
j against the strategy s
k (Λ
s
j
s
k) is given by(10)
Λ
s
j
s
k
=
E
_
P
s
k
-
U
m
i
n
s
j
.If Λ
s
j
s
k
≤
Λ
s
k
s
j, the strategy s
j is preferred to s
k by players [9]. If the maximum regret of all players is within a predefined minimum bound (ε), this strategy profile and the corresponding payoffs constitute the IE. Definition 2 mathematically expresses the IE.Definition 2.
Inference equilibrium (IE) is a strategy profile that can be obtained by repeating a symmetric game with comparing in obtaining payoffs. The IE is a refinement of the Nash equilibrium and it is associated with mixed strategy equilibriums. When a strategy profile has been chosen by all players and all the current strategies’ maximum URs are less than ε, this strategy profile and the corresponding payoffs constitute the IE. That is formally formulated as(11)
m
a
x
n
∈
N
Λ
n
∣
Λ
n
=
m
a
x
Λ
s
i
s
k
∣
s
i
,
s
k
∈
S
≤
ε
,where Λ
(
n
) is the maximum UR of the player n. Therefore, the IE is a near-Nash equilibrium; the state is that the current strategies regret of all players is within a predefined minimum bound (ε). In this study, the existence of IE strongly depends on the value of ε. According to the value size, our proposed game model can reach the IE. If ε value is very high, most strategy profiles reach the IE. If ε value is very low, that is, a negative value, all possible strategy profiles cannot reach the IE.


### 2.3. Utility Function for IoT Systems

In this paper, we develop a new sensor communication rate control scheme for IoT systems. In sensor communication, each machine device only sends or receives a small amount of data, and multiple devices can be grouped as clusters for certain management purposes. To manage such massive accesses, QoS requirements such as delay and throughput are needed for different types of sensor communication services [1–5].In the proposed scheme, we follow the assumption in [4] to implement the sensor services; ap-persistence CSMA/CA system with L classes of devices, class 1 (or
L), corresponds to the highest (orlowest) priority service. The system totally has ∑
i
=
1
L
n
i devices, where n
i represents the number of the ith class devices. The traffic activities of the ith class devices follow the Poisson process with mean arrival rate λ
i and departure rate μ
i. In principle, the setting of parameter p in p-persistent CSMA/CA is equivalent to tuning the size of backoff window in CSMA/CA. If the channel is idle, the device will transmit a packet with probability p
i when new time slot commences. Otherwise, it will wait until the channel is idle [4]. By varying the parameter p
i for the ith class devices, differential QoS provisioning could be easily achieved. For simplicity, we suppose an M/D/1 queuing model with no packet collisions. Therefore, the average output packet rate of the queuing system is equal to the input rate λ
i. Let T
s
i denote the transmission time of a class i device, and the time fraction of that device occupying the channel is given by (λ
i
×
T
s
i). Let ϱ
i represent the probability that the channel is idle for a device of class i in a given slot [4]:(12)
ϱ
i
=
1
-
∑
j
=
1
,
j
≠
i
L
n
j
×
λ
j
×
T
s
j
-
n
i
-
1
×
λ
i
×
T
s
i
.For the device of class i, the transmission probability in an arbitrary slot is represented by (ϱ
i
×
p
i). Following the M/D/1 queuing model, the average service rate of the ith class device (μ
i) and the queuing delay (W
Q
i) is given by(13)
μ
i
=
ϱ
i
×
p
i
T
s
i
,
W
Q
i
=
ρ
i
2
×
μ
i
×
1
-
ρ
i
.Consequently, the total delay of the ith class device (d
i) is given by(14)
d
i
=
W
Q
i
+
W
S
i
=
2
-
ρ
i
2
×
μ
i
×
1
-
ρ
i
s
.
t
.
,
W
S
i
=
1
μ
i
,where the average service time W
S
i is the inverse of the average service rate [4]. Let ρ
i
=
λ
i
/
μ
i denote the utilization coefficient of the ith class of devices. Finally, d
i can be obtained as follows:(15)
d
i
=
2
-
λ
i
/
μ
i
2
×
μ
i
×
1
-
λ
i
/
μ
i
=
2
×
μ
i
-
λ
i
2
×
μ
i
2
-
λ
i
×
μ
i
s
.
t
.
,
ρ
i
=
λ
i
μ
i
≪
1
.In this work, the rate control process is formulated as ann-player inference game. The packet transmission contending devices in IoT are game players and each device has its own class. As a game player, a class i device selects a rate λ
i in its strategy space R
i
∈
λ
m
i
n
i
,
λ
m
a
x
i to send packets, and then it will gain a payoff according to the selected strategy. In the proposed game model, R
i
≅
S and λ
m
i
n
i and λ
m
a
x
i are s
1 and s
n, respectively. Therefore, available strategies in R are defined as discrete and multiple packet transmission rates. The general system model of our proposed scheme is shown in Figure 1.Figure 1
System model for sensor communications.Utility functions quantitatively describe the players’ degree of satisfaction with respect to their action in the game. In the proposed model, the utility function is defined by(16)
u
i
=
λ
i
-
ω
i
×
d
i
,
s
.
t
.
,
ω
i
>
0
,where a tunable parameter ω
i indicates the relative importance weight (delay versus transmission rate) of the ith class devices. To allow the differential QoS provisioning, the higher priority applications have a larger ω value and the lower priority applications have a smaller ω value. By combining (12)–(16), we obtain the explicit expression of the utility function as follows:(17)
u
i
=
λ
i
-
ω
i
×
2
μ
i
-
λ
i
2
μ
i
2
-
λ
i
μ
i
=
λ
i
-
ω
i
×
2
×
ξ
i
×
p
i
/
T
s
i
-
λ
i
2
ξ
i
×
p
i
/
T
s
i
2
-
λ
i
×
ξ
i
×
p
i
/
T
s
i
=
λ
i
+
w
i
×
T
s
i
×
T
s
i
×
λ
i
-
2
×
p
i
×
ξ
i
2
×
p
i
×
p
i
×
ξ
i
+
T
s
i
×
λ
i
×
ξ
i
.From (17), we know the utility function is actually a function of transmission rate λ
i for all services. Finally, the payoff of the ith class devices depends on not only its own strategy but also the other players’ strategies. Therefore, it is represented by u
i
(
λ
i
,
λ
-
i
), where λ
-
i is the set of strategies of all devices without the device i. In the inference game, all the devices aim to maximize their payoffs (i.e., maximizing the transmission rate while minimizing the access delay). Let u
i
∗
(
λ
i
,
λ
-
i
) be the maximum payoff for the device i, and it is used as an index to classify received payoffs into four categories:bad,average,good, andexcellentsatisfaction levels. In our scheme, there is a one-to-one relationship between each category and hypothesis. Therefore, each hypothesis represents the “level of satisfaction.” According to (17), each category can be mapped into each hypothesis (H) as follows:(18)
H
=
H
1
=
excellent  payoff  is  gained
,
if
u
i
C
λ
i
,
λ
-
i
>
Ω
1
×
u
i
∗
λ
i
,
λ
-
i
,
H
2
=
good  payoff  is  gained
,
if
Ω
1
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
>
Ω
2
×
u
i
∗
λ
i
,
λ
-
i
,
H
3
=
average  payoff  is  gained
,
if
Ω
2
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
>
Ω
3
×
u
i
∗
λ
i
,
λ
-
i
,
H
4
=
bad  payoff  is  gained
,
if
Ω
3
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
,where  u
i
C
λ
i
,
λ
-
i is the currently obtained payoff and Ω
i
,
1
≤
i
≤
3 is a threshold parameter for the event classification.


### 2.4. The Main Steps of the Proposed Scheme for IoT Systems

The main contribution of this study lies in defining a new inference game model. In contrast with classical games, the proposed inference game allows a player belief concerning the possible payoffs of each strategy profile; it is represented by an imprecise probability. On the basis of inference game model, we develop a new packet rate control scheme for sensor communications. To evolve into an inference equilibrium, we can capture how devices adapt their strategies based on the iterative feedback process. At the end of each game iteration, players evaluate their current payoff, analyze strategic interactions, and modify the probability of each hypothesis, probability assignment, and the uncertainty degree. In an entirely distributed fashion, this interactive feedback process continues until IoT system reaches the inference equilibrium; this practical and suitable approach can clarify how to better manage the packet rate control decisions in real-world IoT operations. The main steps of the proposed rate control algorithm are given next.Step 1.
At the initial iteration (t
=
1), the probability distributions of P
H, P
(
e
), and P
(
e
∣
H
) are equally distributed. One has (P
t
=
1
H
l
,
1
≤
l
≤
n
=
1
/
n, P
t
=
1
e
j
,
1
≤
j
≤
m
=
1
/
m, and P
t
=
1
(
e
∣
H
)
=
P
t
=
1
e
j
,
1
≤
j
≤
m
/
P
t
=
1
H
l
,
1
≤
l
≤
n). This starting guess guarantees that each strategy enjoys the same benefit at the beginning of the game.Step 2.
As a game player, each machine device selects its packet rate strategy inξ according to (9); it represents the trustworthiness distribution of each strategy preference.Step 3.
At each game round, devices obtain their payoffs based on (12)–(17) in a distributed manner. And then, devices reestimate the selected strategy and modify the probabilities of P
H, P
(
e
), P
(
e
∣
H
), and F
s
(
u
) and adjust E
P
I
·, δ
·, and E
_
P
· values.Step 4.
By using (10) and (11), we can estimate the maximum UR of all current strategies. If it is less than a predefined minimum bound (ε), all players do not change their current strategy; this strategy profile and the corresponding payoffs constitute theinference equilibrium.Step 5.
Proceed to Step2 for the next game iteration. This iterative feedback procedure continues under IoT system dynamics.


## 2.1. Inference Game Model and Inference Process

To model strategic interactive situations involving uncertainty, we develop a new inference game, which is constructed based on the assumption of a player belief regarding the uncertain payoffs. Therefore, an imprecise probability distribution over the set of the possible payoffs is assigned based on the player belief. This means that the game players are not sure about the payoffs of each strategy but assign a set of possible payoffs to each strategy profile. To effectively expect the possible payoffs, we apply some original ideas taken from the Bayesian inference process. For the modeling of uncertainty, our approach has become a key challenge in the real-world decision problems. In this section, we define our inference game model (G) as follows.Definition 1.
An inference game model constitutes a 5-tupleG = (N, S, ξ, F
s, δ
s), where (i)
Nis a set of game players,
(ii)
S
=
s
1
,
s
2
,
…
,
s
n is a nonempty finite set of all pure strategies of players,
(iii)
ξ
=
u
1
,
u
2
,
…
,
u
m is the utility set of all consequent payoffs of each strategy; it is defined as discrete satisfaction levels of players where u
i
,
1
≤
i
≤
m
∈
R; there are m level satisfactions,
(iv)
F
s
=
{
F
s
(
u
)
∣
s
∈
S
,
u
∈
ξ
}  is a probability assignment function, which maps the strategy choice s over the strategy set S onto a consequence u over the strategy set ξ, where F
s
:
F
s
u
→
[
0,1
] and ∑
u
∈
ξ
F
s
(
u
)
=
1,
(v)
δ
s is the uncertainty degree of consequence, which could be caused by strategy choice s  (
0
≤
δ
s
≤
1
).During the inference game process, a strategy s
k
,
1
≤
k
≤
n can cause a consequence u
i
,
1
≤
i
≤
m that is specified by the mapping probability function F
s
k. According to the consequent payoffs,Expected Payoff Interval for the strategy s
k (E
P
I
s
k) is defined as follows:(1)
E
P
I
s
k
=
U
m
i
n
s
k
,
U
m
a
x
s
k
,
s
.
t
.
,
U
m
i
n
s
k
=
m
i
n
u
i
∈
ξ
F
s
k
u
i
×
u
i
,
U
m
a
x
s
k
=
m
a
x
u
i
∈
ξ
F
s
k
u
i
×
u
i
.Under uncertain situations, F
s function is essential for the decision making. In this paper, F
s represents the player belief for outcomes of each strategy. To dynamically adapt the current situation, F
s is updated as new observations become available. Therefore, it is necessary to adopt a scientific inference method in order to adaptively modify F
s.Bayesian inference is a method of statistical inference to provide a logical, quantitative decision. Based on the Bayes theorem and Bayesian probability rules, Bayesian inference summarizes all uncertainty by a “posterior” distribution and gives a “posterior” belief, which may be used as the basis for inferential decisions. Therefore, the concept of Bayesian inference can be used to provide solutions to predict future values based on historical data [13, 14, 17]. In this work, each player predicts each strategy reliability by using Bayesian inference and makes a decision for the next round game strategy. During the inference game round, the player has a chance to reconsider the current strategy with incoming information and reacts to maximize the expected payoff. According to the Bayes theorem and updating rule, the Bayesian inference formula can be expressed as follows [13, 14, 17]:(2)
P
t
H
∣
e
=
P
t
e
∣
H
×
P
t
H
P
t
e
,where P
t
(
H
∣
e
) is the posterior distribution of hypothesis H  under the evidence e and t represents tth round of game process. P
t
(
H
) and P
t
(
e
) are the prior probability of hypothesis H and evidence e, respectively. In the proposed inference game, we define n hypotheses for n payoff levels and m events for m  strategies; they are represented as follows:(3)
H
=
H
1
=
u
1
payoff  is  obtained,
payoff  is  oed
⋮
H
n
=
u
n
payoff  is  obtained
,
e
=
e
1
=
strategy
s
1
is  selected,
payoff  is  oed
⋮
e
m
=
strategy
s
m
is  selected.At each strategy, there are n mapping hypotheses about the payoff distribution; these hypotheses mean the satisfaction degrees about the selected specific strategy.At first, a player does not know the payoff propensity of each strategy but can learn it based on the Bayesian model. In the proposed scheme,P
t
(
e
j
,
1
≤
j
≤
m
) represents the percentage of strategy s
j (i.e., event e
j)’s selection; it is measured by the number of s
j’s selections divided by the total number of all strategy selections. P
t
(
H
l
,
1
≤
l
≤
n
) represents the occurrence ratio of hypothesis H
l; it is measured by the occurrence number of H
l divided by the total number of all hypotheses occurrences (t
n). P
t
(
e
j
∣
H
l
) is the event conditional probability, given the H
l selection; it can be computed as follows:(4)
P
t
e
j
∣
H
l
=
P
t
H
l
,
e
j
P
t
H
l
,
s
.
t
.
,
P
t
H
l
,
e
j
=
h
_
e
l
j
t
n
,where h
_
e
l
j is the number of strategy e
j’s selections when the H
l hypothesis occurs. Therefore, after each interaction, the player dynamically updates its corresponding event conditional probability P
t
(
e
j
∣
H
l
). Finally, the posterior probability P
t
(
H
l
∣
e
j
), which is the occurring probability of hypothesis H
l under the strategy e
j selection circumstance, can be obtained as follows: (5)
P
t
H
l
∣
e
j
=
P
t
e
j
∣
H
l
×
P
t
H
l
P
t
e
j
.Once getting the P
t
(
H
l
∣
e
j
)probability, the player can compute the probability assignment function for the (
t
+
1
)th round strategy selection (F
s
j
t
+
1
u
l). It is given by (6)
F
s
j
t
+
1
u
l
=
F
e
j
t
+
1
H
l
=
P
t
H
l
∣
e
j
,
s.t.,
s
j
∈
S
,
u
l
∈
ξ
,where e
j is the event that the strategy s
j is selected and H
l is the hypothesis that the u
l payoff is obtained (s
j
=
e
j and u
l
=
H
l). According to (6), each player can update his F
s
(
u
) values in an iterative feedback manner.


## 2.2. Uncertainty Degree and Inference Equilibrium

To accurately estimate the expected payoff, we define theuncertainty degree of each strategy. Based on E
P
I
s, we define theuncertainty degree of a specific strategy s
k (δ(s
k)) as follows: (7)
δ
s
k
=
U
m
a
x
s
k
-
U
m
i
n
s
k
max
s
∈
S
⁡
U
m
a
x
s
-
U
m
i
n
s
s
.
t
.
,
0
≤
δ
s
k
≤
1
.In order to make adaptive decisions, we need a preference ordering for strategies. To estimate a strategy preference, theexpected payoff for the strategy s
k (E
_
P
s
k) is defined according to the E
P
I
s
k anduncertainty degree (δ(s
k)):(8)
E
P
s
k
=
U
m
i
n
s
k
+
1
-
δ
s
k
×
U
m
a
x
s
k
-
U
m
i
n
s
k
.At each strategy selection time, players select their strategy to maximize E
_
P
s
k (i.e., max
s
∈
S
⁡
E
_
P
s). According to E
_
P
·, each player can compute the selection probability for the strategy s
k at the (
t
+
1
)th round (
P
t
+
1
(
s
k
)
). It is given by(9)
P
t
+
1
s
k
=
P
t
+
1
e
k
=
E
_
P
s
k
∑
s
j
∈
S
E
_
P
s
j
.
P
t
+
1
s
k represents the preference of strategy s
k at the (
t
+
1
)th game round. Therefore, based on the observation of the strategies’ pastexpected payoffs, players can update each strategy preference. With this information, the player can make a better decision for the next strategy selection.As a solution concept of inference game, we introduce theinference equilibrium(IE), which is more general than the Nash equilibrium. To define the IE, we introduce the concept ofuncertainty regret(UR); it is a method of comparing alternatives due to Savage [18]. In our approach, we first obtain theexpected payoff  for each strategy and then calculate the UR for each alternative. If there are two strategies (i.e., s
k, s
j
∈
S), the UR of strategy s
j against the strategy s
k (Λ
s
j
s
k) is given by(10)
Λ
s
j
s
k
=
E
_
P
s
k
-
U
m
i
n
s
j
.If Λ
s
j
s
k
≤
Λ
s
k
s
j, the strategy s
j is preferred to s
k by players [9]. If the maximum regret of all players is within a predefined minimum bound (ε), this strategy profile and the corresponding payoffs constitute the IE. Definition 2 mathematically expresses the IE.Definition 2.
Inference equilibrium (IE) is a strategy profile that can be obtained by repeating a symmetric game with comparing in obtaining payoffs. The IE is a refinement of the Nash equilibrium and it is associated with mixed strategy equilibriums. When a strategy profile has been chosen by all players and all the current strategies’ maximum URs are less than ε, this strategy profile and the corresponding payoffs constitute the IE. That is formally formulated as(11)
m
a
x
n
∈
N
Λ
n
∣
Λ
n
=
m
a
x
Λ
s
i
s
k
∣
s
i
,
s
k
∈
S
≤
ε
,where Λ
(
n
) is the maximum UR of the player n. Therefore, the IE is a near-Nash equilibrium; the state is that the current strategies regret of all players is within a predefined minimum bound (ε). In this study, the existence of IE strongly depends on the value of ε. According to the value size, our proposed game model can reach the IE. If ε value is very high, most strategy profiles reach the IE. If ε value is very low, that is, a negative value, all possible strategy profiles cannot reach the IE.


## 2.3. Utility Function for IoT Systems

In this paper, we develop a new sensor communication rate control scheme for IoT systems. In sensor communication, each machine device only sends or receives a small amount of data, and multiple devices can be grouped as clusters for certain management purposes. To manage such massive accesses, QoS requirements such as delay and throughput are needed for different types of sensor communication services [1–5].In the proposed scheme, we follow the assumption in [4] to implement the sensor services; ap-persistence CSMA/CA system with L classes of devices, class 1 (or
L), corresponds to the highest (orlowest) priority service. The system totally has ∑
i
=
1
L
n
i devices, where n
i represents the number of the ith class devices. The traffic activities of the ith class devices follow the Poisson process with mean arrival rate λ
i and departure rate μ
i. In principle, the setting of parameter p in p-persistent CSMA/CA is equivalent to tuning the size of backoff window in CSMA/CA. If the channel is idle, the device will transmit a packet with probability p
i when new time slot commences. Otherwise, it will wait until the channel is idle [4]. By varying the parameter p
i for the ith class devices, differential QoS provisioning could be easily achieved. For simplicity, we suppose an M/D/1 queuing model with no packet collisions. Therefore, the average output packet rate of the queuing system is equal to the input rate λ
i. Let T
s
i denote the transmission time of a class i device, and the time fraction of that device occupying the channel is given by (λ
i
×
T
s
i). Let ϱ
i represent the probability that the channel is idle for a device of class i in a given slot [4]:(12)
ϱ
i
=
1
-
∑
j
=
1
,
j
≠
i
L
n
j
×
λ
j
×
T
s
j
-
n
i
-
1
×
λ
i
×
T
s
i
.For the device of class i, the transmission probability in an arbitrary slot is represented by (ϱ
i
×
p
i). Following the M/D/1 queuing model, the average service rate of the ith class device (μ
i) and the queuing delay (W
Q
i) is given by(13)
μ
i
=
ϱ
i
×
p
i
T
s
i
,
W
Q
i
=
ρ
i
2
×
μ
i
×
1
-
ρ
i
.Consequently, the total delay of the ith class device (d
i) is given by(14)
d
i
=
W
Q
i
+
W
S
i
=
2
-
ρ
i
2
×
μ
i
×
1
-
ρ
i
s
.
t
.
,
W
S
i
=
1
μ
i
,where the average service time W
S
i is the inverse of the average service rate [4]. Let ρ
i
=
λ
i
/
μ
i denote the utilization coefficient of the ith class of devices. Finally, d
i can be obtained as follows:(15)
d
i
=
2
-
λ
i
/
μ
i
2
×
μ
i
×
1
-
λ
i
/
μ
i
=
2
×
μ
i
-
λ
i
2
×
μ
i
2
-
λ
i
×
μ
i
s
.
t
.
,
ρ
i
=
λ
i
μ
i
≪
1
.In this work, the rate control process is formulated as ann-player inference game. The packet transmission contending devices in IoT are game players and each device has its own class. As a game player, a class i device selects a rate λ
i in its strategy space R
i
∈
λ
m
i
n
i
,
λ
m
a
x
i to send packets, and then it will gain a payoff according to the selected strategy. In the proposed game model, R
i
≅
S and λ
m
i
n
i and λ
m
a
x
i are s
1 and s
n, respectively. Therefore, available strategies in R are defined as discrete and multiple packet transmission rates. The general system model of our proposed scheme is shown in Figure 1.Figure 1
System model for sensor communications.Utility functions quantitatively describe the players’ degree of satisfaction with respect to their action in the game. In the proposed model, the utility function is defined by(16)
u
i
=
λ
i
-
ω
i
×
d
i
,
s
.
t
.
,
ω
i
>
0
,where a tunable parameter ω
i indicates the relative importance weight (delay versus transmission rate) of the ith class devices. To allow the differential QoS provisioning, the higher priority applications have a larger ω value and the lower priority applications have a smaller ω value. By combining (12)–(16), we obtain the explicit expression of the utility function as follows:(17)
u
i
=
λ
i
-
ω
i
×
2
μ
i
-
λ
i
2
μ
i
2
-
λ
i
μ
i
=
λ
i
-
ω
i
×
2
×
ξ
i
×
p
i
/
T
s
i
-
λ
i
2
ξ
i
×
p
i
/
T
s
i
2
-
λ
i
×
ξ
i
×
p
i
/
T
s
i
=
λ
i
+
w
i
×
T
s
i
×
T
s
i
×
λ
i
-
2
×
p
i
×
ξ
i
2
×
p
i
×
p
i
×
ξ
i
+
T
s
i
×
λ
i
×
ξ
i
.From (17), we know the utility function is actually a function of transmission rate λ
i for all services. Finally, the payoff of the ith class devices depends on not only its own strategy but also the other players’ strategies. Therefore, it is represented by u
i
(
λ
i
,
λ
-
i
), where λ
-
i is the set of strategies of all devices without the device i. In the inference game, all the devices aim to maximize their payoffs (i.e., maximizing the transmission rate while minimizing the access delay). Let u
i
∗
(
λ
i
,
λ
-
i
) be the maximum payoff for the device i, and it is used as an index to classify received payoffs into four categories:bad,average,good, andexcellentsatisfaction levels. In our scheme, there is a one-to-one relationship between each category and hypothesis. Therefore, each hypothesis represents the “level of satisfaction.” According to (17), each category can be mapped into each hypothesis (H) as follows:(18)
H
=
H
1
=
excellent  payoff  is  gained
,
if
u
i
C
λ
i
,
λ
-
i
>
Ω
1
×
u
i
∗
λ
i
,
λ
-
i
,
H
2
=
good  payoff  is  gained
,
if
Ω
1
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
>
Ω
2
×
u
i
∗
λ
i
,
λ
-
i
,
H
3
=
average  payoff  is  gained
,
if
Ω
2
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
>
Ω
3
×
u
i
∗
λ
i
,
λ
-
i
,
H
4
=
bad  payoff  is  gained
,
if
Ω
3
×
u
i
∗
λ
i
,
λ
-
i
≥
u
i
C
λ
i
,
λ
-
i
,where  u
i
C
λ
i
,
λ
-
i is the currently obtained payoff and Ω
i
,
1
≤
i
≤
3 is a threshold parameter for the event classification.


## 2.4. The Main Steps of the Proposed Scheme for IoT Systems

The main contribution of this study lies in defining a new inference game model. In contrast with classical games, the proposed inference game allows a player belief concerning the possible payoffs of each strategy profile; it is represented by an imprecise probability. On the basis of inference game model, we develop a new packet rate control scheme for sensor communications. To evolve into an inference equilibrium, we can capture how devices adapt their strategies based on the iterative feedback process. At the end of each game iteration, players evaluate their current payoff, analyze strategic interactions, and modify the probability of each hypothesis, probability assignment, and the uncertainty degree. In an entirely distributed fashion, this interactive feedback process continues until IoT system reaches the inference equilibrium; this practical and suitable approach can clarify how to better manage the packet rate control decisions in real-world IoT operations. The main steps of the proposed rate control algorithm are given next.Step 1.
At the initial iteration (t
=
1), the probability distributions of P
H, P
(
e
), and P
(
e
∣
H
) are equally distributed. One has (P
t
=
1
H
l
,
1
≤
l
≤
n
=
1
/
n, P
t
=
1
e
j
,
1
≤
j
≤
m
=
1
/
m, and P
t
=
1
(
e
∣
H
)
=
P
t
=
1
e
j
,
1
≤
j
≤
m
/
P
t
=
1
H
l
,
1
≤
l
≤
n). This starting guess guarantees that each strategy enjoys the same benefit at the beginning of the game.Step 2.
As a game player, each machine device selects its packet rate strategy inξ according to (9); it represents the trustworthiness distribution of each strategy preference.Step 3.
At each game round, devices obtain their payoffs based on (12)–(17) in a distributed manner. And then, devices reestimate the selected strategy and modify the probabilities of P
H, P
(
e
), P
(
e
∣
H
), and F
s
(
u
) and adjust E
P
I
·, δ
·, and E
_
P
· values.Step 4.
By using (10) and (11), we can estimate the maximum UR of all current strategies. If it is less than a predefined minimum bound (ε), all players do not change their current strategy; this strategy profile and the corresponding payoffs constitute theinference equilibrium.Step 5.
Proceed to Step2 for the next game iteration. This iterative feedback procedure continues under IoT system dynamics.


## 3. Performance Evaluation

In this section, the performance of proposed sensor communication algorithm is evaluated through a simulation model; we model different random traffic services for machine devices in each iteration. With a simulation study, we compare the performance of our scheme with other existing schemes [4, 5] and can confirm the performance superiority of the proposed approach. To ensure the simulation model is sufficiently generic to be valid in a real-world IoT system, the assumptions implemented in our model are as follows:(i)
Machine devices access IoT system in ap-persistent CSMA/CA manner.
(ii)
There are 10 machine nodes and 4 different (I, II, III, and IV) traffic classes in the system.
(iii)
The class of each device is randomly decided.
(iv)
Pure strategy set of players isS
=
s
1
,
s
2
,
s
3
,
s
4
,
s
5  and s
1
= 30 Kbps, s
2
= 40 Kbps, s
3 = 50 Kbps, s
4 = 60 Kbps, and s
5 = 70 Kbps.
(v)
The process for new connection services is Poisson with rateΦ (connections/s), and the range of offered traffic load was varied from 0 to 3.0.
(vi)
The capacity of network bandwidth is 10 Mbps, and each message consists of Variable Bit Rate (VBR) packets.
(vii)
The predefined minimum bound (ε) is defined as 0.1
×
max
⁡
U
m
a
x
s
-
U
m
i
n
s.
(viii)
Network performance measures obtained on the basis of 50 simulation runs are plotted as a function of the offered traffic load.
(ix)
The size of messages is exponentially distributed with different means for different message applications.
(x)
The queuing model is the M/D/1 and buffer length of each device is 50 packets.There are other performance analysis methods: theoretical or numerical analysis. However, these methods have limited modeling possibility and cannot provide precise performance evaluation. In contrast to these methods, our simulation model can implement a complex realistic model for one real-world IoT system. Details of simulation parameters are listed in Table 1.Table 1
System parameters used in the simulation experiments.
Traffic class
Message application
Bandwidth requirement
CSMA parameter
Connection duration and minimum requirement
1
TDM voice messages
32 Kbps
0.08
30 sec (0.5 min)/32 Kbps
2
Audio/video messages
48 Kbps
0.07
120 sec (2 min)/24 Kbps
64 Kbps
0.07
180 sec (3 min)/32 Kbps
3
File transfer messages
48 Kbps
0.06
120 sec (2 min)/16 Kbps
64 Kbps
0.06
180 sec (3 min)/24 Kbps
4
Web browsing messages
32 Kbps
0.05
300 sec (5 min)/16 Kbps
64 Kbps
0.05
120 sec (2 min)/24 Kbps
Parameter
Value
Description
N
10
A number of game players (machine devices)
n
5
A number of strategies
m
4
A number of consequent payoff levels
L
4
A number of traffic service classes
p
i
0
≤
p
i
≤
1
Packet transmission probability
λ
min
⁡
i
,
λ
max
⁡
i
30 Kbps, 70 Kbps
The minimum and maximum packet transmission rates
ω
1
,
ω
2
,
ω
3
,
ω
4
1, 0.8, 0.6, and 0.4
Relative importance weight for each class device
Ω
1
,
Ω
2
,
Ω
3
0.8, 0.5, and 0.2
Threshold parameters for the event classificationPerformance measures obtained through simulation are QoS violation ratio, normalized packet delay, payoffs, and network throughput, and so forth. In this paper, we compare the performance of the proposed scheme with existing schemes, the DRAC scheme [4] and CBPS scheme [5]. These existing schemes are also developed as effective sensor communication schemes that capture the notion of packet rate control mechanisms.Figure2 shows the performance comparison in terms of QoS violation ratio with different service connection request rates. In this paper, the QoS violation ratio represents the minimum bandwidth requirement violation probabilities (i.e., the fraction of traffic that does not meet its minimum bandwidth bound). The main observation is that the proposed scheme is able to keep a relatively lower QoS violation ratio than other DRAC and CBPS schemes. This property may be useful to the IoT system operators if the relative penalty was incurred from failure to meet the strict QoS requirements.Figure 2
QoS violation ratio.Figure3 demonstrates the performance of each scheme in terms of the normalized end-to-end packet delay. This performance criterion indicates the effectiveness on QoS guarantees of the sensor communication. We can observe that the proposed scheme effectively maintains the lower packet delay; it illustrates the adaptiveness of our control approach under widely different and diversified traffic load situations.Figure 3
Normalized packet delay.In Figure4, the comparison of the normalized payoff is shown. In this work, it is defined as an average utility of players, which is calculated that the payoffs are aggregated per player and then normalized. Due to the inclusion of the adaptive feedback interactive mechanism, the proposed scheme can adapt the current system situation. From the simulation results we obtained, it can be seen that the proposed scheme, in general, can gain better payoffs than other schemes.Figure 4
Normalized payoffs per player.Figure5 shows the normalized IoT system throughput. In IoT system operations, normalized throughput is the rate of successful message delivery over sensor communication. All the schemes have similar trends. As the traffic load increases, system throughput increases linearly. Under various system traffic load conditions, the proposed scheme exhibits superior performance compared with the other existing schemes.Figure 5
Normalized IoT system throughput.Among the results obtained in our simulation model, it is particularly important to mention that our inference game approach concerning how to manage limited information significantly improves the QoS provisioning and IoT system performance. The main goal of our proposed scheme is to provide a better throughput for sensor communication while ensuring the required QoS. To achieve an appropriate network performance, the proposed scheme constantly monitors the current IoT system conditions and can adaptively adjust the packet transmission rate in an interactive feedback manner. Therefore, it is shown that our inference game model plays an important role in sensor communication.


## 4. Summary and Conclusions

In this study, we investigate uncertainty-control game and packet transmission rate control scheme for IoT systems. With the uncertainty about payoffs, we develop a new inference game model and then reveal how ambiguity degrees of belief about consequences impact the outcomes of a game. On the basis of our inference game, we design a new sensor communication scheme. The simulation results show that the usage of the inference game model is indeed beneficial in improving the IoT system performance. We believe that the issue addressed in this study will be increasingly important with the proliferation of IoT applications. In the future, it is interesting to extend our inference game model to various decision-making processes.



---

*Source: 127349-2015-10-04.xml*