# Performance Analysis of Heterogeneous Data Centers in Cloud Computing Using a Complex Queuing Model

Authors: Wei-Hua Bai; Jian-Qing Xi; Jia-Xian Zhu; Shao-Wei Huang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Performance evaluation of modern cloud data centers has attracted considerable research attention among both cloud providers and cloud customers. In this paper, we investigate the heterogeneity of modern data centers and the service process used in these heterogeneous data centers. Using queuing theory, we construct a complex queuing model composed of two concatenated queuing systems and present this as an analytical model for evaluating the performance of heterogeneous data centers. Based on this complex queuing model, we analyze the mean response time, the mean waiting time, and other important performance indicators. We also conduct simulation experiments to confirm the validity of the complex queuing model. We further conduct numerical experiments to demonstrate that the traffic intensity (or utilization) of each execution server, as well as the configuration of server clusters, in a heterogeneous data center will impact the performance of the system. Our results indicate that our analytical model is effective in accurately estimating the performance of the heterogeneous data center.


---

## Body

## 1. Introduction

Cloud computing is a popular paradigm for providing services to users via three fundamental models: Infrastructure as a service (IaaS), Platform as a service (PaaS), and Software as a service (SaaS) [1, 2]. Cloud computing providers offer computing resources (e.g., servers, storage, networks, development platforms, and applications) to users either elastically or dynamically, according to user-demand and form of payment (e.g., pay-as-you-go) [3]. A modern data center is considered a heterogeneous environment because it contains many generations of servers with hardware configurations that may differ, especially in terms of the speed and capacity of the processors. Generally, these servers have been added to the data center gradually and provisioned to replace the existing (or “legacy”) machines already in the data center’s infrastructure [4, 5]. The heterogeneity of this mix of machine platforms affects the performance of data centers and the execution of cloud computing applications.The ability to ensure the desired Quality of Service (QoS), which is a standard element of the Service Level Agreement (SLA) established between consumers and cloud providers, is one the most important business considerations for a cloud computing provider [6, 7]. A typical QoS outlines a set of critical performance indicators: mean response time, mean task queuing length, mean waiting time, mean task number of throughput, task capacity of the system, blocking probability, and probability of immediate service of the system. All of these indicators can be analyzed and described using queuing theory [8].A task completed by the cloud computing center follows several steps:(
1
) a customer’s request is transformed into a cloud computing task and sent to the task queue (the first level queue) maintained by the main scheduler server; (
2
) the scheduler allocates computing resources for the task and dispatches it to a node (execution server) to execute; (
3
) the node takes the task from the second level queue and allocates space in a CPU core (the main computing resource) to complete it; (
4
) the system outputs the result, and the task leaves the system. In this process, blocking will occur if a task leaves the system without having been executed (or the task may have been abandoned by the system) because of capacity constraint at the first level queue. A queuing model for a cloud computing heterogeneous data center can analyze the relationship among three factors: the arrival rate of tasks, the core utilization, and the probability of a node (execution server) being dispatched. Cloud providers can utilize this information to analyze their systems’ management and optimize their allocation of computing resources [9]. As we show in Section 2, much existing research has discussed the problem of how to analyze data center performance. However, these studies have not considered the following important issues:(i)
A cloud center can include a large number of servers of different generations with different CPU speeds and processor capacities. Existing research rarely considers cloud center heterogeneity; rather, studies generally assume that every node operates at the same speed.
(ii)
A cloud system has at least two levels of scheduling processes (service processes): the main scheduler, which allocates computing resources, and the second scheduler (a node/execution server), which executes tasks. The arrival rate of a task to the main scheduler versus to a node is different, and this must be taken into account in the queuing analysis.
(iii)
A node server with more than one speed and processor capacity will have different probabilities of being dispatched, depending on the situation. Each node has its own task queue to maintain, which is tracked by a separate queuing model that is included in the overall system queuing model.
(iv)
A state-of-the-art resource allocation scheme based on the utilization threshold[
δ
l
,
δ
h
] controls nodes that are to be added to a cloud system or removed from it [10].To fill these gaps, in this paper, we employ a complex queuing model to investigate the performance of a cloud center and the relationship among various performance indicators in a heterogeneous data center. We aim to make the following contributions with this paper:(1)
We model the heterogeneous data center as a complex queuing system to characterize the service process. Two concatenated queuing systems compose the complex queuing model. The first is the finite capacity scheduler queuing system, which is used to study the main scheduling server. The second is the execution queuing system, which has multiple servers for each multicore server processor.
(2)
Based on this complex queuing model, and using queuing theory, we analyze the real output rate of the main scheduling server—that is to say, the real arrival rate of the execution queuing systems, the mean response time of the system, the blocking probability, and other performance indicators—and use this analysis to evaluate the performance of, and the relationships among, these indicators in a cloud center.
(3)
We conduct extensive discrete event simulations to evaluate and validate the results of using the complex queuing model to analyze the performance of heterogeneous data centers in cloud computing.The remainder of this paper is organized as follows. Section2 reviews and discusses the related scholarship on queuing models for cloud centers. Section 3 discusses our analytical model in detail. All of the performance metrics obtained by using this complex queuing model are presented in Section 4. In Section 5, we show the simulation results and compare them with our analytical results to evaluate and validate the applicability of the complex queuing model. Finally, conclusions and future work are presented in Section 6.


## 2. Related Work

Although much scholarship has been conducted on the performance analysis of a cloud center based on queuing models [11–16], factors including the loosely coupled architectures of cloud computing and the heterogeneity and dynamic infrastructure of a cloud center have not been considered.In [11], the authors used an M
/
G
/
m
/
m
+
r queuing system to evaluate a cloud computing center and obtained a complete probability distribution of response time, number of tasks in the system, and other important performance metrics. They also considered the different mean waiting times between heterogeneous services and homogeneous services under the same conditions in the system. Based on queuing theory and open Jackson’s networks, a combination of M
/
M
/
1 and M
/
M
/
m was presented to model the cloud platform for QoS requirements in [12]. In [13], the authors presented an M
/
M
/
m queuing system and proposed a synthesis optimization of model, function, and strategy to optimize the performance of services in the cloud center. In [14, 15], the authors introduced a queuing model consisting of three concatenated queuing systems (the schedule queue, M
/
M
/
1, the computation queue, M
/
M
/
m, and the transmission queue, M
/
M
/
1) to characterize the service process in a multimedia cloud and to investigate the resource optimization problems of multimedia cloud computing. In [16], a cloud center was modeled as a G
I
X
/
M
/
S
/
N queuing model to discuss cloud service performance related to fault recovery.In many existing methodologies [17–20], queuing theory has been used to analyze or optimize factors such as power allocation, load distribution, and profit control.In [17], the authors presented an energy proportional model—which treats a server as an M
/
G
/
1 queuing system—as a strategy to improve performance efficiency. In [18], an M
/
M
/
m queuing model, which considers factors such as the requirement γ of a service and the configuration of a multiserver system, is used to optimize multiserver configuration for profit maximization in cloud computing. In [19], the authors presented an M
/
M
/
r
/
k queuing system to model a cloud server farm provided with finite capacity for profit optimization. Another study presents a queuing model for a group of heterogeneous multicore servers with different sizes and speeds to optimize power allocation and load distribution in a cloud computing environment [20]. In [17], the authors used the M
/
M
/
c
/
c queuing system to model a cloud center with different priority classes to analyze the general problem of resource deployment within cloud computing.However, none of the above literature considers that the main scheduler and the execution servers have their own task queuing, that an execution server with different speeds will have different processing times, and that each execution server has a different probability of being dispatched given different arrival rates of tasks. As a result, a queuing model used to model a heterogeneous data center for performance analysis should characterize the service process within the cloud center as well as the infrastructural architecture of a data center.


## 3. Queuing Model for Performance Analysis

In this section, we present a complex queuing model with two concatenated queuing systems to characterize the service process and the heterogeneity of the infrastructural architecture in order to analyze the performance of a heterogeneous data center.


### 3.1. Cloud Center Architecture and the Service Model

In the cloud computing environment, a cloud computing provider builds a data center as the infrastructure to provide service for customers [21]. Figure 1 illustrates the architecture of a heterogeneous data center and its service model. The heterogeneous data center consists of large-scale servers of different generations. It includes the master server, which works as a main scheduler to allocate computing resources for a task or to dispatch a node to execute a task, and large-scale computing nodes, which are multicore servers with different speeds that work as execution servers.Figure 1
Cloud center architecture and the service model.All customers submit requests to the data center for services through the Internet, which has been deployed by the users or supplied by the cloud providers. When the tasks (user requests) arrive at the data center, the main scheduler allocates resources for the task or dispatches nodes; the task is then distributed to the execution servers to be executed. The main scheduler has a waiting buffer with a finite capacity, in which it saves waiting tasks that cannot be scheduled immediately. The buffer’s finite capacity means that some requests cannot enter the waiting buffer and will instead leave the system immediately after arriving. Each execution server receives all requests from the main scheduler and maintains its own task queue.After a task has been completed, it leaves the system and the result is sent back to the customer.


### 3.2. Other Queuing Models for Reference

#### 3.2.1. Multiple-Server Queue with the Same Service Rate

A queuing model containing multiple servers with the same service rate is shown in Figure2(a). Using this queuing model, the authors treated a cloud computing data center as an M
/
G
/
m
/
m
+
r queuing system in [11], as an M
/
M
/
r
/
k queuing system in [19], and as an M
/
M
/
m queuing system in [12, 13, 18]. Because each server has the same service rate, this queuing model treats each server in the data center the same and assigns it the same dispatching probability p
=
1
/
n.Figure 2
Other queuing models for reference.
(a)
(b)This queuing model simplifies analysis and makes it easier to deduce an equation for the important performance metrics (e.g., the mean response time, mean number of tasks in the system, and mean waiting time). However, the process presented in this model does not apply to the service process in a cloud data center. It ignores the scheduling activity of the main server and the different speeds of execution servers of different generations.


#### 3.2.2. Multiple-Server Queue with Different Service Rates

A queuing model including multiple servers with different service rates is shown in Figure2(b). We can use this queuing system to model a heterogeneous data center that includes multicore servers with different speeds. Each execution server in the data center has a different service rate μ
i and dispatching probability p
i. We assume that the conditions are N
=
2  and task arrival rate λ. The state-transition-probability diagram for N
=
2 is shown in Figure 3.Figure 3
State-transition-probability diagram forN
=
2.When a task arrives and there are two servers that are idle, the scheduler will select one of them to execute the task. The selection probability of the number 1 server isp
1 and that of the number 2 server is p
2 (p
2
=
1
-
p
1). The state of S
0 indicates that there is no task in the queuing system; S
10 indicates that the task has been distributed to the number 1 server, and the number 2 server is idle; S
01 indicates that the task has been distributed to the number 2 server, and the number 1 server is idle; S
2 indicates that there are two tasks in the system, and each has been distributed to one of the two servers. S
3
,
S
4
,
…
,
S
n indicate that there are 3,4
,
…
,
n tasks in the system, and two of them are in the two execution servers. Let μ
1 and μ
2 denote the service rates of the number 1 and number 2 servers, respectively. The system’s service rate is μ
=
∑
i
=
1
2
μ
i. Figure 3 shows how this model calculates probability when ρ
=
λ
/
μ and α
=
μ
2
/
μ
1 [22]:E
1
π
01
=
ρ
1
+
2
ρ
·
1
+
α
α
ρ
+
p
2
π
0
,
E
2
π
10
=
ρ
1
+
2
ρ
·
1
+
α
ρ
+
p
1
π
0
,
E
3
π
0
=
1
-
ρ
1
+
ρ
1
+
1
+
α
2
ρ
-
1
-
α
2
p
1
/
α
1
+
2
ρ
.FromE
1–E
3, we can see that if N
≥
3, it can use π
n
=
ρ
π
n
-
1 to calculate the probability of each state in Figure 3. The state-transition-probability diagram for N
≥
3 will be more complex.Although it considers the heterogeneity of data centers, this model still does not accurately describe the service process in the cloud center; also, ifN
≥
3, the performance analysis will be more difficult.


### 3.3. Queuing Model for Performance Analysis

A queuing model of heterogeneous data centers with a large number of execution servers with multiple cores of different generations is shown in Figure4. The two concatenated queuing systems—the scheduler queuing system and the execution queuing system—make up the complex queuing model that characterizes the heterogeneity of the data center and of the service processes involved in cloud computing. This model uses a monitor to obtain performance metrics—including the task number in the queuing, L
q, the mean waiting time of a task, W
q, and the utilization of the execution server, ρ. The model then sends this information to the balance controller and optimizes performance by proposing an optimal strategy based on these metrics. This is a project we will consider doing in future.Figure 4
Queuing model of heterogeneous data centers.The complex queuing model is presented as follows.The master server works as the main scheduler and maintains scheduler queuing for all requests from all users. Since the process of allocating resources for tasks in the cloud computing environment should consider all resources in the data center, the master server is treated as anM
/
M
/
1
/
k queuing system with finite capacity k.Each node in a data center works as an execution server, which is a multicore server that hasm
i identical cores with the core execution speed f
i (GIPS, measured by giga instructions per second). Since each multicore execution server can parallel-process multiple tasks, it is treated as an M
/
M
/
c queuing system.The stream of tasks is a Poisson process with arrival rateλ (the task interarrival times are independent and identically distributed exponential random variables with a rate of 1
/
λ). The related notations of the system parameters are listed in the Notations.Since arrivals are independent of the queuing state and the scheduler queuing system is a finite capacity queuing model, the blocking probability of the scheduling systemp
b will be greater than or equal to 0 (p
b
≥
0) and λ
≥
λ
e, to yield the following equations:(1)
λ
e
=
λ
1
-
p
b
,
(2)
λ
i
=
λ
e
p
i
,
1
≤
i
≤
n
,
∑
p
i
=
1
,
(3)
μ
i
=
1
I
-
/
f
i
=
f
i
I
-
,
(4)
ρ
i
=
λ
i
C
i
μ
i
=
λ
i
I
-
C
i
f
i
,
ρ
i
∈
δ
1
,
δ
h
.


## 3.1. Cloud Center Architecture and the Service Model

In the cloud computing environment, a cloud computing provider builds a data center as the infrastructure to provide service for customers [21]. Figure 1 illustrates the architecture of a heterogeneous data center and its service model. The heterogeneous data center consists of large-scale servers of different generations. It includes the master server, which works as a main scheduler to allocate computing resources for a task or to dispatch a node to execute a task, and large-scale computing nodes, which are multicore servers with different speeds that work as execution servers.Figure 1
Cloud center architecture and the service model.All customers submit requests to the data center for services through the Internet, which has been deployed by the users or supplied by the cloud providers. When the tasks (user requests) arrive at the data center, the main scheduler allocates resources for the task or dispatches nodes; the task is then distributed to the execution servers to be executed. The main scheduler has a waiting buffer with a finite capacity, in which it saves waiting tasks that cannot be scheduled immediately. The buffer’s finite capacity means that some requests cannot enter the waiting buffer and will instead leave the system immediately after arriving. Each execution server receives all requests from the main scheduler and maintains its own task queue.After a task has been completed, it leaves the system and the result is sent back to the customer.


## 3.2. Other Queuing Models for Reference

### 3.2.1. Multiple-Server Queue with the Same Service Rate

A queuing model containing multiple servers with the same service rate is shown in Figure2(a). Using this queuing model, the authors treated a cloud computing data center as an M
/
G
/
m
/
m
+
r queuing system in [11], as an M
/
M
/
r
/
k queuing system in [19], and as an M
/
M
/
m queuing system in [12, 13, 18]. Because each server has the same service rate, this queuing model treats each server in the data center the same and assigns it the same dispatching probability p
=
1
/
n.Figure 2
Other queuing models for reference.
(a)
(b)This queuing model simplifies analysis and makes it easier to deduce an equation for the important performance metrics (e.g., the mean response time, mean number of tasks in the system, and mean waiting time). However, the process presented in this model does not apply to the service process in a cloud data center. It ignores the scheduling activity of the main server and the different speeds of execution servers of different generations.


### 3.2.2. Multiple-Server Queue with Different Service Rates

A queuing model including multiple servers with different service rates is shown in Figure2(b). We can use this queuing system to model a heterogeneous data center that includes multicore servers with different speeds. Each execution server in the data center has a different service rate μ
i and dispatching probability p
i. We assume that the conditions are N
=
2  and task arrival rate λ. The state-transition-probability diagram for N
=
2 is shown in Figure 3.Figure 3
State-transition-probability diagram forN
=
2.When a task arrives and there are two servers that are idle, the scheduler will select one of them to execute the task. The selection probability of the number 1 server isp
1 and that of the number 2 server is p
2 (p
2
=
1
-
p
1). The state of S
0 indicates that there is no task in the queuing system; S
10 indicates that the task has been distributed to the number 1 server, and the number 2 server is idle; S
01 indicates that the task has been distributed to the number 2 server, and the number 1 server is idle; S
2 indicates that there are two tasks in the system, and each has been distributed to one of the two servers. S
3
,
S
4
,
…
,
S
n indicate that there are 3,4
,
…
,
n tasks in the system, and two of them are in the two execution servers. Let μ
1 and μ
2 denote the service rates of the number 1 and number 2 servers, respectively. The system’s service rate is μ
=
∑
i
=
1
2
μ
i. Figure 3 shows how this model calculates probability when ρ
=
λ
/
μ and α
=
μ
2
/
μ
1 [22]:E
1
π
01
=
ρ
1
+
2
ρ
·
1
+
α
α
ρ
+
p
2
π
0
,
E
2
π
10
=
ρ
1
+
2
ρ
·
1
+
α
ρ
+
p
1
π
0
,
E
3
π
0
=
1
-
ρ
1
+
ρ
1
+
1
+
α
2
ρ
-
1
-
α
2
p
1
/
α
1
+
2
ρ
.FromE
1–E
3, we can see that if N
≥
3, it can use π
n
=
ρ
π
n
-
1 to calculate the probability of each state in Figure 3. The state-transition-probability diagram for N
≥
3 will be more complex.Although it considers the heterogeneity of data centers, this model still does not accurately describe the service process in the cloud center; also, ifN
≥
3, the performance analysis will be more difficult.


## 3.2.1. Multiple-Server Queue with the Same Service Rate

A queuing model containing multiple servers with the same service rate is shown in Figure2(a). Using this queuing model, the authors treated a cloud computing data center as an M
/
G
/
m
/
m
+
r queuing system in [11], as an M
/
M
/
r
/
k queuing system in [19], and as an M
/
M
/
m queuing system in [12, 13, 18]. Because each server has the same service rate, this queuing model treats each server in the data center the same and assigns it the same dispatching probability p
=
1
/
n.Figure 2
Other queuing models for reference.
(a)
(b)This queuing model simplifies analysis and makes it easier to deduce an equation for the important performance metrics (e.g., the mean response time, mean number of tasks in the system, and mean waiting time). However, the process presented in this model does not apply to the service process in a cloud data center. It ignores the scheduling activity of the main server and the different speeds of execution servers of different generations.


## 3.2.2. Multiple-Server Queue with Different Service Rates

A queuing model including multiple servers with different service rates is shown in Figure2(b). We can use this queuing system to model a heterogeneous data center that includes multicore servers with different speeds. Each execution server in the data center has a different service rate μ
i and dispatching probability p
i. We assume that the conditions are N
=
2  and task arrival rate λ. The state-transition-probability diagram for N
=
2 is shown in Figure 3.Figure 3
State-transition-probability diagram forN
=
2.When a task arrives and there are two servers that are idle, the scheduler will select one of them to execute the task. The selection probability of the number 1 server isp
1 and that of the number 2 server is p
2 (p
2
=
1
-
p
1). The state of S
0 indicates that there is no task in the queuing system; S
10 indicates that the task has been distributed to the number 1 server, and the number 2 server is idle; S
01 indicates that the task has been distributed to the number 2 server, and the number 1 server is idle; S
2 indicates that there are two tasks in the system, and each has been distributed to one of the two servers. S
3
,
S
4
,
…
,
S
n indicate that there are 3,4
,
…
,
n tasks in the system, and two of them are in the two execution servers. Let μ
1 and μ
2 denote the service rates of the number 1 and number 2 servers, respectively. The system’s service rate is μ
=
∑
i
=
1
2
μ
i. Figure 3 shows how this model calculates probability when ρ
=
λ
/
μ and α
=
μ
2
/
μ
1 [22]:E
1
π
01
=
ρ
1
+
2
ρ
·
1
+
α
α
ρ
+
p
2
π
0
,
E
2
π
10
=
ρ
1
+
2
ρ
·
1
+
α
ρ
+
p
1
π
0
,
E
3
π
0
=
1
-
ρ
1
+
ρ
1
+
1
+
α
2
ρ
-
1
-
α
2
p
1
/
α
1
+
2
ρ
.FromE
1–E
3, we can see that if N
≥
3, it can use π
n
=
ρ
π
n
-
1 to calculate the probability of each state in Figure 3. The state-transition-probability diagram for N
≥
3 will be more complex.Although it considers the heterogeneity of data centers, this model still does not accurately describe the service process in the cloud center; also, ifN
≥
3, the performance analysis will be more difficult.


## 3.3. Queuing Model for Performance Analysis

A queuing model of heterogeneous data centers with a large number of execution servers with multiple cores of different generations is shown in Figure4. The two concatenated queuing systems—the scheduler queuing system and the execution queuing system—make up the complex queuing model that characterizes the heterogeneity of the data center and of the service processes involved in cloud computing. This model uses a monitor to obtain performance metrics—including the task number in the queuing, L
q, the mean waiting time of a task, W
q, and the utilization of the execution server, ρ. The model then sends this information to the balance controller and optimizes performance by proposing an optimal strategy based on these metrics. This is a project we will consider doing in future.Figure 4
Queuing model of heterogeneous data centers.The complex queuing model is presented as follows.The master server works as the main scheduler and maintains scheduler queuing for all requests from all users. Since the process of allocating resources for tasks in the cloud computing environment should consider all resources in the data center, the master server is treated as anM
/
M
/
1
/
k queuing system with finite capacity k.Each node in a data center works as an execution server, which is a multicore server that hasm
i identical cores with the core execution speed f
i (GIPS, measured by giga instructions per second). Since each multicore execution server can parallel-process multiple tasks, it is treated as an M
/
M
/
c queuing system.The stream of tasks is a Poisson process with arrival rateλ (the task interarrival times are independent and identically distributed exponential random variables with a rate of 1
/
λ). The related notations of the system parameters are listed in the Notations.Since arrivals are independent of the queuing state and the scheduler queuing system is a finite capacity queuing model, the blocking probability of the scheduling systemp
b will be greater than or equal to 0 (p
b
≥
0) and λ
≥
λ
e, to yield the following equations:(1)
λ
e
=
λ
1
-
p
b
,
(2)
λ
i
=
λ
e
p
i
,
1
≤
i
≤
n
,
∑
p
i
=
1
,
(3)
μ
i
=
1
I
-
/
f
i
=
f
i
I
-
,
(4)
ρ
i
=
λ
i
C
i
μ
i
=
λ
i
I
-
C
i
f
i
,
ρ
i
∈
δ
1
,
δ
h
.


## 4. Performance Metrics of the Queuing Model

In this section, we will use the complex queuing model with the two concatenated queuing systems proposed in Section3.3 to analyze the performance problems of a heterogeneous data center in cloud computing. We consider the performance metrics of the scheduler queuing system, the execution queuing systems, and the entire queuing system.


### 4.1. The Scheduler Queuing System

Since management of computing resources and scheduling tasks across entire data center and cloud environments are done by a unified resources management platform (e.g., Mesos [23] and YARN [24]) and the system should be accompanied by finite capacity, the master server is treated as an M
/
M
/
1
/
k queuing system with finite capacity k. The state-transition-probability diagram for the scheduler queuing model is shown in Figure 5.Figure 5
State-transition-probability diagram for the scheduler queuing model.If the scheduler utilizationρ
s
=
λ
/
μ
s and ρ
s
<
1, then the probability that i tasks are in the queuing system π
i
s is [22](5)
π
0
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
,
π
i
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
ρ
s
i
,
i
=
1,2
,
…
,
k
.Ifi
≥
k, the task request could not enter the queuing system. The blocking probability p
b is(6)
P
b
=
π
k
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
ρ
s
k
.From (1) and (6), the master server throughput rate (the effective arrival rate of the execution queuing systems) λ
e is(7)
λ
e
=
λ
1
-
p
b
=
λ
1
-
ρ
s
k
1
-
ρ
s
k
+
1
.The tasks waiting in the queueC
-
w
s and the tasks scheduled by the main scheduler C
-
s
c
h
s are, respectively,(8)
C
-
w
s
=
∑
i
=
0
k
-
1
k
π
i
+
1
s
=
ρ
s
2
π
0
s
∑
i
=
1
k
-
1
k
ρ
s
k
-
1
,
C
-
sch
s
=
1
-
π
0
s
=
ρ
s
-
ρ
s
k
+
1
1
-
ρ
s
k
+
1
.From (8), the mean task number (including the tasks waiting in the queue and the tasks scheduled by the main scheduler) in the scheduler queuing system C
-
t
o
t
a
l
s is(9)
C
-
total
s
=
C
-
w
s
+
C
-
sch
s
=
ρ
s
1
-
ρ
s
-
k
+
1
ρ
s
k
+
1
1
-
ρ
s
k
+
1
.Using (7) and (9), ρ
s
=
λ
/
μ
s; applying Little’s formulas (the response time t = the tasks number in the system/the tasks arrival rate), the mean task response time of the scheduler queuing system T
-
(
s
) is(10)
T
-
s
=
C
-
total
s
λ
e
=
1
-
k
+
1
ρ
s
k
+
k
ρ
s
k
+
1
μ
s
1
-
ρ
s
1
-
ρ
s
k
=
1
μ
s
-
λ
-
k
λ
k
μ
s
k
+
1
-
λ
k
.


### 4.2. The Execution Queuing Systems

Assume that there aren execution servers in the data center. Each execution server has C
i cores and can be treated as an M
/
M
/
C
i queuing system.The utilization of one core in theith execution server ρ
i
0 is(11)
ρ
i
0
=
λ
i
μ
i
=
λ
i
I
-
f
i
.From the state-transition-probability diagram for anM
/
M
/
C
i queuing system [22], let π
i
,
m
(
e
) indicate the probability that there are m task requests (including waiting in the queue and being executed) in the execution queuing system of the execution server S
i:(12)
π
i
,
m
e
=
π
i
,
0
e
·
ρ
i
0
m
m
!
=
π
i
,
0
e
·
C
i
m
m
!
ρ
i
m
,
0
≤
m
<
C
i
π
i
,
0
e
·
ρ
i
0
m
C
i
!
C
i
m
-
C
i
=
π
i
,
0
e
·
C
i
C
i
C
i
!
ρ
i
m
,
m
≥
C
i
.Under the stability condition, there are constraints as follows:1
∑
m
=
0
∞
π
i
,
m
e
=
∑
m
=
0
C
i
-
1
(
1
/
m
!
)
ρ
i
0
m
+
(
C
i
/
C
i
!
)
ρ
i
0
C
i
-
1
(
ρ
i
/
1
-
ρ
i
)
π
i
,
0
=
1.
2
ρ
i
<
1.We can derive the probability that there are0 task requests in the execution queuing system π
i
,
0
(
e
):(13)
π
i
,
0
e
=
∑
m
=
0
C
i
-
1
ρ
i
0
m
m
!
+
ρ
i
0
C
i
C
i
!
·
1
1
-
ρ
i
-
1
.The probability that a newly arrived task request from the main scheduler to theith execution server will be executed immediately q
i is(14)
q
i
=
1
-
∑
m
=
C
i
∞
π
i
,
m
e
=
1
-
∑
m
=
C
i
∞
C
i
C
i
C
i
!
ρ
i
m
π
i
,
0
e
=
C
i
-
ρ
i
0
-
C
i
π
i
,
C
i
e
C
i
-
ρ
i
0
.The mean probability of a newly arrived task request entering the execution queuing systemsq
-
(
e
) is(15)
q
-
e
=
∑
i
=
1
n
p
i
q
i
.In theith execution server, the tasks waiting in the queue C
-
w
_
i
(
e
) and the tasks executed by the ith execution server C
-
exc
_
i
(
e
) are, respectively,(16)
C
-
w
_
i
e
=
∑
m
=
C
i
∞
m
-
C
i
π
i
,
m
e
,
C
-
exc
_
i
e
=
∑
m
=
0
C
i
m
π
i
,
m
e
+
C
i
∑
m
=
C
i
+
1
∞
π
i
,
m
e
.From (16), the mean task number (including the tasks waiting in the queue and the tasks executed by the execution server) in the ith execution server C
-
t
o
t
a
l
_
i
(
e
) is(17)
C
-
total
_
i
e
=
C
-
w
_
i
e
+
C
-
exc
_
i
e
=
π
i
,
0
e
·
ρ
i
0
C
i
+
1
C
i
-
1
!
C
i
-
ρ
i
0
2
+
ρ
i
0
.Applying Little’s formulas, the mean task response time of theith execution queuing systems T
-
i
(
e
) and the mean waiting time for execution of a task in the ith execution queuing systems W
-
i
(
e
) are, respectively,(18)
T
-
i
e
=
C
-
total
_
i
e
λ
i
=
I
-
f
i
1
+
π
i
,
0
e
·
ρ
i
0
C
i
C
i
-
1
!
C
i
-
ρ
i
0
2
,
W
-
i
e
=
C
-
w
_
i
e
λ
i
=
1
λ
i
∑
m
=
C
i
∞
m
-
C
i
π
i
,
m
e
=
π
i
,
0
e
·
I
-
·
ρ
i
0
C
i
f
i
C
i
-
1
!
C
i
-
ρ
i
0
2
.The mean response time of the execution queuing systemsT
-
(
e
) for a group of n execution servers in the data center is(19)
T
-
e
=
∑
i
=
1
n
P
i
T
-
i
e
.The mean waiting time of the execution queuing systemsW
-
(
e
) in a group of n execution servers in the data center is(20)
W
-
e
=
∑
i
=
1
n
P
i
W
-
i
e
.


### 4.3. The Performance Metrics of the Entire Queuing System

In order to analyze the performance of heterogeneous data centers in cloud computing, we consider the main performance metrics of a complex queuing system, which consists of two concatenated queuing systems, as follows.(i) Response Time. Based on analyzing the two concatenated queuing systems, the equilibrium response time in heterogeneous data centers is the sum of the response time of the two phases. The response time equation can be formulated as(21)
T
-
=
T
-
s
+
T
-
e
=
T
-
s
+
∑
i
=
1
n
P
i
T
-
i
e
.(ii) Waiting Time. The mean waiting time for a task request is(22)
W
-
=
W
-
s
+
W
-
e
=
C
-
W
s
λ
e
+
∑
i
=
1
n
P
i
W
-
i
e
=
ρ
s
μ
s
1
1
-
ρ
s
-
k
ρ
s
k
-
1
1
-
ρ
s
k
+
∑
i
=
1
n
P
i
W
-
i
e
.(iii) Loss (Blocking) Probability. Since the scheduler queuing system is a finite capacity queuing model, blocking will occur:(23)
P
1
=
P
b
.From (6), the loss (blocking) probability of the system is related to the finite capacity of the scheduler queuing system k and the scheduling rate of the main scheduler server μ
s. The parameters of k or μ
s can be adjusted to obtain different loss (blocking) probabilities.(iv) Probability of Immediate Execution. Here, the probability of immediate execution indicates the probability of a task request being executed immediately by an execution server after being scheduled by the scheduler server and sent to the execution server:(24)
q
-
=
q
-
e
.(v) Execution Server Utilization Optimization Model Based on the Arrival Rate
λ
e. Our analytical model can be used to optimize the performance for a data center. This is one of the significant applications of the model.Based on the utilization thresholdδ
l
,
δ
h of the execution servers, we can determine the number of execution servers and the utilization of each execution server in the system, by using the calculation model, which is based on our analytical model, to configure the execution servers in the data center to deal with tasks with the arrival rate λ
e. It can minimize the mean response time of the system.Assume the use of FCFS as the scheduling strategy, and set the heterogeneous data center as a group ofn heterogeneous execution servers S
1
,
S
2
,
…
,
S
n with multicores C
1
,
C
2
,
…
,
C
n and execution speeds of f
1
,
f
2
,
…
,
f
n. The execution server utilization optimization model based on arrival rate λ
e can be formulated as follows:(25)
Min
⁡
λ
e
,
I
¯
,
f
1
,
…
,
f
n
,
C
1
,
…
,
C
n
∑
i
=
1
n
Z
i
P
i
T
¯
i
e
S.T.
1
ρ
i
=
0
,
Z
i
=
0
∈
δ
l
,
δ
h
,
Z
i
=
1
,
1
≤
i
≤
n
2
λ
i
=
ρ
i
C
i
f
i
I
¯
,
1
≤
i
≤
n
3
∑
i
=
1
n
λ
i
=
λ
e
4
P
i
=
ρ
i
C
i
f
i
I
¯
λ
e
,
1
≤
i
≤
n
5
∑
i
=
1
n
P
i
=
1
6
T
-
i
e
=
I
-
f
i
1
+
∑
m
=
0
C
i
-
1
C
i
ρ
i
m
C
i
-
1
!
+
1
C
i
-
1
!
·
C
i
ρ
i
C
i
1
-
ρ
i
-
1
·
C
i
ρ
i
C
i
-
1
ρ
i
C
i
!
1
-
ρ
i
,
1
≤
i
≤
n
7
Z
i
∈
0,1
,
1
≤
i
≤
n
.The utilization of the execution servers will determine mean response time, occupied time, and resource capacity; therefore, the execution server utilization optimization model based on arrival rateλ
e can be used to analyze the problem of optimal resource cost or the problem of energy efficiency. We will accomplish this research in future work.In this paper, we just consider the special case—the condition ofλ
e
∈
(
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
,
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i. In a small case, we can use a numerical method to gain the values of ρ
i for optimal performance. Under the condition of λ
e
∈
(
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
,
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i, the parameter of Z
i in (25) is Z
i
=
1; (25) can be rewritten as follows:(26)
Min
⁡
λ
e
,
I
¯
,
f
1
,
…
,
f
n
,
C
1
,
…
,
C
n
∑
i
=
1
n
ρ
i
C
i
λ
e
1
+
∑
m
=
0
C
i
-
1
C
i
ρ
i
m
C
i
-
1
!
+
1
C
i
-
1
!
·
C
i
ρ
i
C
i
1
-
ρ
i
-
1
·
C
i
ρ
i
C
i
-
1
ρ
i
C
i
!
1
-
ρ
i
S.T.
1
δ
l
≤
ρ
i
≤
δ
h
,
1
≤
i
≤
n
2
∑
i
=
1
n
ρ
i
C
i
f
i
I
¯
=
λ
e
3
∑
i
=
1
n
ρ
i
C
i
f
i
I
¯
λ
e
=
1
.


## 4.1. The Scheduler Queuing System

Since management of computing resources and scheduling tasks across entire data center and cloud environments are done by a unified resources management platform (e.g., Mesos [23] and YARN [24]) and the system should be accompanied by finite capacity, the master server is treated as an M
/
M
/
1
/
k queuing system with finite capacity k. The state-transition-probability diagram for the scheduler queuing model is shown in Figure 5.Figure 5
State-transition-probability diagram for the scheduler queuing model.If the scheduler utilizationρ
s
=
λ
/
μ
s and ρ
s
<
1, then the probability that i tasks are in the queuing system π
i
s is [22](5)
π
0
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
,
π
i
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
ρ
s
i
,
i
=
1,2
,
…
,
k
.Ifi
≥
k, the task request could not enter the queuing system. The blocking probability p
b is(6)
P
b
=
π
k
s
=
1
-
ρ
s
1
-
ρ
s
k
+
1
ρ
s
k
.From (1) and (6), the master server throughput rate (the effective arrival rate of the execution queuing systems) λ
e is(7)
λ
e
=
λ
1
-
p
b
=
λ
1
-
ρ
s
k
1
-
ρ
s
k
+
1
.The tasks waiting in the queueC
-
w
s and the tasks scheduled by the main scheduler C
-
s
c
h
s are, respectively,(8)
C
-
w
s
=
∑
i
=
0
k
-
1
k
π
i
+
1
s
=
ρ
s
2
π
0
s
∑
i
=
1
k
-
1
k
ρ
s
k
-
1
,
C
-
sch
s
=
1
-
π
0
s
=
ρ
s
-
ρ
s
k
+
1
1
-
ρ
s
k
+
1
.From (8), the mean task number (including the tasks waiting in the queue and the tasks scheduled by the main scheduler) in the scheduler queuing system C
-
t
o
t
a
l
s is(9)
C
-
total
s
=
C
-
w
s
+
C
-
sch
s
=
ρ
s
1
-
ρ
s
-
k
+
1
ρ
s
k
+
1
1
-
ρ
s
k
+
1
.Using (7) and (9), ρ
s
=
λ
/
μ
s; applying Little’s formulas (the response time t = the tasks number in the system/the tasks arrival rate), the mean task response time of the scheduler queuing system T
-
(
s
) is(10)
T
-
s
=
C
-
total
s
λ
e
=
1
-
k
+
1
ρ
s
k
+
k
ρ
s
k
+
1
μ
s
1
-
ρ
s
1
-
ρ
s
k
=
1
μ
s
-
λ
-
k
λ
k
μ
s
k
+
1
-
λ
k
.


## 4.2. The Execution Queuing Systems

Assume that there aren execution servers in the data center. Each execution server has C
i cores and can be treated as an M
/
M
/
C
i queuing system.The utilization of one core in theith execution server ρ
i
0 is(11)
ρ
i
0
=
λ
i
μ
i
=
λ
i
I
-
f
i
.From the state-transition-probability diagram for anM
/
M
/
C
i queuing system [22], let π
i
,
m
(
e
) indicate the probability that there are m task requests (including waiting in the queue and being executed) in the execution queuing system of the execution server S
i:(12)
π
i
,
m
e
=
π
i
,
0
e
·
ρ
i
0
m
m
!
=
π
i
,
0
e
·
C
i
m
m
!
ρ
i
m
,
0
≤
m
<
C
i
π
i
,
0
e
·
ρ
i
0
m
C
i
!
C
i
m
-
C
i
=
π
i
,
0
e
·
C
i
C
i
C
i
!
ρ
i
m
,
m
≥
C
i
.Under the stability condition, there are constraints as follows:1
∑
m
=
0
∞
π
i
,
m
e
=
∑
m
=
0
C
i
-
1
(
1
/
m
!
)
ρ
i
0
m
+
(
C
i
/
C
i
!
)
ρ
i
0
C
i
-
1
(
ρ
i
/
1
-
ρ
i
)
π
i
,
0
=
1.
2
ρ
i
<
1.We can derive the probability that there are0 task requests in the execution queuing system π
i
,
0
(
e
):(13)
π
i
,
0
e
=
∑
m
=
0
C
i
-
1
ρ
i
0
m
m
!
+
ρ
i
0
C
i
C
i
!
·
1
1
-
ρ
i
-
1
.The probability that a newly arrived task request from the main scheduler to theith execution server will be executed immediately q
i is(14)
q
i
=
1
-
∑
m
=
C
i
∞
π
i
,
m
e
=
1
-
∑
m
=
C
i
∞
C
i
C
i
C
i
!
ρ
i
m
π
i
,
0
e
=
C
i
-
ρ
i
0
-
C
i
π
i
,
C
i
e
C
i
-
ρ
i
0
.The mean probability of a newly arrived task request entering the execution queuing systemsq
-
(
e
) is(15)
q
-
e
=
∑
i
=
1
n
p
i
q
i
.In theith execution server, the tasks waiting in the queue C
-
w
_
i
(
e
) and the tasks executed by the ith execution server C
-
exc
_
i
(
e
) are, respectively,(16)
C
-
w
_
i
e
=
∑
m
=
C
i
∞
m
-
C
i
π
i
,
m
e
,
C
-
exc
_
i
e
=
∑
m
=
0
C
i
m
π
i
,
m
e
+
C
i
∑
m
=
C
i
+
1
∞
π
i
,
m
e
.From (16), the mean task number (including the tasks waiting in the queue and the tasks executed by the execution server) in the ith execution server C
-
t
o
t
a
l
_
i
(
e
) is(17)
C
-
total
_
i
e
=
C
-
w
_
i
e
+
C
-
exc
_
i
e
=
π
i
,
0
e
·
ρ
i
0
C
i
+
1
C
i
-
1
!
C
i
-
ρ
i
0
2
+
ρ
i
0
.Applying Little’s formulas, the mean task response time of theith execution queuing systems T
-
i
(
e
) and the mean waiting time for execution of a task in the ith execution queuing systems W
-
i
(
e
) are, respectively,(18)
T
-
i
e
=
C
-
total
_
i
e
λ
i
=
I
-
f
i
1
+
π
i
,
0
e
·
ρ
i
0
C
i
C
i
-
1
!
C
i
-
ρ
i
0
2
,
W
-
i
e
=
C
-
w
_
i
e
λ
i
=
1
λ
i
∑
m
=
C
i
∞
m
-
C
i
π
i
,
m
e
=
π
i
,
0
e
·
I
-
·
ρ
i
0
C
i
f
i
C
i
-
1
!
C
i
-
ρ
i
0
2
.The mean response time of the execution queuing systemsT
-
(
e
) for a group of n execution servers in the data center is(19)
T
-
e
=
∑
i
=
1
n
P
i
T
-
i
e
.The mean waiting time of the execution queuing systemsW
-
(
e
) in a group of n execution servers in the data center is(20)
W
-
e
=
∑
i
=
1
n
P
i
W
-
i
e
.


## 4.3. The Performance Metrics of the Entire Queuing System

In order to analyze the performance of heterogeneous data centers in cloud computing, we consider the main performance metrics of a complex queuing system, which consists of two concatenated queuing systems, as follows.(i) Response Time. Based on analyzing the two concatenated queuing systems, the equilibrium response time in heterogeneous data centers is the sum of the response time of the two phases. The response time equation can be formulated as(21)
T
-
=
T
-
s
+
T
-
e
=
T
-
s
+
∑
i
=
1
n
P
i
T
-
i
e
.(ii) Waiting Time. The mean waiting time for a task request is(22)
W
-
=
W
-
s
+
W
-
e
=
C
-
W
s
λ
e
+
∑
i
=
1
n
P
i
W
-
i
e
=
ρ
s
μ
s
1
1
-
ρ
s
-
k
ρ
s
k
-
1
1
-
ρ
s
k
+
∑
i
=
1
n
P
i
W
-
i
e
.(iii) Loss (Blocking) Probability. Since the scheduler queuing system is a finite capacity queuing model, blocking will occur:(23)
P
1
=
P
b
.From (6), the loss (blocking) probability of the system is related to the finite capacity of the scheduler queuing system k and the scheduling rate of the main scheduler server μ
s. The parameters of k or μ
s can be adjusted to obtain different loss (blocking) probabilities.(iv) Probability of Immediate Execution. Here, the probability of immediate execution indicates the probability of a task request being executed immediately by an execution server after being scheduled by the scheduler server and sent to the execution server:(24)
q
-
=
q
-
e
.(v) Execution Server Utilization Optimization Model Based on the Arrival Rate
λ
e. Our analytical model can be used to optimize the performance for a data center. This is one of the significant applications of the model.Based on the utilization thresholdδ
l
,
δ
h of the execution servers, we can determine the number of execution servers and the utilization of each execution server in the system, by using the calculation model, which is based on our analytical model, to configure the execution servers in the data center to deal with tasks with the arrival rate λ
e. It can minimize the mean response time of the system.Assume the use of FCFS as the scheduling strategy, and set the heterogeneous data center as a group ofn heterogeneous execution servers S
1
,
S
2
,
…
,
S
n with multicores C
1
,
C
2
,
…
,
C
n and execution speeds of f
1
,
f
2
,
…
,
f
n. The execution server utilization optimization model based on arrival rate λ
e can be formulated as follows:(25)
Min
⁡
λ
e
,
I
¯
,
f
1
,
…
,
f
n
,
C
1
,
…
,
C
n
∑
i
=
1
n
Z
i
P
i
T
¯
i
e
S.T.
1
ρ
i
=
0
,
Z
i
=
0
∈
δ
l
,
δ
h
,
Z
i
=
1
,
1
≤
i
≤
n
2
λ
i
=
ρ
i
C
i
f
i
I
¯
,
1
≤
i
≤
n
3
∑
i
=
1
n
λ
i
=
λ
e
4
P
i
=
ρ
i
C
i
f
i
I
¯
λ
e
,
1
≤
i
≤
n
5
∑
i
=
1
n
P
i
=
1
6
T
-
i
e
=
I
-
f
i
1
+
∑
m
=
0
C
i
-
1
C
i
ρ
i
m
C
i
-
1
!
+
1
C
i
-
1
!
·
C
i
ρ
i
C
i
1
-
ρ
i
-
1
·
C
i
ρ
i
C
i
-
1
ρ
i
C
i
!
1
-
ρ
i
,
1
≤
i
≤
n
7
Z
i
∈
0,1
,
1
≤
i
≤
n
.The utilization of the execution servers will determine mean response time, occupied time, and resource capacity; therefore, the execution server utilization optimization model based on arrival rateλ
e can be used to analyze the problem of optimal resource cost or the problem of energy efficiency. We will accomplish this research in future work.In this paper, we just consider the special case—the condition ofλ
e
∈
(
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
,
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i. In a small case, we can use a numerical method to gain the values of ρ
i for optimal performance. Under the condition of λ
e
∈
(
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
,
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i, the parameter of Z
i in (25) is Z
i
=
1; (25) can be rewritten as follows:(26)
Min
⁡
λ
e
,
I
¯
,
f
1
,
…
,
f
n
,
C
1
,
…
,
C
n
∑
i
=
1
n
ρ
i
C
i
λ
e
1
+
∑
m
=
0
C
i
-
1
C
i
ρ
i
m
C
i
-
1
!
+
1
C
i
-
1
!
·
C
i
ρ
i
C
i
1
-
ρ
i
-
1
·
C
i
ρ
i
C
i
-
1
ρ
i
C
i
!
1
-
ρ
i
S.T.
1
δ
l
≤
ρ
i
≤
δ
h
,
1
≤
i
≤
n
2
∑
i
=
1
n
ρ
i
C
i
f
i
I
¯
=
λ
e
3
∑
i
=
1
n
ρ
i
C
i
f
i
I
¯
λ
e
=
1
.


## 5. Numerical Validation

In order to validate the performance analysis equations presented above, we built a heterogeneous cloud computing data center farm in the CloudSim platform and a tasks generator with using the discrete event tool MATLAB. In the numerical examples and the simulation experiments we present, the parameters are illustrative and can be changed to suit the cloud computing environment.


### 5.1. Performance Simulation

In this section, we describe the simulation experiments that we conducted to validate the performance analysis equations of the performance metrics. In all cases, the mean number of instructions of a task to be executed by the execution server wasI
¯
=
1 (giga instructions). In the CloudSim, we considered a heterogeneous data center with a group of n
=
10 multicore execution servers Host#1, Host#2,…, Host#10 and a main scheduler server Ms. The traffic intensity of the main scheduler server was ρ
s
=
0.80. The finite capacity of the scheduler k
=
0.1
λ. The different settings of the Host#i (1
≤
i
≤
10) are shown in Table 1. The allocation policy of the VM-scheduler is space-shared, which makes one task in one VM. The total computing ability (computing power) of these two groups of execution servers with different settings was the same.Table 1
Hosts and VMs resources setting.
HostID (PM)
Setting: ST1
Setting: ST2
CoreNum.(VM Num., C
i)
Computingpower (GIPS, f
i)
VMs’ setting
CoreNum.(VM Num., C
i)
Computingpower (GIPS, f
i)
VMs’ setting
Host#1
2
2
1 Core/2 G
2
2
1 Core/2 G
Host#2
4
2
1 Core/2 G
2
2
1 Core/2 G
Host#3
6
2
1 Core/2 G
2
2
1 Core/2 G
Host#4
8
2
1 Core/2 G
4
2
1 Core/2 G
Host#5
10
2
1 Core/2 G
4
2
1 Core/2 G
Host#6
12
2
1 Core/2 G
4
2
1 Core/2 G
Host#7
14
2
1 Core/2 G
8
4
1 Core/4 G
Host#8
16
2
1 Core/2 G
8
4
1 Core/4 G
Host#9
18
2
1 Core/2 G
12
5
1 Core/5 G
Host#10
20
2
1 Core/2 G
12
5
1 Core/5 GIn Table1, the heterogeneous data centers of ST1 and ST2 are configured with 10 PMs and 110 VMs and 10 PMs and 58 VMs, respectively. The total computing powers of ST1 and ST2 are all 220 (GIPS). The hardware environment is 2×Dell PowerEdge T720 (2×CPUs: Xeon 6-core E5-2630 2.3 G, 12 cores in total).The interarrival times of task requests were independent and followed an exponential distribution with arrival rateλ (the number of task requests per second). The cloudlet information is created by tasks generator, which generates task arrival interval time, task scheduling time, and task length.In the experiments, we assigned each multicore execution server—which had a dispatched probability according to its core speeds or set traffic intensity—to each multicore execution server to control a task. The rule for setting the dispatched probability and the traffic intensity, which can be changed by administrators, was as follows.(i) Setting the Dispatched Probability.Consider(27)
P
i
=
C
i
f
i
∑
i
=
1
10
C
i
f
i
,
1
≤
i
≤
10
.Under this pattern, the traffic intensity of each execution server was the same and may have exceeded the scope of the utilization thresholdδ
l
,
δ
h. Using these two groups of execution servers (ST1 and ST2), the dispatched probability of each execution server was set using (27).Simulation Experiments  1.We have the following:(1)
Tasks number: cloudletsNo = 100000, tasks length (GIPS):-
log
⁡
(
rand
(
1
,
cloudletNo
)
)
/
(
f
i
/
I
¯
), and  λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5.
(2)
The main scheduler computing power:μ
s
=
λ
/
ρ
s, hosts (PMs) and VMs setting: ST1 and St2 (shown in Table 1), and the probability of host (PM) dispatched:P
i (27).
(3)
Process: simulation experiment times: 30. In every time, we recorded the loss tasks number in the main scheduler, the immediate execution tasks number (if the arrival time equates to start time), the response time, and the waiting time of each host. After 30 times’ experiments, we calculated the average values of the loss (blocking) probabilityP
l
(
P
b
), the probability of immediate execution q
¯, the mean response time T
¯, and the mean waiting time W
¯. Then, we calculated the 95% confidence interval (95% C.I.) for each metric to validate these metrics in the analysis model.
(4)
The analytical results are calculated by using the analytical model with (21)–(24). The settings of each parameter are λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5, ρ
s
=
0.80, k
=
0.1
λ, I
¯
=
1, P
i (27), and C
i and f
i shown in Table 1.The simulations and the analytical results (λ: 157.5
,
172.5,187.5) are shown in Tables 2–5.Table 2
The simulations and analytical results ofP
l
(
P
b
).
λ
Host set
Simulation 95% C.I. forP
l
(
P
b
)
Analytical results
Mean
Lower
Upper
157.5
ST1
0.0058
0.005683
0.005917
0.005759
ST2
0.00573
0.005491
0.005969
0.005759
172.5
ST1
0.00452
0.004404
0.004636
0.004586
ST2
0.00462
0.004504
0.004736
0.004586
187.5
ST1
0.00295
0.002853
0.003047
0.002916
ST2
0.00296
0.002842
0.003078
0.002916Table 3
The simulations and analytical results ofq
¯.
λ
Host set
Simulation 95% C.I. forq
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.80593
0.803737
0.808123
0.806097
ST2
0.72404
0.723025
0.725055
0.724773
172.5
ST1
0.6867
0.683968
0.689432
0.688503
ST2
0.60165
0.599881
0.603419
0.60225
187.5
ST1
0.526
0.524375
0.527625
0.525132
ST2
0.44677
0.445745
0.447795
0.447063Table 4
The simulations and analytical results ofW
¯.
λ
Host set
Simulation 95% C.I. forW
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.05987
0.059035
0.060705
0.060115
ST2
0.08227
0.081676
0.082864
0.082617
172.5
ST1
0.09591
0.094468
0.097352
0.096286
ST2
0.1249
0.123706
0.126094
0.12491
187.5
ST1
0.17872
0.174778
0.182662
0.178457
ST2
0.21112
0.207124
0.215116
0.213743Table 5
The simulations and analytical results ofT
¯.
λ
Host set
Simulation 95% C.I. forT
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.56496
0.565194
0.56374
0.56618
ST2
0.35094
0.350361
0.351519
0.351333
172.5
ST1
0.60029
0.598638
0.601942
0.600923
ST2
0.39317
0.391785
0.394555
0.393184
187.5
ST1
0.6829
0.678949
0.686851
0.682724
ST2
0.47903
0.474915
0.483145
0.481646The comparisons between the analytical and the simulations results (λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5) are shown in Figures 6–9.Figure 6
Simulation 95% C.I. for Pl versus analytical result.Figure 7
Simulation 95% C.I. forq
¯ versus analytical result.Figure 8
Simulation 95% C.I. forW
¯ versus analytical result.Figure 9
Simulation 95% C.I. forT
¯ versus analytical result.By comparing the simulation results obtained from CloudSim and the analytical results calculated by using the model, we can observe that the analytical results ofP
l
(
P
b
), q
¯, W
¯, and T
¯ are all in the range of 95% C.I., which are shown in Tables 2–5 and Figures 6–9. It demonstrates that the analytical results are in agreement with the CloudSim simulation results. It also confirms that the metrics of P
l
(
P
b), q
¯, W
¯, and T
¯ in the analytical model can be trusted under the confidence level of 95%.Simulation Experiments  2. Use the same dataset (task number: 100000, ST1 and ST2) as the simulation experiments 1 to complete 30 times’ experiments under the arrival rate λ
=
187.5. From this experiment, we recorded the response and waiting times of each host in ST1 and ST2 (T
¯
i
(
e
) and W
¯
i
(
e
)
,
1
≤
i
≤
10). Then, we can get the 95% confidence interval (95% C.I.) of T
¯
i
(
e
) and W
¯
i
(
e
)to validate these metrics in the model.The analytical results are gotten by using (18) and (19).In Tables6 and 7, we demonstrate the simulation results and the 95% C.I. of T
¯
i
(
e
) and W
¯
i
(
e
) (1
≤
i
≤
10) of every host in ST1 and ST2 with a task request arrival rate λ
=
187.5.Table 6
The simulations and analytical results of each host in ST1.
HostID
Simulation 95% C.I. forT
¯
i
(
e
)
Analytical results ofT
¯
i
(
e
)
Simulation 95% C.I. forW
¯
i
(
e
)
Analytical results ofW
¯
i
(
e
)
Mean
Lower
Upper
Mean
Lower
Upper
Host#1
1.808109
1.730226
1.885993
1.79946
1.308227
1.231192
1.385263
1.29946
Host#2
1.090659
1.064057
1.117261
1.073279
0.590882
0.565038
0.616725
0.573279
Host#3
0.840732
0.82654
0.854924
0.845942
0.341209
0.327489
0.354929
0.345942
Host#4
0.737109
0.730784
0.743434
0.738017
0.237314
0.231432
0.243195
0.238017
Host#5
0.674368
0.66885
0.679887
0.676187
0.174809
0.169671
0.179947
0.176187
Host#6
0.634818
0.630513
0.639123
0.636685
0.135545
0.13165
0.139441
0.136685
Host#7
0.607036
0.603298
0.610775
0.609575
0.107477
0.104036
0.110919
0.109575
Host#8
0.589295
0.58681
0.59178
0.59
0.089395
0.087273
0.091518
0.09
Host#9
0.576559
0.572962
0.580156
0.57532
0.076586
0.073578
0.079594
0.07532
Host#10
0.565364
0.563253
0.567474
0.563981
0.065318
0.063482
0.067154
0.063981Table 7
The simulations and analytical results of each host in ST2.
HostID
Simulation 95% C.I. forT
¯
i
(
e
)
Analytical results ofT
¯
i
(
e
)
Simulation 95% C.I. forW
¯
i
(
e
)
Analytical results ofW
¯
i
(
e
)
Mean
Lower
Upper
Mean
Lower
Upper
Host#1
1.846295
1.774547
1.918044
1.79946
1.347014
1.275931
1.418096
1.29946
Host#2
1.79655
1.742504
1.850596
1.79946
1.296655
1.243793
1.349516
1.29946
Host#3
1.806673
1.742178
1.871167
1.79946
1.307555
1.244333
1.370776
1.29946
Host#4
1.082836
1.062659
1.103014
1.073279
0.582073
0.562148
0.601997
0.573279
Host#5
1.065777
1.042977
1.088577
1.073279
0.5667
0.544379
0.589021
0.573279
Host#6
1.055886
1.03763
1.074143
1.073279
0.556777
0.538945
0.574609
0.573279
Host#7
0.369245
0.367006
0.371485
0.369009
0.119423
0.117318
0.121527
0.119009
Host#8
0.368186
0.365717
0.370656
0.369009
0.118227
0.115911
0.120543
0.119009
Host#9
0.255173
0.254154
0.256191
0.254674
0.055091
0.054138
0.056044
0.054674
Host#10
0.254464
0.253357
0.255571
0.254674
0.054359
0.053383
0.055335
0.054674The comparisons between the analytical and the simulations results of the hosts are shown in Figures10-11.Figure 10
Simulation 95% C.I. forT
¯
i
(
e
) versus analytical result.Figure 11
Simulation 95% C.I. forW
¯
i
(
e
) versus analytical result.As shown in Tables6 and 7 and Figures 10 and 11, we can observe that the analytical results of T
¯
i
(
e
) and W
¯
i
(
e
) for all the hosts in ST1 and ST2 are all in the range of 95% C.I. It demonstrates that the analytical results are in agreement with the CloudSim simulation results and the hosts’ analytical model can be trusted under the confidence level of 95%.After comparing the results, we also conclude that(1)
the relative errors of the simulation and the analytics are less than 3.5%,
(2)
under the same arrival rateλ
e, the performance metrics will be different with different settings despite having the same computing abilities.(ii) Setting the Traffic Intensity. The traffic intensity/utilization ρ
i (1
≤
i
≤
10) was set in the scope of δ
l
,
δ
h (setting δ
l
=
0.65 and δ
h
=
0.85). Under this pattern, the scope of the task request arrival rate of the ith execution server will be λ
i
∈
δ
l
C
i
f
i
/
I
¯
,
δ
h
C
i
f
i
/
I
¯ (1
≤
i
≤
10) according to (4). A group of m (m
≤
n) multicore execution servers can handle the effective arrival rate λ
e
′:(28)
λ
e
′
=
∑
i
=
1
m
λ
i
⟹
δ
l
I
¯
∑
i
=
1
m
C
i
f
i
≤
λ
e
′
≤
δ
h
I
¯
∑
i
=
1
m
C
i
f
i
.Simulation Experiments  3. Let us consider using the same hosts set to deal with the same arrival rate λ
=
172 ((28), 172
∈
[
220
×
0.65
,
220
×
0.85
]) at designated rates of traffic intensity/utilization ρ
i (1
≤
i
≤
10) setting. The different ρ
i (1
≤
i
≤
10) sets are shown in Table 8 as TIs1 and TIs2, respectively.Table 8
The results of each host with different traffic intensity in ST1.
HostID
Traffic intensity set 1 (TIs1)
Traffic intensity set 2 (TIs2)
ρ
i
Simulation
Analytical results
Relative error
ρ
i
Simulation
Analytical results
Relative error
Host#1
0.751
1.143158
1.146792
0.32%
0.651
0.87184
0.867756
0.47%
Host#2
0.755
0.76765
0.764214
0.45%
0.665
0.642525
0.640293
0.35%
Host#3
0.756
0.648089
0.647759
0.05%
0.686
0.580317
0.583304
0.51%
Host#4
0.759
0.595157
0.597003
0.31%
0.735
0.578816
0.577729
0.19%
Host#5
0.761
0.571153
0.572848
0.30%
0.749
0.560845
0.560706
0.02%
Host#6
0.764
0.554577
0.555183
0.11%
0.782
0.56216
0.562954
0.14%
Host#7
0.766
0.541873
0.543499
0.30%
0.786
0.551055
0.550656
0.07%
Host#8
0.771
0.534157
0.535349
0.22%
0.807
0.551097
0.551994
0.16%
Host#9
0.815
0.54695
0.547189
0.04%
0.811
0.542748
0.544765
0.37%
Host#10
0.798
0.52541
0.525306
0.02%
0.813
0.538181
0.538257
0.01%We completed 30 times with these settings to get the mean values. The mean response time results of the execution queuing systemsT
¯
i
(
e
) (1
≤
i
≤
10) are shown in Table 8.From Table8, we can see that the maximum relative error between the simulation and the analysis is 0.51%. By setting the traffic intensity of TIs1 and TIs2 for each host in the group of ST1, we find different mean response times for the execution queuing systems. The T
¯
(
e
) in TIs1 has fallen from 0.571203 (second) to 0.5607632 (second) in TIs2—an increase of 1.8%. It shows that the same hardware resource will have different performance with different traffic intensity setting for each host.We can compare the results of our analytical calculation with the results of simulation data shown in Tables2–8 and Figures 6–11, and all of the analytical results’ relative errors are less than 3.5%. The results also showed that all of the analytical results are in the range of 95% C.I. for the metrics. This shows that our analysis agrees with our simulation results, which confirms the validity of our analytical model.


### 5.2. Numerical Examples

We demonstrate some numerical examples to analyze the relationship among performance metrics with using the analytical model.(i) Numerical Example  1. We use the two groups of execution servers (hosts in ST1 and ST2) shown in Table 1 with the same total computing ability to handle an arrival rate λ
=
155. The execution servers in ST1 and ST2 are sorted by computing ability (computing power). In the experiments, we adjust the traffic intensity of each execution server, so that we can analyze the mean response time T
¯ and the mean waiting time W
¯ of the system. We select 16 sets of traffic intensity, and each set has 10 ρ
i (1
≤
i
≤
10) to fit 10 execution servers.The results of the experiments are shown in Figures12 and 13. In the 16 experiments, we set similar traffic intensity for the ith execution server in ST1 and ST2. Although ST1 and ST2 have the same total computing ability, the mean response time T
¯ and the mean waiting time W
¯ of the system differ significantly from each other.Figure 12
The mean response times of ST1 and ST2.Figure 13
The mean waiting times of ST1 and ST2.In Figures12 and 13, we can see that adjusting the traffic intensity (utilization) of each execution server can enhance the response and waiting times. In ST1, the mean response time T
¯ is decreased from 0.707159 (second) to 0.554670 (second), and response time has been enhanced by 21.56%. The mean waiting time W
¯ is decreased from 0.201998 (second) to 0.049509 (second), and response time has been enhanced by 75.49%. In ST2, the mean response time T
¯ is decreased from 0.523104 (second) to 0.333339 (second), and response time has been enhanced by 36.27%. The mean waiting time W
¯ is decreased from 0.240814 (second) to 0.067228 (second), and response time has been enhanced by 72.08%. We can see that the traffic intensity (utilization) of each execution server has a significant impact on the performance of the heterogeneous data center.The configuration of a server cluster in a heterogeneous data center is an important factor that greatly impacts the system’s performance. Again, Figures12 and 13 show the results of our 16 experiments, and we can see that the mean response time T
¯ of ST2 is better than ST1 under a similar traffic intensity (utilization) setting. Mean response time improves by 35.18% on average, and the maximum is 39.9%. However, the mean waiting time W
¯ of ST2 is worse than ST1. Mean waiting time is decreased by 24.95% on average and the maximum is 29%. It is important for a cloud provider to configure the server cluster into a reasonable structure to provide services for the customer. This conclusion will be confirmed in numerical example 2.(ii) Numerical Example  2. Let us consider the third group of execution servers, named ST3, that includes 10 servers, each of which is configured as C
i
=
4 and f
i
=
5.5 (1
≤
i
≤
10). The other conditions are the same as the conditions in simulation experiments 1. Assume that the arrival rate λ is set from 150 to 187.5, which means that the traffic intensity of each execution server would be in the threshold δ
l
,
δ
h.The performance results (T
¯, W
¯, and q
¯) of the three groups (ST1, ST2, and ST3) of execution servers are shown in Figures 14–16. Different configurations of server clusters will have different performance results. The best performance of the mean response time is group ST3, and the worst is ST1, as shown in Figure 14. The best performance of mean waiting time is group ST1, and the worst is ST3, as shown in Figure 15. In Figure 16, ST1 has the maximum immediate execution probability, while the ST3 has the minimum value. We can use this queuing model to estimate the performance results of a server cluster with a certain configuration under different λ. In order to enhance the performance of mean response time, configuring the server cluster in a reasonable structure is an efficient method.Figure 14
The mean response times of ST1, ST2, and ST3.Figure 15
The mean waiting times of ST1, ST2, and ST3.Figure 16
Immediate execution probability of ST1, ST2, and ST3.In practice, users will present some constraints when they ask cloud computing providers for services. They will present constraints as follows:(1)
The task request arrival rateλ
i
n
∈
λ
l
,
λ
h.
(2)
The mean response timeτ
r
≤
T
l.
(3)
The mean wasting timeτ
w
≤
W
¯
l or the immediate execution probability ζ
≥
q
¯
l.Cloud computing providers could configure the server cluster to serve customers under certain constraints by using our queuing model. In our future work, we will use the complex queuing model to further research dynamic cluster technology in a heterogeneous data center to learn how it handles different tasks with different arrival rates under various constraints.(iii) Numerical Example  3. One application of our analytical model is to use this model for optimal system performance by finding a reasonable traffic intensity setting for each execution server. Numerical example 3 shows the optimization under the condition of (
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
≤
λ
e
≤
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i to get the values of ρ
i.Assume using three execution serversS
1 (C
1
=
4, f
1
=
4), S
2 (C
2
=
4, f
2
=
3), and S
3 (C
3
=
6, f
3
=
4) to handle the arrival rate λ
e
=
38.3 (16
+
12
+
24
×
0.65
,
16
+
12
+
24
×
0.85). Equations (25) and (26)—the execution server utilization optimization model based on the arrival rate λ
e—can use a numerical method to get the minimum mean response time of this system under the condition of λ
e
=
38.3. As Figure 17 shows, we can see that the minimum mean response time T
¯
m
i
n
=
0.36317 when ρ
1
=
0.72712, ρ
2
=
0.67627, and ρ
3
=
0.77295. Controlling the traffic intensity (or utilization) of each execution server allows for the control of the dispatched probability of each execution server and the optimization of system performance.Figure 17
The mean response time with different traffic intensity settings.The traffic intensity (or utilization) of each execution server will impact the response time, computing resource allocation, and energy usage of the system; therefore, we will further optimize the execution server utilization optimization model in future work.


## 5.1. Performance Simulation

In this section, we describe the simulation experiments that we conducted to validate the performance analysis equations of the performance metrics. In all cases, the mean number of instructions of a task to be executed by the execution server wasI
¯
=
1 (giga instructions). In the CloudSim, we considered a heterogeneous data center with a group of n
=
10 multicore execution servers Host#1, Host#2,…, Host#10 and a main scheduler server Ms. The traffic intensity of the main scheduler server was ρ
s
=
0.80. The finite capacity of the scheduler k
=
0.1
λ. The different settings of the Host#i (1
≤
i
≤
10) are shown in Table 1. The allocation policy of the VM-scheduler is space-shared, which makes one task in one VM. The total computing ability (computing power) of these two groups of execution servers with different settings was the same.Table 1
Hosts and VMs resources setting.
HostID (PM)
Setting: ST1
Setting: ST2
CoreNum.(VM Num., C
i)
Computingpower (GIPS, f
i)
VMs’ setting
CoreNum.(VM Num., C
i)
Computingpower (GIPS, f
i)
VMs’ setting
Host#1
2
2
1 Core/2 G
2
2
1 Core/2 G
Host#2
4
2
1 Core/2 G
2
2
1 Core/2 G
Host#3
6
2
1 Core/2 G
2
2
1 Core/2 G
Host#4
8
2
1 Core/2 G
4
2
1 Core/2 G
Host#5
10
2
1 Core/2 G
4
2
1 Core/2 G
Host#6
12
2
1 Core/2 G
4
2
1 Core/2 G
Host#7
14
2
1 Core/2 G
8
4
1 Core/4 G
Host#8
16
2
1 Core/2 G
8
4
1 Core/4 G
Host#9
18
2
1 Core/2 G
12
5
1 Core/5 G
Host#10
20
2
1 Core/2 G
12
5
1 Core/5 GIn Table1, the heterogeneous data centers of ST1 and ST2 are configured with 10 PMs and 110 VMs and 10 PMs and 58 VMs, respectively. The total computing powers of ST1 and ST2 are all 220 (GIPS). The hardware environment is 2×Dell PowerEdge T720 (2×CPUs: Xeon 6-core E5-2630 2.3 G, 12 cores in total).The interarrival times of task requests were independent and followed an exponential distribution with arrival rateλ (the number of task requests per second). The cloudlet information is created by tasks generator, which generates task arrival interval time, task scheduling time, and task length.In the experiments, we assigned each multicore execution server—which had a dispatched probability according to its core speeds or set traffic intensity—to each multicore execution server to control a task. The rule for setting the dispatched probability and the traffic intensity, which can be changed by administrators, was as follows.(i) Setting the Dispatched Probability.Consider(27)
P
i
=
C
i
f
i
∑
i
=
1
10
C
i
f
i
,
1
≤
i
≤
10
.Under this pattern, the traffic intensity of each execution server was the same and may have exceeded the scope of the utilization thresholdδ
l
,
δ
h. Using these two groups of execution servers (ST1 and ST2), the dispatched probability of each execution server was set using (27).Simulation Experiments  1.We have the following:(1)
Tasks number: cloudletsNo = 100000, tasks length (GIPS):-
log
⁡
(
rand
(
1
,
cloudletNo
)
)
/
(
f
i
/
I
¯
), and  λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5.
(2)
The main scheduler computing power:μ
s
=
λ
/
ρ
s, hosts (PMs) and VMs setting: ST1 and St2 (shown in Table 1), and the probability of host (PM) dispatched:P
i (27).
(3)
Process: simulation experiment times: 30. In every time, we recorded the loss tasks number in the main scheduler, the immediate execution tasks number (if the arrival time equates to start time), the response time, and the waiting time of each host. After 30 times’ experiments, we calculated the average values of the loss (blocking) probabilityP
l
(
P
b
), the probability of immediate execution q
¯, the mean response time T
¯, and the mean waiting time W
¯. Then, we calculated the 95% confidence interval (95% C.I.) for each metric to validate these metrics in the analysis model.
(4)
The analytical results are calculated by using the analytical model with (21)–(24). The settings of each parameter are λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5, ρ
s
=
0.80, k
=
0.1
λ, I
¯
=
1, P
i (27), and C
i and f
i shown in Table 1.The simulations and the analytical results (λ: 157.5
,
172.5,187.5) are shown in Tables 2–5.Table 2
The simulations and analytical results ofP
l
(
P
b
).
λ
Host set
Simulation 95% C.I. forP
l
(
P
b
)
Analytical results
Mean
Lower
Upper
157.5
ST1
0.0058
0.005683
0.005917
0.005759
ST2
0.00573
0.005491
0.005969
0.005759
172.5
ST1
0.00452
0.004404
0.004636
0.004586
ST2
0.00462
0.004504
0.004736
0.004586
187.5
ST1
0.00295
0.002853
0.003047
0.002916
ST2
0.00296
0.002842
0.003078
0.002916Table 3
The simulations and analytical results ofq
¯.
λ
Host set
Simulation 95% C.I. forq
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.80593
0.803737
0.808123
0.806097
ST2
0.72404
0.723025
0.725055
0.724773
172.5
ST1
0.6867
0.683968
0.689432
0.688503
ST2
0.60165
0.599881
0.603419
0.60225
187.5
ST1
0.526
0.524375
0.527625
0.525132
ST2
0.44677
0.445745
0.447795
0.447063Table 4
The simulations and analytical results ofW
¯.
λ
Host set
Simulation 95% C.I. forW
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.05987
0.059035
0.060705
0.060115
ST2
0.08227
0.081676
0.082864
0.082617
172.5
ST1
0.09591
0.094468
0.097352
0.096286
ST2
0.1249
0.123706
0.126094
0.12491
187.5
ST1
0.17872
0.174778
0.182662
0.178457
ST2
0.21112
0.207124
0.215116
0.213743Table 5
The simulations and analytical results ofT
¯.
λ
Host set
Simulation 95% C.I. forT
¯
Analytical results
Mean
Lower
Upper
157.5
ST1
0.56496
0.565194
0.56374
0.56618
ST2
0.35094
0.350361
0.351519
0.351333
172.5
ST1
0.60029
0.598638
0.601942
0.600923
ST2
0.39317
0.391785
0.394555
0.393184
187.5
ST1
0.6829
0.678949
0.686851
0.682724
ST2
0.47903
0.474915
0.483145
0.481646The comparisons between the analytical and the simulations results (λ: 157.5, 160.5, 163.5, 166.5, 169.5, 172.5, 175.5, 178.5, 181.5, 184.5, 187.5) are shown in Figures 6–9.Figure 6
Simulation 95% C.I. for Pl versus analytical result.Figure 7
Simulation 95% C.I. forq
¯ versus analytical result.Figure 8
Simulation 95% C.I. forW
¯ versus analytical result.Figure 9
Simulation 95% C.I. forT
¯ versus analytical result.By comparing the simulation results obtained from CloudSim and the analytical results calculated by using the model, we can observe that the analytical results ofP
l
(
P
b
), q
¯, W
¯, and T
¯ are all in the range of 95% C.I., which are shown in Tables 2–5 and Figures 6–9. It demonstrates that the analytical results are in agreement with the CloudSim simulation results. It also confirms that the metrics of P
l
(
P
b), q
¯, W
¯, and T
¯ in the analytical model can be trusted under the confidence level of 95%.Simulation Experiments  2. Use the same dataset (task number: 100000, ST1 and ST2) as the simulation experiments 1 to complete 30 times’ experiments under the arrival rate λ
=
187.5. From this experiment, we recorded the response and waiting times of each host in ST1 and ST2 (T
¯
i
(
e
) and W
¯
i
(
e
)
,
1
≤
i
≤
10). Then, we can get the 95% confidence interval (95% C.I.) of T
¯
i
(
e
) and W
¯
i
(
e
)to validate these metrics in the model.The analytical results are gotten by using (18) and (19).In Tables6 and 7, we demonstrate the simulation results and the 95% C.I. of T
¯
i
(
e
) and W
¯
i
(
e
) (1
≤
i
≤
10) of every host in ST1 and ST2 with a task request arrival rate λ
=
187.5.Table 6
The simulations and analytical results of each host in ST1.
HostID
Simulation 95% C.I. forT
¯
i
(
e
)
Analytical results ofT
¯
i
(
e
)
Simulation 95% C.I. forW
¯
i
(
e
)
Analytical results ofW
¯
i
(
e
)
Mean
Lower
Upper
Mean
Lower
Upper
Host#1
1.808109
1.730226
1.885993
1.79946
1.308227
1.231192
1.385263
1.29946
Host#2
1.090659
1.064057
1.117261
1.073279
0.590882
0.565038
0.616725
0.573279
Host#3
0.840732
0.82654
0.854924
0.845942
0.341209
0.327489
0.354929
0.345942
Host#4
0.737109
0.730784
0.743434
0.738017
0.237314
0.231432
0.243195
0.238017
Host#5
0.674368
0.66885
0.679887
0.676187
0.174809
0.169671
0.179947
0.176187
Host#6
0.634818
0.630513
0.639123
0.636685
0.135545
0.13165
0.139441
0.136685
Host#7
0.607036
0.603298
0.610775
0.609575
0.107477
0.104036
0.110919
0.109575
Host#8
0.589295
0.58681
0.59178
0.59
0.089395
0.087273
0.091518
0.09
Host#9
0.576559
0.572962
0.580156
0.57532
0.076586
0.073578
0.079594
0.07532
Host#10
0.565364
0.563253
0.567474
0.563981
0.065318
0.063482
0.067154
0.063981Table 7
The simulations and analytical results of each host in ST2.
HostID
Simulation 95% C.I. forT
¯
i
(
e
)
Analytical results ofT
¯
i
(
e
)
Simulation 95% C.I. forW
¯
i
(
e
)
Analytical results ofW
¯
i
(
e
)
Mean
Lower
Upper
Mean
Lower
Upper
Host#1
1.846295
1.774547
1.918044
1.79946
1.347014
1.275931
1.418096
1.29946
Host#2
1.79655
1.742504
1.850596
1.79946
1.296655
1.243793
1.349516
1.29946
Host#3
1.806673
1.742178
1.871167
1.79946
1.307555
1.244333
1.370776
1.29946
Host#4
1.082836
1.062659
1.103014
1.073279
0.582073
0.562148
0.601997
0.573279
Host#5
1.065777
1.042977
1.088577
1.073279
0.5667
0.544379
0.589021
0.573279
Host#6
1.055886
1.03763
1.074143
1.073279
0.556777
0.538945
0.574609
0.573279
Host#7
0.369245
0.367006
0.371485
0.369009
0.119423
0.117318
0.121527
0.119009
Host#8
0.368186
0.365717
0.370656
0.369009
0.118227
0.115911
0.120543
0.119009
Host#9
0.255173
0.254154
0.256191
0.254674
0.055091
0.054138
0.056044
0.054674
Host#10
0.254464
0.253357
0.255571
0.254674
0.054359
0.053383
0.055335
0.054674The comparisons between the analytical and the simulations results of the hosts are shown in Figures10-11.Figure 10
Simulation 95% C.I. forT
¯
i
(
e
) versus analytical result.Figure 11
Simulation 95% C.I. forW
¯
i
(
e
) versus analytical result.As shown in Tables6 and 7 and Figures 10 and 11, we can observe that the analytical results of T
¯
i
(
e
) and W
¯
i
(
e
) for all the hosts in ST1 and ST2 are all in the range of 95% C.I. It demonstrates that the analytical results are in agreement with the CloudSim simulation results and the hosts’ analytical model can be trusted under the confidence level of 95%.After comparing the results, we also conclude that(1)
the relative errors of the simulation and the analytics are less than 3.5%,
(2)
under the same arrival rateλ
e, the performance metrics will be different with different settings despite having the same computing abilities.(ii) Setting the Traffic Intensity. The traffic intensity/utilization ρ
i (1
≤
i
≤
10) was set in the scope of δ
l
,
δ
h (setting δ
l
=
0.65 and δ
h
=
0.85). Under this pattern, the scope of the task request arrival rate of the ith execution server will be λ
i
∈
δ
l
C
i
f
i
/
I
¯
,
δ
h
C
i
f
i
/
I
¯ (1
≤
i
≤
10) according to (4). A group of m (m
≤
n) multicore execution servers can handle the effective arrival rate λ
e
′:(28)
λ
e
′
=
∑
i
=
1
m
λ
i
⟹
δ
l
I
¯
∑
i
=
1
m
C
i
f
i
≤
λ
e
′
≤
δ
h
I
¯
∑
i
=
1
m
C
i
f
i
.Simulation Experiments  3. Let us consider using the same hosts set to deal with the same arrival rate λ
=
172 ((28), 172
∈
[
220
×
0.65
,
220
×
0.85
]) at designated rates of traffic intensity/utilization ρ
i (1
≤
i
≤
10) setting. The different ρ
i (1
≤
i
≤
10) sets are shown in Table 8 as TIs1 and TIs2, respectively.Table 8
The results of each host with different traffic intensity in ST1.
HostID
Traffic intensity set 1 (TIs1)
Traffic intensity set 2 (TIs2)
ρ
i
Simulation
Analytical results
Relative error
ρ
i
Simulation
Analytical results
Relative error
Host#1
0.751
1.143158
1.146792
0.32%
0.651
0.87184
0.867756
0.47%
Host#2
0.755
0.76765
0.764214
0.45%
0.665
0.642525
0.640293
0.35%
Host#3
0.756
0.648089
0.647759
0.05%
0.686
0.580317
0.583304
0.51%
Host#4
0.759
0.595157
0.597003
0.31%
0.735
0.578816
0.577729
0.19%
Host#5
0.761
0.571153
0.572848
0.30%
0.749
0.560845
0.560706
0.02%
Host#6
0.764
0.554577
0.555183
0.11%
0.782
0.56216
0.562954
0.14%
Host#7
0.766
0.541873
0.543499
0.30%
0.786
0.551055
0.550656
0.07%
Host#8
0.771
0.534157
0.535349
0.22%
0.807
0.551097
0.551994
0.16%
Host#9
0.815
0.54695
0.547189
0.04%
0.811
0.542748
0.544765
0.37%
Host#10
0.798
0.52541
0.525306
0.02%
0.813
0.538181
0.538257
0.01%We completed 30 times with these settings to get the mean values. The mean response time results of the execution queuing systemsT
¯
i
(
e
) (1
≤
i
≤
10) are shown in Table 8.From Table8, we can see that the maximum relative error between the simulation and the analysis is 0.51%. By setting the traffic intensity of TIs1 and TIs2 for each host in the group of ST1, we find different mean response times for the execution queuing systems. The T
¯
(
e
) in TIs1 has fallen from 0.571203 (second) to 0.5607632 (second) in TIs2—an increase of 1.8%. It shows that the same hardware resource will have different performance with different traffic intensity setting for each host.We can compare the results of our analytical calculation with the results of simulation data shown in Tables2–8 and Figures 6–11, and all of the analytical results’ relative errors are less than 3.5%. The results also showed that all of the analytical results are in the range of 95% C.I. for the metrics. This shows that our analysis agrees with our simulation results, which confirms the validity of our analytical model.


## 5.2. Numerical Examples

We demonstrate some numerical examples to analyze the relationship among performance metrics with using the analytical model.(i) Numerical Example  1. We use the two groups of execution servers (hosts in ST1 and ST2) shown in Table 1 with the same total computing ability to handle an arrival rate λ
=
155. The execution servers in ST1 and ST2 are sorted by computing ability (computing power). In the experiments, we adjust the traffic intensity of each execution server, so that we can analyze the mean response time T
¯ and the mean waiting time W
¯ of the system. We select 16 sets of traffic intensity, and each set has 10 ρ
i (1
≤
i
≤
10) to fit 10 execution servers.The results of the experiments are shown in Figures12 and 13. In the 16 experiments, we set similar traffic intensity for the ith execution server in ST1 and ST2. Although ST1 and ST2 have the same total computing ability, the mean response time T
¯ and the mean waiting time W
¯ of the system differ significantly from each other.Figure 12
The mean response times of ST1 and ST2.Figure 13
The mean waiting times of ST1 and ST2.In Figures12 and 13, we can see that adjusting the traffic intensity (utilization) of each execution server can enhance the response and waiting times. In ST1, the mean response time T
¯ is decreased from 0.707159 (second) to 0.554670 (second), and response time has been enhanced by 21.56%. The mean waiting time W
¯ is decreased from 0.201998 (second) to 0.049509 (second), and response time has been enhanced by 75.49%. In ST2, the mean response time T
¯ is decreased from 0.523104 (second) to 0.333339 (second), and response time has been enhanced by 36.27%. The mean waiting time W
¯ is decreased from 0.240814 (second) to 0.067228 (second), and response time has been enhanced by 72.08%. We can see that the traffic intensity (utilization) of each execution server has a significant impact on the performance of the heterogeneous data center.The configuration of a server cluster in a heterogeneous data center is an important factor that greatly impacts the system’s performance. Again, Figures12 and 13 show the results of our 16 experiments, and we can see that the mean response time T
¯ of ST2 is better than ST1 under a similar traffic intensity (utilization) setting. Mean response time improves by 35.18% on average, and the maximum is 39.9%. However, the mean waiting time W
¯ of ST2 is worse than ST1. Mean waiting time is decreased by 24.95% on average and the maximum is 29%. It is important for a cloud provider to configure the server cluster into a reasonable structure to provide services for the customer. This conclusion will be confirmed in numerical example 2.(ii) Numerical Example  2. Let us consider the third group of execution servers, named ST3, that includes 10 servers, each of which is configured as C
i
=
4 and f
i
=
5.5 (1
≤
i
≤
10). The other conditions are the same as the conditions in simulation experiments 1. Assume that the arrival rate λ is set from 150 to 187.5, which means that the traffic intensity of each execution server would be in the threshold δ
l
,
δ
h.The performance results (T
¯, W
¯, and q
¯) of the three groups (ST1, ST2, and ST3) of execution servers are shown in Figures 14–16. Different configurations of server clusters will have different performance results. The best performance of the mean response time is group ST3, and the worst is ST1, as shown in Figure 14. The best performance of mean waiting time is group ST1, and the worst is ST3, as shown in Figure 15. In Figure 16, ST1 has the maximum immediate execution probability, while the ST3 has the minimum value. We can use this queuing model to estimate the performance results of a server cluster with a certain configuration under different λ. In order to enhance the performance of mean response time, configuring the server cluster in a reasonable structure is an efficient method.Figure 14
The mean response times of ST1, ST2, and ST3.Figure 15
The mean waiting times of ST1, ST2, and ST3.Figure 16
Immediate execution probability of ST1, ST2, and ST3.In practice, users will present some constraints when they ask cloud computing providers for services. They will present constraints as follows:(1)
The task request arrival rateλ
i
n
∈
λ
l
,
λ
h.
(2)
The mean response timeτ
r
≤
T
l.
(3)
The mean wasting timeτ
w
≤
W
¯
l or the immediate execution probability ζ
≥
q
¯
l.Cloud computing providers could configure the server cluster to serve customers under certain constraints by using our queuing model. In our future work, we will use the complex queuing model to further research dynamic cluster technology in a heterogeneous data center to learn how it handles different tasks with different arrival rates under various constraints.(iii) Numerical Example  3. One application of our analytical model is to use this model for optimal system performance by finding a reasonable traffic intensity setting for each execution server. Numerical example 3 shows the optimization under the condition of (
δ
l
/
I
¯
)
∑
i
=
1
m
C
i
f
i
≤
λ
e
≤
(
δ
h
/
I
¯
)
∑
i
=
1
m
C
i
f
i to get the values of ρ
i.Assume using three execution serversS
1 (C
1
=
4, f
1
=
4), S
2 (C
2
=
4, f
2
=
3), and S
3 (C
3
=
6, f
3
=
4) to handle the arrival rate λ
e
=
38.3 (16
+
12
+
24
×
0.65
,
16
+
12
+
24
×
0.85). Equations (25) and (26)—the execution server utilization optimization model based on the arrival rate λ
e—can use a numerical method to get the minimum mean response time of this system under the condition of λ
e
=
38.3. As Figure 17 shows, we can see that the minimum mean response time T
¯
m
i
n
=
0.36317 when ρ
1
=
0.72712, ρ
2
=
0.67627, and ρ
3
=
0.77295. Controlling the traffic intensity (or utilization) of each execution server allows for the control of the dispatched probability of each execution server and the optimization of system performance.Figure 17
The mean response time with different traffic intensity settings.The traffic intensity (or utilization) of each execution server will impact the response time, computing resource allocation, and energy usage of the system; therefore, we will further optimize the execution server utilization optimization model in future work.


## 6. Conclusions and Future Work

Performance analysis of heterogeneous data centers is a crucial aspect of cloud computing for both cloud providers and cloud service customers. Based on an analysis of the characteristics of heterogeneous data centers and their service processes, we propose a complex queuing model composed of two concatenated queuing systems—the main schedule queue and the execution queue—to evaluate the performance of heterogeneous data centers. We theoretically analyzed mean response time, mean waiting time, and other important performance indicators. We also conducted simulation experiments to validate the complex queuing model. The simulation results and the calculated values demonstrate that the complex queuing model provides results with a high degree of accuracy for each performance metric and allows for a sophisticated analysis of heterogeneous data centers.We have further conducted some numerical examples to analyze factors such as the traffic intensity (or utilization) of each execution server and the configuration of server clusters in a heterogeneous data center; these are factors that significantly impact the performance of the system. Based on this complex queuing model, we plan to extend our analytical model to the study of dynamic cluster technology in a heterogeneous data center. In doing so, we aim to help service providers optimize resource allocation using the execution server utilization optimization model.



---

*Source: 980945-2015-07-09.xml*