IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 2017

2007

PAPER

Special Section on Network Virtualization, Network Softwarization and Fusion Platform of Computing and Networking

Prediction-Based Cloud Bursting Approach and Its Impact on Total
Cost for Business-Critical Web Systems
Yukio OGAWA†a) , Go HASEGAWA†† , Members, and Masayuki MURATA†† , Fellow

SUMMARY Cloud bursting temporarily expands the capacity of acloud-based service hosted in a private data center by renting public data
center capacity when the demand for capacity spikes. To determine theoptimal resources of a business-critical web system deployed over private
and public data centers, this paper presents a cloud bursting approach basedon long- and short-term predictions of requests to the system. In a private
data center, a dedicated pool of virtual machines (VMs) is assigned to the
web system on the basis of one-week predictions. Moreover, in both private
and public data centers, VMs are activated on the basis of one-hour predictions. We formulate a problem that includes the total cost and responsetime constraints and conduct numerical simulations. The results indicatethat our approach is tolerant of prediction errors and only slightly dependent
on the processing power of a single VM. Even if the website receives bursty
requests and one-hour predictions include a mean absolute percentage error(MAPE) of 0.2, the total cost decreases to half the existing cost of provisioning in the private date center alone. At the same time, 95% of responsetime is kept below 0.15 s.
key words: cloud bursting, hybrid cloud, request prediction, total cost

1.

Introduction

Computing resources, e.g., physical and virtual servers, areassigned dedicatedly to an application system for achieving
both high availability and desired performance without being
affected by other systems. This type of resource allocation
is practical for business-critical application systems in enterprise private data centers. Such systems are, however,generally built to handle peak workloads, which results inthem being underutilized for most of the time [1]. An effective approach for maximizing the resource utilization to
improve the cost efficiency of such existing systems is cloud
bursting [2]. In this approach, an application system uses
fixed resources in a private data center for the majority of
its computing. The system further bursts into a public data
center and temporarily combines on-demand resources when
private resources are insufficient. We take this approach toprovision virtual machines (VMs) for business-critical web
systems. Our goal is to minimize the total cost of a computing platform while satisfying response time constraints.
We thus focus on determining the right amount of VMs inboth private and public data centers (i.e., in a hybrid cloud
Manuscript received December 12, 2016.
Manuscript revised March 18, 2017.
Manuscript publicized May 16, 2017.
† The author is with the Center for Multimedia Aided Education,
Muroran Institute of Technology, Muroran-shi, 050-8585 Japan.
†† The authors are with the Graduate School of Information Science and Technology, Osaka University, Suita-shi, 565-0871 Japan.
a) E-mail: y-ogawa@mmm.muroran-it.ac.jp
DOI: 10.1587/transcom.2016NNP0006

environment) in advance in order to adaptively adjust VMs
to meet the current workloads.
Automatic extra resource allocation during increased
demand and its termination when demand decreases is a major research topic in cloud environments [3]–[5]. Resource
allocations are classified into two methods: reactive andproactive [6]. The reactive allocation method reconfigures
an application system to meet the system’s requirements for
quality of service (QoS) after detecting changes in the workload, utilization, etc. of the system (e.g., [7]). If the requirements can be seriously violated from when the changes aredetected to when the system reconfiguration is completed,
the reconfiguration should be proactively triggered on thebasis of estimating the future changes.
Studies on automating cloud bursting in a proactive
manner are roughly divided into two categories on the basisof whether workload demand is known ahead of time or not.
In the first category, the future workload is known in advance.
This category includes high-performance computing for scientific applications, in which there is a trade-off between
the completion time of the tasks and the amount of required
resources. The number of tasks is known ahead of timeand resources are adaptively scheduled for the tasks to meetdeadlines [8]–[10]. In the second category, future workload
is unknown. Accordingly, future demand must be estimatedto adjust the trade-off between application constraints, suchas response time and throughput, and computing resource
economics, such as cost and configuration overhead, e.g., in
the cases of enterprise applications [11], a video streaming
service [12], and production systems [13]. Our target falls
into the second category, in which an application platform isdynamically reconfigured to optimize the trade-off on the basis of predicting the demand for the application. Prediction
errors thus can greatly affect the optimization. Although
the previous studies [11]–[13] supposed different deployment scenarios and cost frameworks form ours, they, as well
as autoscaling studies targeting a single cloud environment(e.g., [14], [15]), will give us a clue to consider the framework of our cost optimization problem. However, the impact
of prediction accuracy on the total cost and its trade-off withapplication constraints has not been sufficiently discussed in
the above studies. We thus focus on describing the impact
for our deployment scenario.
A business-critical application system is often assigned
a dedicated cluster of physical servers because availability ofthe system is determined at the cluster to which redundancy
techniques for the VMs are applied [16]. We thus reallocate

Copyright © 2017 The Institute of Electronics, Information and Communication Engineers

IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 20172008

not only VMs but also physical servers in a private data center. A software-defined networking (SDN) framework makethis reallocation feasible [17], although physical servers havea longer reallocation interval than VMs in practical deployment. We therefore present a two-step approach to adjust
computing resources in a hybrid cloud environment: assigning physical servers in a private data center on the basis ofa long-term (e.g., a week) prediction, and activating VMs
in both private and public data centers on the basis of ashort-term (e.g., an hour) prediction.
In this paper, we present a cloud bursting approachbased on long- and short-term predictions for physical andvirtual servers, respectively. Previously, we described a costmodel of an application platform in a hybrid cloud environment [18]. We hence focus on evaluating the impact ofprediction accuracy on our cloud bursting approach by using
trace data of actual websites. The long-term prediction is,of course, not as accurate as the short-term prediction. Our
main contributions are therefore to demonstrate that:
1. the prediction errors of the long-term physical server
provisioning do not affect the optimized total cost much.
2. the prediction errors of the short-term VM allocation
are handled by using the confidence interval for theprediction, and the allocation can enable the application
system to satisfy response time constraints.
The rest of this paper is organized as follows. In Sect. 2,
we introduce an operational procedure. In Sect. 3, we definea cost model. In Sect. 4, we describe a method for evaluations. Then, in Sect. 5, we evaluate our approach. Finally, inSect. 6, we give conclusions.
2.

Overview of a Cloud Busting Approach

We first give an overview of an application system in a hybrid cloud environment (called a hybrid cloud system) and
explain our operational procedure. In accordance with thecost model of Weinman [19], we decrease fixed capacities
to improve the utilization of application systems in a private data center and add on-demand resources in a publicdata center during the peak time. As shown in Fig. 1, in theprivate data center, a dedicated set of physical servers (i.e.,a pool of VMs) is reallocated to an application system on
the basis of long-term workload predictions every weekend;
this interval is planned by considering long-term workload
variations, as well as management aspects such as the recovery time and cost when this reallocation fails. Moreover,
the amount of VMs required in the system is planned on thebasis of short-term predictions every hour; this interval is set
corresponding to the billing interval of the public data center.
If the amount required is less than the amount available in theprivate data center, the minimum VMs alone are activated,
and unnecessary VMs are powered off or put to sleep. In
contrast, when the required number of VMs is more than the
maximum number of VMs in the private data center at thattime, the shortage of VMs is compensated for by additionallyallocating on-demand VMs in the public data center.

Fig. 1

Overview of cloud bursting approach.

In above deployment scenario, the total number of physical servers may be fixed during a time horizon (e.g., durationof renting the physical servers), and hence the private data
center may supply a too large/small VM pool to each application system. Nevertheless, the amount of physical servers
required by an application system is around the average capacity used by the system and fluctuations in the amount aremuch smaller than the peak-average difference. Moreover,there can be more than a hundred of application systems inan enterprise data center [20]. We therefore assume thatthe fluctuations in an application system are offset by thosein other systems and thereby the total number of physicalservers remains nearly constant in the time horizon.
We also note that, while we dynamically reconfigure
computing resources, we fix the configurations of a widearea network (WAN) between private and public data centersand local-area networks (LANs) in them. The WAN, which ismanaged by another service provider, can not be dynamicallyreconfigured from the private data center and thus should
provide sufficient bandwidth to handle peak workloads ofbusiness-critical application systems. The LANs usually
supply sufficient bandwidth as well, because the LANs aremuch cheaper than computing resources [21].
3.

Model of a Hybrid Cloud System

In this section, we describe a cost model and response time
constraints of a hybrid cloud system. We sometimes call
VMs in private and public data centers private VMs andpublic VMs.
3.1

Cost Model

The VMs in both private and public data centers are controlled at fixed intervals called time slots, each of which isindexed by t (t = 1, · · · , T ). A hybrid cloud system has parameters that change with time slots t as depicted in Fig. 2and summarized in Table 1. Here, the size (Nt ) and processing rate (Λt ) of a dedicated VM pool are altered at the endof each interval of w time slots.
Our objective is to minimize the total cost of an appli-

OGAWA et al.: PREDICTION-BASED CLOUD BURSTING APPROACH AND ITS IMPACT ON TOTAL COST FOR BUSINESS-CRITICAL WEB SYSTEMS

2009&
F (Nt , nt ) = cps

Fig. 2Main parameters used for explaining cloud bursting approach.

Table 1xt
nt , n0t
λt , λ0t
Nt , Λ t

U (nt0, λ t0 ) = cvm nt0 + ctr dλ t0,

Average rate of requests to application system
Numbers of VMs allocated and turned on for the application
system deployed over private and public data centers
Average rates of requests sent to VMs in a private data centerand a public data center, respectively (λt + λ0t = xt )Capacity of the VM pool dedicated for the application systemin the private data center and its average processing rate(nt ≤ Nt , λt ≤ Λt ))Cost of renting a physical server per time slot
Number of VMs per physical server
Energy charge rate
Energy consumed per physical server
Energy-proportional coefficient [22]Cost of an on-demand VM per time slot
Cost of forwarding requests per unit size
Average amount of transferred data per request
campus website and 4100 bytes for a consumer website)(c) Constants for defining cost for operation and management
cst
nst
α

T
X


aF (Nt , nt ) + a 0U (nt0, λ t0 ) + O(Nt , nt0 ) ,where, on the right side, the first term is the cost for using ondemand VMs, and the second term is the cost for transferring
requests to/from the VMs and synchronizing data storages.
Note that we do not count the cost for traversing a WAN, e.g.,(virtual) dedicated network or the internet, between private
and public data centers because we assume that the hybrid
cloud system shares the WAN with other application systems
and that the WAN is charged at a flat rate.
Finally, the cost for operation and management, i.e., thecost of the IT staff members who manage and operate theapplication hosting platform across private and public data
centers, is defined by using the constants in Table 2(c) as!
 α1O(Nt , nt0 ) = cst
Nt + nt0
,
(4)
nst

3.2

Personnel cost per time slot per staff member
Number of VMs managed by a staff member
Constant for specifying economics of scale (α ≤1) [23]

cation hosting platform, C, defined as the sum of the cost
related to the fixed private VMs, F, that are relevant to the
on-demand public VMs, U, and that for the operation andmanagement, O, over a time horizon.
Objective: minimizeC=(3)

where the IT staff members are prepared to support the sumof the maximum number of private VMs and the average
number of public VMs. We also assume economics ofscale [23].

(b) Constants related to VMs assigned in public data center
cvm
ctr
d

where,
j N k on the right side, the first term is the cost of rentingt
the cost fornvm physical servers. The second term is
jn k
powering the physical servers [22], where nvmt physicalservers are needed for allocating and turning on nt VMs.
Here, the second term assumes that each physical server hasenergy proportionality [24].
Second, we define the cost related to the public VMs byreferring the constants in Table 2(b) as

Parameters changing with time slot t.

Table 2 Constants for describing cost related to servers.
(a) Constants related to VMs assigned in private data center
cps
nvm
cec
pps
e

'
&
'
!
Nt
nt
nt
+ cec pps (1−e)
+e
,
nvm
nvm
nvm
(2)

(1)

t=1

where a is a constant for determining the total cost including
all of the servers, networks, storages, etc. from the cost
related to the servers alone in the private data center, and a 0
is that in the public data center.
First, the cost related to the private VMs is defined byusing the constants in Table 2(a) as

Constraints on Response Time Performance

There is a trade-off between application latency and resource
amount given to the application system. We thus pose constraints on response time: in both private and public data
centers, qth percentiles of response time distribution foreach time slot (r q and r q0) are not more than a thresholdr c . Here, q is the target probability. When we define thecumulative distribution function of response time (R definedin Sect. 3.4), the above relationship for the private data centeris replaced with an alternative relationship: the probability
determined by the number of private VMs (nt ), the request
rate processed by these VMs (λ t ), and the threshold time
(r c ) is not less than the target probability (q), as shown inConstraint (5). The same relationship is also given to the
public data center by Constraint (6). Here, we add the notation ˆ to the parameter of a predicted value.
Subject to:q 
r q ≤ r c R(nt , λ̂ t , r c ) ≥
(∀t)
(5)
100 

q
r q0 ≤ r c R(nt0, λ̂ t0, r c ) ≥
(∀t)
(6)
100

IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 20172010

In these constraints, the numbers of private VMs (nt ) andpublic VMs (nt0 ) are determined by using the predicted valuesof request rates ( λ̂ t and λ̂ t0 ). The actual qth percentiles (r qand r q0) can exceed r c due to prediction errors.
3.3

Request Rate Prediction

We adopt the autoregressive integrated moving average(ARIMA) model [25] to predict the request rates. Whendefining the backward shift operator B by Bx t = x t−1 , theoriginal time series, x t , is transformed into a stationary timeseries yt = (1 − B) d (1 − B s ) D x t by applying the dth-ordernon-periodic differencing and the Dth-order periodic differencing. This yt is then expressed as a function of its pastvalues and/or past errors, as follows.
p
X

q
X
*
φi B yt + .1 +
θj B j +/  t
yt =
i=1
j=1
,
i

(7)

where φi, θj are the parameters, and  t is the error term that
follows  t ∼ N (0, σ 2 ). The confidence interval of the onetime-slot-ahead prediction is the standard deviation of theerrors (σ), which means yt+1 ∼ N P( ŷt+1, σ 2 ). Moreover,∞when yt+h is expressed as yt+h = τ=0ψτ  t+h−τ (whereψτ is the parameter calculated from the observed
andPh−1 values
ψ0 = 1 ), yt+h follows yt+h ∼ N ( ŷt+h, σ 2 τ=0ψτ2 ).
3.4

To introduce the response time constraints explained in
Sect. 3.2, we define the cumulative distribution function of
response time at time slot t by applying the M/M/m queuing
model [26]. Since a web system is supposed to be implemented asynchronously so that it can respond quickly to arequest without waiting for the request to be completed, we
adopt the waiting time distribution, not the sojourn time distribution. Let r, r 0 , and µ be the response time from theapplication system at t, a constant network latency, and average processing rate of requests per VM, respectively. The
cumulative distribution function R is defined as
R(nt , λ t , r) = 1−π(nt , λ t )e−(nt µ−λt )(r−r0 ) (r ≥ r 0 ), (8)where π(nt , λ t ) is the probability of requests to be queued att. This probability is defined as

ρt =

λt
.
µ

1: for each time slot t (t = 1, · · · , T ) do2:
if t mod w = 0 then3:
Predict { x̂t +h | h = 1, 2, · · · , w } according to Eq.(7).
4:
Nt +1 ← VMPoolSize( x̂t +1, x̂t +2, · · · , x̂t +w ).
5:
The number of jdedicatedphysical servers in the next week
N t +1 k
is given by nvm.
6:
end if7:
Predict x̂t +1 according to Eq. (7).
8:{nt +1, n0t +1, λ̂0t +1 } ← VMAllocSize( x̂t +1, Nt +1 ).
9: end for

Algorithm 2 Sizing of VM pool in private data center
1: function VMPoolSize( x̂t +1, x̂t +2, · · · , x̂t +w )2:
Nt +1 ← 0 and Λt +1 ← 0.
3:
while Λt +1 < max( x̂t +1, x̂t +2, · · · , x̂t +w ) do
4:
Calculate Λt +1 so as to satisfy Constraint (5) with substituting
Nt +1 .
5:for each t + h (h = 1, 2, · · · , w) do
6:{nt +h , n0t +h , λ0t +h } ← VMAllocSize( x̂t +h , Nt +1 ).
7:
Calculate Ct +h in accordance with Objective (1).
8:
end for
9:
The cost of the VM pool in this interval (denoted by C ∗ (Nt +1 ))P
is given by h=1,2,···, w Ct +h .
10:
Nt +1 ← Nt +1 + 1.
11:
end while
12:
return Nt +1 ← arg min C ∗ (Nt +1 ).
13: end function

N t +1the networks which have sufficient bandwidth.

Estimation of Response Time Distribution

π(nt , λ t ) =Algorithm 1 Resource allocation in hybrid cloud system

−1
nX

t −1
nt ρnt tρlt 
nt ρnt t+
nt ! (nt − ρt )  nt ! (nt − ρt ) l=0 l! 



(9)

Note that the above function is in the case of the private data
center, but this function is also applied for the public data
center. We also note that the network latency, r 0 , corresponds
the sums of latencies of a WAN and LANs, indeed. We
regard r 0 as a constant, because we neglect the buffering
delay and consider propagation and transmission delays in

4.

Method for Dynamically Allocating Resources

As explained in Sect. 2, we use long and short-term VM
provisioning. The size of a VM pool in the private data
center over the next w-time-slot interval ({Nt+h | h =1, · · · , w}(Nt+1 = · · · = Nt+w )) is determined on the basisof the predictions from one-time-slot-ahead to w-time-slot
ahead ({ x̂ t+h | h = 1, · · · , w}) at the end of each w-time-slot
interval, while the numbers of private and public VMs at the0 ) are recalculated by using thenext time slot (nt+1 and nt+1
one-time-slot-ahead prediction x̂ t+1 and Nt+1 at each timeslot, as given in Algorithm (1).
The VM pool size in the next interval (Nt+1 ) is determined so as to minimize Objective (1), which is counted up
with increasing Nt+1 from 0 (i.e., the case of a pure public data center) to more than the maximum of { x̂ t+h | h =1, · · · , w} (i.e., the case of a pure private data center), as
described in Algorithm (2). Moreover, the numbers of pri0 )vate and public VMs in the next time slot (nt+1 and nt+1
are determined by comparing the processing ability of the
VM pool in the private data center (Λt+1 ) with the predicted
request rate to an application system ( x̂ t+1 ), as shown in
Algorithm (3).
5.

Evaluation

In this section, we evaluate the total cost and response timeof web systems and analyze the effect of prediction errors on

OGAWA et al.: PREDICTION-BASED CLOUD BURSTING APPROACH AND ITS IMPACT ON TOTAL COST FOR BUSINESS-CRITICAL WEB SYSTEMS

2011

Algorithm 3 Sizing of VMs allocated in both data centers
1: function VMAllocSize( x̂t +h , Nt +1 )2:
Calculate Λt +1 so as to satisfy Constraint (5) with substituting
Nt +1 .
3:
if x̂t +h ≤ Λt +1 then4:
λ̂t +h ← x̂t +h .
5:
Calculate nt +h so as to satisfy Constraint (5) with substituting
λ̂t +h .
6:
λ̂0t +h ← 0 and n0t +h ← 0.
7:
else
8:
λ̂t +h ← Λt +1 and nt +h ← Nt +1 .
9:
λ̂0t +h ← x̂t +h − Λt +1 .
10:
Calculate n0t +h so as to satisfy Constraint (6) with substituting
λ̂0t +h .
11:
end if12:
return {nt +h , n0t +h , λ̂0t +h }.
13: end function

them through numerical simulations based on trace data ofactual web systems. In the evaluations, each time slot is set
to one-hour long in accordance with the billing interval of aprominent public data center [27].
5.1of transferred data per request (d) was 7800 bytes for a campus website and 4100 bytes for a consumer website; thesevalues were calculated on the basis of the figures recordedin the trace data.
Moreover, the personnel cost for operation and management (cst ) was set to ¥900,000 per month /(30 × 24) =¥1250 per time slot per staff member. A single staff member is assumed to be able to operate and manage up to 100
VMs (i.e., nst = 100) [21]. Furthermore, the constant for
specifying economics of scale (α) was set to 0.6 [23].
The processing rate of each private and public VM(µ) was set to 5.5 requests/s for the campus web and 275
requests/s for the consumer web. The µ of the consumer
web was set so that the maximum number of VMs for the
consumer web was similar to that for the campus web. In addition, the cost related to the servers was assumed to be 50%of the total cost including all of the devices and equipments
in a private data center [21]. The cost related to VMs was
similarly supposed to be 80% of the total cost in a publicdata center [27]. We thereby set a = 2 and a 0 = 1.25 in
Objective (1).

Simulation Settings
5.1.3

5.1.1

Datasets

We used the arrival traces collected from two web application
systems.
• 5-month access log (from April 1 to August 26, 2014)
for a campus website of a university with about 30,000
students and staff members (called a campus web).
• 2.5-month access log (from April 30 to July 16, 1998)
for the 1998 FIFA World Cup website [28] (called a
consumer web).
5.1.2

Cost Model

The description of physical servers in the private data center
was given as follows. All physical servers, which have 8 CPU
cores and a 32-GB memory each [29], were assumed to be
used on a three-year lease. The price of a single physicalserver was set to ¥600,000. Thus, the cost of renting aphysical server per time slot (cps ) was ¥600, 000/(3 × 365 ×24) =¥22.8 per physical server per hour. Each physicalserver had up to 2 VMs (i.e., nvm = 2). The energy charge
rate (cec ) was ¥16 per kWh [30], and the power consumptionof a single physical server (pps ) was set to 550 W . In addition,
the energy-proportional coefficient (e) of the physical servers
was set to 0.6 [22].
On the other hand, each public VM was assumed to
be a m4.2xlarge instance at Amazon EC2 [27], which performed similarly to a single VM when nvm = 2 in the private
data center. The price of a single on-demand VM (cvm ) was$0.732 per hour. The price for transferring data to/from thepublic data center (ctr ) was set to $0/$0.14 per GB, respectively, as well [27]. We converted dollars into yen at an
exchange rate of ¥120 to $1. In addition, the average amountResponse Time Constraints

In Constraints (5) and (6), target probability q was definedas 95%. The threshold of response time (r c ) was set to0.15 s because users can notice the response time when the
response delay exceeds this threshold [31]. Furthermore, thesums of latency of a WAN and LANs (r 0 in Eq. (8)) were set
to 0.001 s for the private data center and 0.14 s for the publicdata center.
5.1.4

Request Rate Prediction

Based on the observation of the trace data, the datasets had
weekly, i.e., 24 × 7 = 168 time slots, periodicity. We convert
the time series into a logarithmic scale for counteracting theeffect of the rapid increase and decrease. We then applied
the transformation of yt = (1− B)(1− B168 ) log10 x t in orderto make the original time series x t stationary. At each timeslot, we extracted the last three weeks, i.e., 24 × 21 = 504
time slots, of data and identified the values of p and q of
ARI M A(p, 1, q) by changing p (0 ≤ p ≤ 5) and q (0 ≤ q ≤5) until no lower AIC (Akaike Information Criterion) could
be found [25].
5.2
5.2.1

Evaluation Results
Prediction Error of Request Rate

We performed the allocation process shown in Algorithm (1)48 times by changing the starting time slot of a time horizon. Figure 3 shows an example of bursty requests and theirpredictions. Figure 4 shows the prediction accuracy, where
we analyzed the mean absolute percentage error (MAPE)
P
defined as T1 Tt=1 |xtx−tx̂t | . For the 168-time-slot, i.e., oneweek, predictions, the campus web showed relatively small

IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 2017

2012

Fig. 3

Example of bursty requests and predictions (for consumer web).

Fig. 5

Fig. 4

Prediction errors.

in the public data center and that for transferring data fromthem increased rapidly when Nt was close to 0 because thenumber of VMs in the public data center greatly expanded
at that time. Furthermore, the cost for VM operation andmanagement was reduced nearly linearly with Nt . As shownin Fig. 5, Objective (1) was minimized when Nt was
j k2 and
was almost unchanged until Nt = 4. In this case, 22 (= 1)physical server was reassigned to the consumer web in thenext week.
5.2.3

error (0.34 on average) because it had regular predictable patterns, while the consumer web showed a large error (0.94 on
average) because it sometimes received unexpected request
spikes. In contrast, the one-time-slot-ahead, i.e., one-hour,
predictions indicated relatively small errors in both webs (0.2
and 0.1 on average).
5.2.2

Sizing of VM Pool in Private Data CenterIn the private data center, the dedicated VM pool was resizedonce a week to minimize Objective (1). For example, Fig. 5
shows the Objective (1) value of a certain week as a functionof the size of the private VM pool (Nt ) in the case of theconsumer web, where the Objective (1) value is expressed interms of the cost relative to that when the application is deployed by using an existing provisioning approach (denoted
by Cexisting ). This Cexisting is calculated for when the system
is assigned private VMs able to handle the maximum request
rate of the time horizon and all VMs always stay active inthe private data center. In Fig. 5, the consumer web wasprocessed in the private data center alone when Nt was 11
and in the public data center alone when Nt was 0.
The cost associated with the physical servers in the private data center decreased linearly with Nt , where most of
the cost went to rent the physical servers under our evaluation settings. The cost of energy consumed by the physicalservers did not significantly change when Nt was large because the number of active VMs was almost the same in thatcase. On the other hand, the cost for using on-demand VMs

Example of total cost as function of Nt (consumer web, nvm = 2).

Total Cost and Response Time

Figure 6(a) indicates the evaluation results of the total cost,which is expressed as the ratio of the optimized one (C) tothe existing one (Cexisting ). Figure 6(b) further shows thoseof the response ratio of more than the threshold r c (0.15 s).
Each ideal assumes a case in which the future requests are
known a priori.
For the campus web, the relative total cost corresponded
to its ideal, and the response ratio of more than the thresholdr c (0.15 s) was totally below the (transformed) target probability of 0.05 (= 1 - q (0.95)) because both one-hour andone-week predictions had high accuracy. The response ratios
were almost the same for the starting time slots because theratio depended on the number of active VMs. Each number
was determined on the basis of the corresponding one-hourahead prediction having the same accuracy (see Fig. 4).
For the consumer web, the total cost was slightly larger
than its ideal, while the response ratio of more than 0.15 s
was much more than the target of 0.05 and reached 0.23. The
slight difference in the total cost was mainly caused by errors
in the one-week predictions. In this case, these errors shiftedto the positive side, resulting in an over-provisioned VM poolin the private data center. Furthermore, this optimized cost
was lower than that for the campus web, which meant thatthe consumer web was less utilized than the campus web
when both webs ran in the existing manner. On the other
hand, the response time was degraded by errors of the onehour predictions. Although the consumer web had small
MAPEs in the one-hour predictions, it sometimes received

OGAWA et al.: PREDICTION-BASED CLOUD BURSTING APPROACH AND ITS IMPACT ON TOTAL COST FOR BUSINESS-CRITICAL WEB SYSTEMS

2013

Fig. 7

Fig. 6

Evaluation results when nvm = 2.

bursty requests exceeding estimated values of the one-hour
predictions (see arrows in Fig. 3); these errors made VMs
under-provisioned, resulting in delaying the response time.
In addition, the ideal case of the consumer web was muchless than the target probability of 0.05 because the processing
rate of a single VM for the web was relatively large, which
absorbed a certain level of request rate variation.
5.2.4

Handling of One-Hour Prediction Errors

Since the VMs are allocated and activated/deactivated on
the basis of the point estimates for future request rates, estimation errors sometimes made VMs under-provisioned,
resulting in degrading the response time as in the case of
the consumer web explained in the previous section. To
prevent this response delay, we thus use the upper bound of
the interval estimate instead of the point estimate. Figure 7
shows the total cost and the response ratio as functions of
the upper bounds of the confidence interval for the one-hour
predictions in the case of the consumer web. The error bars
indicate the maximum and minimum of the 48 trials. Here,
we still used the point estimates for the one-week predictions.
Note that we provide two additional cases when nvm = 4 andnvm = 8 in Fig. 7, which will be explained in Sect. 5.2.6.

One-hour error handling for consumer web.

Figure 7 indicates a trade-off between the total cost
and the response-time performance. When we provisioned
with the upper bound of a 99.9% confidence interval, theresponse ratio was below the target probability (0.05) andthe total cost increased but still remained half that of Cexisting .
The errors of the one-hour predictions were relatively small,which suppressed the increase of the total cost. Figure 7 also
shows that the response ratio did not decrease below about
0.04 even when we applied the upper bound of a more than
99.99 % confidence interval; this reveals the limitation of theprediction-based provisioning using the fixed length (i.e., 1hour) for time slots in our evaluation environment. We also
note that the selection of a confidence level determines how
strictly the response time constraints are enforced, and thisstrictness depends on the requirements of a corresponding
application system.
5.2.5

Impact of One-Week Prediction Errors on Total Cost

Errors of one-week predictions change the size of the VM
pool in the private data center, which can impact the total
cost. On the x-axis of Fig. 8, positive and negative values
mean the upper and lower bounds of the confidence interval
for the one-week predictions. Zero on the x-axis means thatthe point estimates are applied. For example, 50% on thex-axis means that the private VM pool is over-provisioned
when the upper bound of a 50% confidence interval is used.
In contrast, -50% means that the pool is under-provisioned
when the lower bound of a 50% confidence interval is used.
We evaluated up to a 95% confidence interval for the oneweek predictions. Here, to make the response ratio less than
the target probability, for the one-hour predictions, the point
estimate was applied to the campus web and the upper boundof a 99.9% confidence interval was applied to the consumer
web. Note that 2 additional cases when nvm = 4 and nvm = 8
in Fig. 8 will be also discussed in Sect. 5.2.6.
Figure 8 reveals that the underestimate of the size of theprivate VM pool had little effect on the total cost. This was
also true for the overestimate, until we used the upper boundof a 50% confidence interval when nvm = 2 for the consumer
web (Fig. 8(b)). Although the one-week prediction values,

IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 2017

2014

Fig. 8Impact of one-week prediction error.

of course, included larger errors especially for the consumer
web, the total cost was tolerant of the prediction errors for thefollowing reasons. When we predicted the request rates, weconverted them into a logarithmic scale. Owing to this, lower
bounds of the confidence interval had smaller fluctuations
than upper bounds. Furthermore, the total cost stayed at an
equilibrium while Nt was in the range of up to two from thevalue making the cost optimal (see around Nt = 3 in Fig. 5).
These advantages come from the VM pool in the private data
center being provisioned for the average rate of requests, notfor the maximum rate.
5.2.6

Effect of the Processing Power of a Single VM

From Sect. 5.2.2 to Section 5.2.5, while mentioning the evaluation results when all physical servers in the private data
center had up to 2 VMs (i.e., nvm = 2) each and all VMs
in the public data center performed similarly, we discussed
how our provisioning approach is tolerant of prediction errors. When a single VM, however, has a large processing
power, it can absorb a somewhat high level of request fluctuation; this may lead to the error tolerance. We therefore
additionally examined cases when the VMs have less processing capacity.

Figure 7 shows the effect of a single VM processing
power on one-hour prediction errors, the resulting total costand response delay. Note that horizontal positions of theplots mentioned in Sect. 5.2.6 have been adjusted to keep themarkers visible. When all physical servers had up to 4 VMs(nvm = 4) and even 8 VMs (nvm = 8) each in the private
data center and VMs of corresponding performances (suchas m4.xlarge ($0.366 per hour) and m4.large ($0.183 perhour) instances at Amazon EC2 [27]) were provided in the
public data center, the total cost slightly became smaller andthe response ratio above 0.15 s appeared larger. This was
because a single VM of smaller processing capacity had lesssurplus capacity; this cut the total cost and enabled the VM
itself to afford lower request fluctuations caused by one-hour
prediction errors as well.
Figure 8 furthermore indicates the effect of a single VM
capacity on one-week prediction errors and the resulting total
cost. In the case of the campus web shown in Fig. 8(a), thetotal cost decreased as the processing capacity of a single
VM shrank from nvm = 2 to nvm = 8, for the same reason
as in the case of the one-hour error handling above. In thiscase, the total cost was little affected by the VM pool overand under-provisioned when the upper and lower bounds ofthe confidence interval were used, respectively, because thecampus web showed relatively small errors for one-week
predictions. Moreover, in the case of the consumer web
given in Fig. 8(b), when we used the lower bound of a 95%
confidence interval, the total cost was slightly increased as
the processing capacity became small. This was because thesmaller VMs in the under-provisioned private VM pool had
little redundant capacity and needed more on-demand public
VMs of relatively high cost. On the other hand, when weused the upper bound of a 95% confidence interval, the total
cost was significantly increased as the processing capacity
enlarged from nvm = 8 to nvm = 2 for the following reasons.
The request rates was converted into a logarithmic scale
for being predicted and the consumer web had larger errors
for one-week predictions shown in Fig. 4. These made theprivate VM pool of the consumer web much over-provisioned
when the upper bound of a 95% confidence interval wasused. The over-provisioned private VM pool had the largest
redundant capacity when the processing capacity of a single
VM is the biggest (i.e., nvm = 2); this greatly increased thetotal cost.
As explained above, the total cost and the response
ratio was affected by the processing capacity of a single VM.
However, the total cost could still be half the existing costand 95% of response time was less than 0.15 s, even if thepredicted request ratio contained somewhat large errors.
6.

Conclusion

This paper presented a cloud bursting approach in which
we assign a dedicated VM pool for a business-critical web
system in a private data center on the basis of one-week
predictions and determine which VMs in private and public data centers should be active on the basis of one-hour

OGAWA et al.: PREDICTION-BASED CLOUD BURSTING APPROACH AND ITS IMPACT ON TOTAL COST FOR BUSINESS-CRITICAL WEB SYSTEMS

2015

predictions. We evaluated how prediction errors affect theapproach through numerical simulations based on trace dataof actual web systems. To avoid the response delay causedby the one-hour prediction errors, we needed to apply theupper bound of a wider confidence interval, resulting increasing the total cost of the system. However, when theupper bound of a 99.9% confidence interval was used for the
one-hour predictions to keep 95% of response time less than
0.15 s, the total cost was still half the existing cost, i.e, thecost when VMs are provided to handle the maximum request
rate and all VMs always stay active in the private data center alone. Furthermore, the total cost was nearly unchanged
when the VM pool in the private data center was under- orover-provisioned for the one-week predictions. These characteristics of our approach, furthermore, are only slightly
dependent on the processing capacity of a single VM.
The length of time slots used for predicting request rates
and handling VMs was fixed to one hour, which may limit
the prediction accuracy of request rates and the resulting
distribution of response time as well. Moreover, dynamic
adjustment of the confidence level is beyond the scope of
this paper. A topic of future study is therefore improving
the provisioning accuracy by shortening the time slots andadjusting dynamically the confidence level, with using more
trace data for further analysis of this study.
References[1] M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, G. Lee, D.A. Patterson, A. Rabkin, I. Stoica, and M. Zaharia,“Above the clouds: A berkeley view of cloud computing,” Technical Report, UCB/EECS-2009-28, EECS Department, University ofCalifornia, Berkeley, Feb. 2009.
[2] J. Barr, “Cloudbursting - hybrid application hosting,” https://aws.
amazon.com/jp/blogs/aws/cloudbursting/, Aug. 2008. accessed Dec.
9, 2016.
[3] A.R. Hummaida, N.W. Paton, and R. Sakellariou, “Adaptation incloud resource configuration: A survey,” J. Cloud Comput., vol.5,
no.1, pp.57:1–57:16, Dec. 2016.
[4] T. Lorido-Botran, J. Miguel-Alonso, and J. Lozano, “A review of
auto-scaling techniques for elastic applications in cloud environments,” J. Grid Comput., vol.12, no.4, pp.559–592, Dec. 2014.
[5] H. Alipour, Y. Liu, and A. Hamou-Lhadj, “Analyzing auto-scaling
issues in cloud environments,” Proc. 24th Annual International Conference on Computer Science and Software Engineering, pp.75–89,
Nov. 2014.
[6] R.N. Calheiros, E. Masoumi, R. Ranjan, and R. Buyya, “Workload
prediction using ARIMA model and its impact on cloud applications’
QoS,” IEEE Trans. Cloud Comput., vol.3, no.4, pp.449–458, Oct
2015.
[7] Y. Niu, B. Luo, F. Liu, J. Liu, and B. Li, “When hybrid cloud meets
flash crowd: Towards cost-effective service provisioning,” Proc.
2015 IEEE Conference on Computer Communications, pp.1044–
1052, April 2015.
[8] H.Y. Chu and Y. Simmhan, “Cost-efficient and resilient job lifecycle management on hybrid clouds,” Proc. IEEE 28th International
Parallel and Distributed Processing Symposium, pp.327–336, May
2014.
[9] M. HoseinyFarahabady, H. Samani, L. Leslie, Y.C. Lee, andA. Zomaya, “Handling uncertainty: Pareto-efficient bot schedulingon hybrid clouds,” Proc. 42nd International Conference on Parallel
Processing, pp.419–428, Oct. 2013.

[10] S. Imai, T. Chestna, and C.A. Varela, “Accurate resource prediction for hybrid IaaS clouds using workload-tailored elastic computeunits,” Proc. IEEE/ACM 6th International Conference on Utility andCloud Computing, pp.171–178, Dec. 2013.
[11] T. Guo, U. Sharma, P. Shenoy, T. Wood, and S. Sahu, “Cost-aware
cloud bursting for enterprise applications,” ACM Trans. Internet
Technol., vol.13, no.3, pp.10:1–10:24, May 2014.
[12] H. Zhang, G. Jiang, K. Yoshihira, and H. Chen, “Proactive workload
management in hybrid cloud computing,” IEEE Trans. Netw. Serv.
Manage., vol.11, no.1, pp.90–100, March 2014.
[13] M. Bjorkqvist, L. Chen, and W. Binder, “Cost-driven service provisioning in hybrid clouds,” Proc. 5th IEEE International Conference on Service-Oriented Computing and Applications, pp.1–8, Dec.
2012.
[14] G. Sun, Z. Lu, J. Wu, X. Wang, and P. Hung, “A novel reactivepredictive hybrid resource provision method in cloud datacenter,”
pp.33–47, Lecture Notes in Computer Science, vol.9464, pp.33–47,
Springer Nature, 2015.
[15] N. Roy, A. Dubey, and A. Gokhale, “Efficient autoscaling in the cloudusing predictive models for workload forecasting,” Proc. 2011 IEEE
4th International Conference on Cloud Computing, pp.500–507, July
2011.
[16] VMware, Inc., “vSphere and vSphere with Operations Management.”
http://www.vmware.com/products/vsphere.html. accessed Dec. 9,
2016.
[17] C. Dixon, D. Olshefski, V. Jain, C. DeCusatis, W. Felter, J. Carter,
M. Banikazemi, V. Mann, J.M. Tracey, and R. Recio, “Software
defined networking to support the software defined environment,”
IBM J. Res. & Dev., vol.58, no.2/3, pp.3:1–3:14, March 2014.
[18] Y. Ogawa, G. Hasegawa, and M. Murata, “Cloud bursting approachbased on predicting requests for business-critical web systems,” Proc.
International Conference on Computing, Networking and Communications, pp.443–447, Jan. 2017.
[19] J. Weinman, “Hybrid cloud economics,” IEEE Cloud Comput., vol.3,
no.1, pp.18–22, Jan. 2016.
[20] C. Tang, M. Steinder, M. Spreitzer, and G. Pacifici, “A scalable
application placement controller for enterprise data centers,” Proc.
16th International Conference on World Wide Web, pp.331–340,
May 2007.
[21] A. Greenberg, J. Hamilton, D.A. Maltz, and P. Patel, “The cost of
a cloud: Research problems in data center networks,” SIGCOMM
Comput. Communi. Rev., vol.39, pp.68–73, Dec. 2008.
[22] D. Wong and M. Annavaram, “KnightShift: Scaling the energy
proportionality wall through server-level heterogeneity,” Proc. 45th
Annual IEEE/ACM International Symposium on Microarchitecture,
pp.119–130, Dec. 2012.
[23] F.T. Moore, “Economies of scale: Some statistical evidence,” The
Quarterly Journal of Economics, vol.73, no.2, pp.232–245, May
1959.
[24] L.A. Barroso and U. Hölzle, “The case for energy-proportional computing,” IEEE Computer, vol.40, no.12, pp.33–37, Dec. 2007.
[25] R.J. Hyndman and G. Athanasopoulos, “Forecasting: Principles andpractice.” https://www.otexts.org/book/fpp. accessed Dec. 9, 2016.
[26] R. Jain, The Art Of Computer Systems Performance Analysis, John
Wiley & Sons, April 1991.
[27] Amazon.com, Inc., “Amazon Elastic Compute Cloud (EC2).”
http://aws.amazon.com/ec2/. accessed Dec. 9, 2016.
[28] The Internet Traffic Archive, “1998 World Cup Web Site Access Logs.” http://ita.ee.lbl.gov/html/contrib/WorldCup.html. accessed Dec. 9, 2014.
[29] Dell Inc., “PowerEdge R430.” http://www.dell.com/jp/business/p/
poweredge-r430/pd. accessed Dec. 9, 2016.
[30] Tokyo Electric Power Company Holdings, Inc. http://www.tepco.co.
jp/ep/corporate/plan_h/index-j.html. accessed Dec. 9, 2016.
[31] J.D. McCabe, Network Analysis, Architecture and Design, Second
Edition, Morgan Kaufmann Publishers, San Francisco, 2003.

IEICE TRANS. COMMUN., VOL.E100–B, NO.11 NOVEMBER 20172016

Yukio Ogawa
received his M.S. degreein Science from Nagoya University, Japan, in
1994, and Ph.D. degree in Information Scienceand Technology from Osaka University, Japan,
in 2012. He joined the Hitachi Central ResearchLaboratory in Japan, in 1994. He is currently an
associate professor at the Center for Multimedia
Aided Education, Muroran Institute of Technology. His research interests include network architectures for cloud systems. He is a memberof IEEE, ACM and IEICE.

Go Hasegawa
received the M.E. and D.E.
degrees in Information and Computer Sciencesfrom Osaka University, Osaka, Japan, in 1997
and 2000, respectively. From July 1997 to June
2000, he was a Research Assistant in the Graduate School of Economics, Osaka University. He
is now an Associate Professor at the Cybermedia Center, Osaka University. His research is in
the area of transport architecture for future highspeed networks and overlay networks. He is amember of IEEE and IEICE.

Masayuki Murata
received the M.E. and
D.E. degrees in Information and Computer Science from Osaka University, Japan, in 1984 and1988, respectively. In April 1984, he joinedTokyo Research Laboratory, IBM Japan, as a
Researcher. From September 1987 to January
1989, he was an Assistant Professor with Computation Center, Osaka University. In February
1989, he moved to the Department of Information and Computer Sciences, Faculty of Engineering Science, Osaka University. In April
1999, he became a Professor of Cybermedia Center, Osaka University,and is now with Graduate School of Information Science and Technology,
Osaka University since April 2004. He has more than nine hundred papersof international and domestic journals and conferences. His research interests include computer communication networks, performance modelingand evaluation. He is a member of IEEE, ACM and IEICE.