# Analyzing Big Data with the Hybrid Interval Regression Methods

Authors: Chia-Hui Huang; Keng-Chieh Yang; Han-Ying Kao
Journal: The Scientific World Journal
 (2014)

---

## Abstract

Big data is a new trend at present, forcing the significant impacts on information technologies. In big data applications, one of the most concerned issues is dealing with large-scale data sets that often require computation resources provided by public cloud services. How to analyze big data efficiently becomes a big challenge. In this paper, we collaborate interval regression with the smooth support vector machine (SSVM) to analyze big data. Recently, the smooth support vector machine (SSVM) was proposed as an alternative of the standard SVM that has been proved more efficient than the traditional SVM in processing large-scale data. In addition the soft margin method is proposed to modify the excursion of separation margin and to be effective in the gray zone that the distribution of data becomes hard to be described and the separation margin between classes.


---

## Body

## 1. Introduction

Big data has become one of new research frontiers. Generally speaking, big data is a collection of large-scale and complex data sets that it becomes more difficult to process using current database management systems and traditional data processing applications. In 2012, Gartner Inc. gave a definition of big data as “Big data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization” [1]. The trend of big data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data.One of the major applications of the future parallel, distributed, and cloud systems is in big data analytic [2–5]. Most concerned issues are dealing with large-scale sets which often require computation resources provided by public cloud services. How to analyze big data efficiently becomes a big challenge.The support vector machine (SVM) has shown to be an efficient approach for a variety of data mining, classification, analysis, pattern recognition, and distribution estimation [6–14]. Recently, using SVM to solve the interval regression model [15] has become an alternative approach. Hong and Hwang [16] evaluated interval regression models with quadratic loss SVM. Bisserier et al. [17] proposed a revisited fuzzy regression method where a linear model is identified from Crisp-Inputs Fuzzy-Outputs (CISO) data. D’Urso et al. [18] presented fuzzy clusterwise regression analysis with LR fuzzy response variable and numeric explanatory variables. The suggested model is to allow for linear and nonlinear relationship between the output and input variables. Jeng et al. [19] developed a support vector interval regression networks (SVIRNs) based on both SVM and neural networks. Huang and Kao [20] proposed a soft-margin SVM for interval regression analysis. Huang [21] solved interval regression model with reduced support vector machine.However, there are several main problems while using SVM model.(1)
Big data: when dealing with big data sets, the solution by using SVM with a nonlinear kernel may be difficult to be found.
(2)
Noises and interaction: the distribution of data becomes hard to be described and the separation margin between classes becomes a “gray” zone.
(3)
Unbalance: the number of samples from one class is much larger than the number of samples from other classes. It causes the excursion of separation margin.Under this circumstance, developing an efficient method to analyze big data becomes important. The smooth support vector machine (SSVM) has been proved more efficient than the traditional SVM in processing large-scale data [22–24]. The main idea of SSVM is solved by a fast Newton-Armijo algorithm [25] and has been extended to nonlinear separation surfaces by using a nonlinear kernel technology [24].In this study, we collaborate interval regression [15] with SSVM to analyze big data. The main idea of SSVM is solved by a fast Newton-Armijo algorithm and has been extended to nonlinear separation surfaces by using a nonlinear kernel technology. Additionally, to modify the excursion of separation margin and to be effective in the gray zone, the soft margin method is proposed. The experiment results show that the proposed methods are more efficient than existing methods.This study is organized as follows. Section2 reviews the current methods for interval regression analysis. Section 3 proposes the soft margin method and the formulation of interval regression with SSVM to analyze big data. Section 4 gives a numerical example by the proposed methods dealing with big data which is extracted from Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) [26]. Finally, Section 5 gives the concluding remarks.


## 2. Literature Review

Since Tanaka et al. [27] introduced the fuzzy regression model with symmetric fuzzy parameters, the properties of fuzzy regression have been studied extensively by many researchers. Fuzzy regression model can be simplified to interval regression analysis which is considered as the simplest version of possibilistic regression analysis with interval coefficients. An interval linear regression model is described as
(1)
Y
(
x
j
)
=
A
0
+
A
1
x
1
j
+
⋯
+
A
n
x
n
j
,
where Y
(
x
j
), j
=
1,2
,
…
,
q, is the estimated interval corresponding to the real input vector x
j
=
(
x
1
j
,
x
2
j
,
…
,
x
n
j
)
t. An interval coefficient A
i is defined as (
a
i
,
c
i
), where a
i is the center and c
i is the radius. Hence, A
i can also be represented as
(2)
A
i
=
[
a
i
-
c
i
,
a
i
+
c
i
]
=
{
a
i
-
c
i
≤
a
≤
a
i
+
c
i
}
.The interval linear regression model (1) can also be expressed as
(3)
Y
(
x
j
)
=
A
0
+
A
1
x
1
j
+
⋯
+
A
n
x
n
j
=
(
a
0
,
c
0
)
+
(
a
1
,
c
1
)
x
1
j
+
⋯
+
(
a
n
,
c
n
)
x
n
j
=
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
,
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)
.For a data set with crisp inputs and interval outputs, two interval regression models, the possibility and necessity models, are considered. By assumption, the center coefficients of the possibility regression model and the necessity regression model are the same [15]. For this data set, the possibility and necessity estimation models are defined as
(4)
Y
*
(
x
j
)
=
A
0
*
+
A
1
*
x
1
j
+
⋯
+
A
n
*
x
n
j
Y
*
(
x
j
)
=
A
0
*
+
A
1
*
x
1
j
+
⋯
+
A
n
*
x
n
j
,
where the interval coefficients A
i
* and A
i
* are defined as A
i
*
=
(
a
i
*
,
c
i
*
) and A
i
*
=
(
a
i
*
,
c
i
*
), respectively. The interval Y
*
(
x
j
) estimated by the possibility model must include the observed interval Y
j and the interval Y
*
(
x
j
) estimated by the necessity model must be included in the observed interval Y
j.In this section, we review the current methods which are ordinarily used for interval regression analysis.


### 2.1. Tanaka and Lee’s Approach

Tanaka and Lee [15] proposed an interval regression analysis with a quadratic programming (QP) approach which gives more diverse spread coefficients than a linear programming (LP) one.The interval regression analysis by QP approach unifying the possibility and necessity models subject to the inclusion relations,Y
*
(
x
j
)
⊆
Y
j
⊆
Y
*
(
x
j
), can be represented as
(5)
min
⁡
∑
j
=
1
q
‍
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
2
+
φ
∑
i
=
0
n
‍
(
a
i
2
+
c
i
2
)
i
s
.
t
.
0
Y
*
(
x
j
)
⊆
Y
j
⊆
Y
*
(
x
j
)
,
j
=
1,2
,
…
,
q
s
.
t
.
0
i
c
i
,
d
i
≥
0
,
i
=
0,1
,
…
,
n
,
where φ is an extremely small positive number and makes the influence of the term φ
∑
i
=
0
n
‍
(
a
i
2
+
c
i
2
) on the objective function negligible. The constraints of the inclusion relations are equivalent to
(6)
Y
*
(
x
j
)
⊆
Y
j
⟺
{
y
j
-
e
j
≤
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
-
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
+
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)
≤
y
j
+
e
j
,
(7)
Y
j
⊆
Y
*
(
x
j
)
⟺
{
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
-
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)

-
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
≤
y
j
-
e
j

y
j
+
e
j
≤
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
+
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)

y
j
+
e
j
≤
+
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
,

where x
j is the jth input vector and Y
j is the corresponding interval output that consists of a center y
j and a radius e
j denoted by Y
j
=
(
y
j
,
e
j
).


### 2.2. Hong and Hwang’s Approach

Hong and Hwang [16] evaluated interval regression model combining the possibility and necessity estimation formulation with the principle of quadratic loss support vector machine (QLSVM). This version of SVM utilizes the quadratic loss function. The QLSVM performs interval nonlinear regression analysis by constructing an interval linear regression function in high-dimensional feature space.With the principle of QLSVM, the interval nonlinear regression model is given as follows:(8)
max
⁡
-
1
2
(
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
2
j
-
λ
2
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
4
i
-
λ
4
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
4
i
+
λ
4
i
*
)
(
λ
4
j
+
λ
4
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
4
j
+
λ
4
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
λ
1
i
λ
1
j
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
λ
1
i
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
)
max
⁡
-
1
2
C
∑
i
=
1
n
‍
λ
1
i
2
-
1
2
C
∑
i
=
1
n
‍
(
λ
2
i
2
+
λ
2
i
*
2
)
max
⁡
+
∑
i
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
y
i
+
∑
i
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
4
i
-
λ
4
i
*
)
y
i
max
⁡
+
∑
i
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
ϵ
i
-
∑
i
=
1
n
‍
(
λ
4
i
+
λ
4
i
*
)
ϵ
i
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
2
i
*
,
λ
3
i
,
λ
3
i
*
,
λ
4
i
,
λ
4
i
*
≥
0
,
where λ
1
i, λ
2
i, λ
2
i
*, λ
3
i, λ
3
i
*, λ
4
i, and λ
4
i
* are Lagrange multipliers. K
(
*
) is a nonlinear kernel. The followings are well-known nonlinear kernels, where σ, γ, r, h, and θ are kernel parameters:(1)
Gaussian (radial basis) kernel:e
-
∥
x
i
-
x
j
∥
2
/
2
σ
2, σ
>
0 [10],
(2)
hyperbolic tangent kernel:tanh
⁡
(
γ
x
i
x
j
t
+
θ
), γ
>
0 [12],
(3)
polynomial kernel:(
γ
x
i
x
j
t
+
r
)
h, h
∈
N, γ
>
0, and r
≥
0 [14].The advantage of Hong and Hwang’s approach is a model-free method in the sense that there is no need to assume the underlying model function for interval nonlinear regression model with crisp inputs and interval output.


### 2.3.  Huang’s Approach

There are two problems while using the traditional SVM model. (1) Large scale: when dealing with large-scale data sets, the solution may be difficult to be found when using SVM with nonlinear kernels; (2) Unbalance: the number of samples from one class is much larger than the number of samples from the other classes. It causes the excursion of separation margin.To resolve these problems, Huang [21] proposed a reduced support vector machine (RSVM) approach in evaluating interval regression models. RSVM has been proven more efficient than the traditional SVM in processing large-scale data.With the principle of RSVM, the interval nonlinear regression model is listed as follows:(9)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
λ
1
i
λ
1
j
Q
·
,
K
t
Q
·
,
K
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
2
j
-
λ
2
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
∑
i
,
j
=
1
n
‍
λ
1
i
Q
·
,
K
(
λ
2
j
-
λ
2
j
*
)
x
j
max
⁡
+
∑
i
,
j
=
1
n
‍
λ
1
i
Q
·
,
K
(
λ
3
j
-
λ
3
j
*
)
x
j
max
⁡
+
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
(
λ
2
j
+
λ
2
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
+
∑
i
,
j
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
4
C
∑
i
=
1
n
‍
λ
1
i
2
+
∑
i
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
ϵ
i
+
∑
i
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
ϵ
i
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
2
i
*
,
λ
3
i
,
λ
3
i
*
≥
0
,
where λ
1
i, λ
2
i, λ
2
i
*, λ
3
i, and λ
3
i
* are Lagrange multipliers. Q is a positive semidefinite matrix in RSVM. K
(
*
) is a nonlinear kernel.The advantage of Huang’s approach is to reduce the number of support vectors by randomly selecting a subset of samples. While processing with large-scale data sets, the solution can be found easily by the proposed method with nonlinear kernels.


## 2.1. Tanaka and Lee’s Approach

Tanaka and Lee [15] proposed an interval regression analysis with a quadratic programming (QP) approach which gives more diverse spread coefficients than a linear programming (LP) one.The interval regression analysis by QP approach unifying the possibility and necessity models subject to the inclusion relations,Y
*
(
x
j
)
⊆
Y
j
⊆
Y
*
(
x
j
), can be represented as
(5)
min
⁡
∑
j
=
1
q
‍
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
2
+
φ
∑
i
=
0
n
‍
(
a
i
2
+
c
i
2
)
i
s
.
t
.
0
Y
*
(
x
j
)
⊆
Y
j
⊆
Y
*
(
x
j
)
,
j
=
1,2
,
…
,
q
s
.
t
.
0
i
c
i
,
d
i
≥
0
,
i
=
0,1
,
…
,
n
,
where φ is an extremely small positive number and makes the influence of the term φ
∑
i
=
0
n
‍
(
a
i
2
+
c
i
2
) on the objective function negligible. The constraints of the inclusion relations are equivalent to
(6)
Y
*
(
x
j
)
⊆
Y
j
⟺
{
y
j
-
e
j
≤
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
-
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
+
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)
≤
y
j
+
e
j
,
(7)
Y
j
⊆
Y
*
(
x
j
)
⟺
{
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
-
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)

-
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
≤
y
j
-
e
j

y
j
+
e
j
≤
(
a
0
+
∑
i
=
1
n
‍
a
i
x
i
j
)
+
(
c
0
+
∑
i
=
1
n
‍
c
i
|
x
i
j
|
)

y
j
+
e
j
≤
+
(
d
0
+
∑
i
=
1
n
‍
d
i
|
x
i
j
|
)
,

where x
j is the jth input vector and Y
j is the corresponding interval output that consists of a center y
j and a radius e
j denoted by Y
j
=
(
y
j
,
e
j
).


## 2.2. Hong and Hwang’s Approach

Hong and Hwang [16] evaluated interval regression model combining the possibility and necessity estimation formulation with the principle of quadratic loss support vector machine (QLSVM). This version of SVM utilizes the quadratic loss function. The QLSVM performs interval nonlinear regression analysis by constructing an interval linear regression function in high-dimensional feature space.With the principle of QLSVM, the interval nonlinear regression model is given as follows:(8)
max
⁡
-
1
2
(
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
2
j
-
λ
2
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
4
i
-
λ
4
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
4
j
-
λ
4
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
(
λ
4
i
+
λ
4
i
*
)
(
λ
4
j
+
λ
4
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
4
j
+
λ
4
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
+
∑
i
,
j
=
1
n
‍
λ
1
i
λ
1
j
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
2
000
-
2
∑
i
,
j
=
1
n
‍
λ
1
i
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
)
max
⁡
-
1
2
C
∑
i
=
1
n
‍
λ
1
i
2
-
1
2
C
∑
i
=
1
n
‍
(
λ
2
i
2
+
λ
2
i
*
2
)
max
⁡
+
∑
i
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
y
i
+
∑
i
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
4
i
-
λ
4
i
*
)
y
i
max
⁡
+
∑
i
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
ϵ
i
-
∑
i
=
1
n
‍
(
λ
4
i
+
λ
4
i
*
)
ϵ
i
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
2
i
*
,
λ
3
i
,
λ
3
i
*
,
λ
4
i
,
λ
4
i
*
≥
0
,
where λ
1
i, λ
2
i, λ
2
i
*, λ
3
i, λ
3
i
*, λ
4
i, and λ
4
i
* are Lagrange multipliers. K
(
*
) is a nonlinear kernel. The followings are well-known nonlinear kernels, where σ, γ, r, h, and θ are kernel parameters:(1)
Gaussian (radial basis) kernel:e
-
∥
x
i
-
x
j
∥
2
/
2
σ
2, σ
>
0 [10],
(2)
hyperbolic tangent kernel:tanh
⁡
(
γ
x
i
x
j
t
+
θ
), γ
>
0 [12],
(3)
polynomial kernel:(
γ
x
i
x
j
t
+
r
)
h, h
∈
N, γ
>
0, and r
≥
0 [14].The advantage of Hong and Hwang’s approach is a model-free method in the sense that there is no need to assume the underlying model function for interval nonlinear regression model with crisp inputs and interval output.


## 2.3.  Huang’s Approach

There are two problems while using the traditional SVM model. (1) Large scale: when dealing with large-scale data sets, the solution may be difficult to be found when using SVM with nonlinear kernels; (2) Unbalance: the number of samples from one class is much larger than the number of samples from the other classes. It causes the excursion of separation margin.To resolve these problems, Huang [21] proposed a reduced support vector machine (RSVM) approach in evaluating interval regression models. RSVM has been proven more efficient than the traditional SVM in processing large-scale data.With the principle of RSVM, the interval nonlinear regression model is listed as follows:(9)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
λ
1
i
λ
1
j
Q
·
,
K
t
Q
·
,
K
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
2
j
-
λ
2
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
∑
i
,
j
=
1
n
‍
λ
1
i
Q
·
,
K
(
λ
2
j
-
λ
2
j
*
)
x
j
max
⁡
+
∑
i
,
j
=
1
n
‍
λ
1
i
Q
·
,
K
(
λ
3
j
-
λ
3
j
*
)
x
j
max
⁡
+
∑
i
,
j
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
(
λ
3
j
-
λ
3
j
*
)
K
(
x
i
,
x
j
)
max
⁡
-
1
2
∑
i
,
j
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
(
λ
2
j
+
λ
2
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
+
∑
i
,
j
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
∑
i
,
j
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
(
λ
3
j
+
λ
3
j
*
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
-
1
4
C
∑
i
=
1
n
‍
λ
1
i
2
+
∑
i
=
1
n
‍
(
λ
2
i
-
λ
2
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
3
i
-
λ
3
i
*
)
y
i
max
⁡
-
∑
i
=
1
n
‍
(
λ
2
i
+
λ
2
i
*
)
ϵ
i
+
∑
i
=
1
n
‍
(
λ
3
i
+
λ
3
i
*
)
ϵ
i
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
2
i
*
,
λ
3
i
,
λ
3
i
*
≥
0
,
where λ
1
i, λ
2
i, λ
2
i
*, λ
3
i, and λ
3
i
* are Lagrange multipliers. Q is a positive semidefinite matrix in RSVM. K
(
*
) is a nonlinear kernel.The advantage of Huang’s approach is to reduce the number of support vectors by randomly selecting a subset of samples. While processing with large-scale data sets, the solution can be found easily by the proposed method with nonlinear kernels.


## 3. Proposed Methods

In this section we first propose the soft margin method to modify the excursion of separation margin and to be effective in the gray zone. Then the formulation of interval regression with SSVM to analyze big data is introduced.


### 3.1. Soft Margin

In a conventional SVM, the sign function is used as the decision-making function. The separation threshold of the sign function is 0, which results in an excursion of separation margin for unbalanced data sets. The aim of the hard-margin separation margin is to find a hyperplane with the largest distance to the nearest training data. However, the limitations of the hard-margin formulation are as follows:(1)
there is no separating hyperplane for certain training data;
(2)
complete separation with zero training error will lead to suboptimal prediction error;
(3)
it is difficult to deal with the gray zone between classes.Thus, the soft margin method is proposed to modify the excursion of separation margin and to be effective in the gray zone. The soft margin is defined as(10)
f
-
(
δ
)
=
arctan
(
-
δ
·
s
+
ϑ
·
s
)
π
+
0.5
,
f
+
(
δ
)
=
arctan
(
δ
·
s
-
ϑ
·
s
)
π
+
0.5
,
where δ is the decision value. ϑ and s are offset parameter and scale parameter which need to be estimated using statistical method.With the soft margin as shown in Figure1, the predication of the class labels can be determined as follows:
(11)
y
(
x
)
=
{
-
1
,
if
(
ν
r
<
f
-
(
δ
)
,
δ
<
ϑ
)
or
(
ν
r
>
f
+
(
δ
)
,
δ
>
ϑ
)
+
1
,
if
(
ν
r
>
f
-
(
δ
)
,
δ
<
ϑ
)
or
(
ν
r
<
f
+
(
δ
)
,
δ
>
ϑ
)
,
where ν
r is a random number between 0 and 1.Figure 1
Soft margin.


### 3.2. Interval Regression with SSVM

The main idea of smooth support vector machine (SSVM) is solved by a fast Newton-Armijo algorithm [25] and has been extended to nonlinear separation surfaces by using a nonlinear kernel technology [24].Suppose thatm training data {
x
i
,
y
i
}, i
=
1,2
,
…
,
m, are given, where x
i
∈
R
n are the input patterns and y
i
∈
{
-
1,1
} are the related target values of two-class pattern classification case. Then the standard support vector machine with a linear kernel [14] is
(12)
min
⁡
w
,
b
,
ξ
1
2
∥
w
∥
2
+
C
∑
i
=
1
m
‍
ξ
i
2
s
.
t
.
y
i
(
w
t
x
i
+
b
)
≥
1
-
ξ
i
s
.
t
.
ξ
i
≥
0
,
i
=
1,2
,
…
,
m
,
where b is the location of hyperplane relative to the origin. The regularization constant C is a positive parameter to control the tradeoff between the training error and the part of maximizing the margin that is achieved by minimizing ∥
w
∥
2. ξ
i is the slack variable with weight C
/
2. ∥
w
∥ is the Euclidean norm of w which is the normal to the following hyperplanes:
(13)
w
t
x
i
+
b
=
+
1
,
for
y
i
=
+
1
,
(14)
w
t
x
i
+
b
=
-
1
,
for
y
i
=
-
1
.The first hyperplane (13) bounds the class {
+
1
} and the second hyperplane (14) bounds the class {
-
1
}. The linear separating hyperplane is
(15)
w
t
x
i
+
b
=
0
.In Lee and Mangasarian’s approach [24], b
2
/
2 is added to the objective function of (12). This is equivalent to adding a constant feature to the training data and finding a separating hyperplane through the origin. Consider
(16)
min
⁡
w
,
b
,
ξ
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
ξ
i
2
s
.
t
.
0
y
i
(
w
t
x
i
+
b
)
≥
1
-
ξ
i
s
.
t
.
0
ξ
i
≥
0
,
i
=
1,2
,
…
,
m
,
where ξ
i
=
{
1
-
y
i
(
w
t
x
i
+
b
)
}
+ for all i and the “+” function is defined as x
+
=
max
⁡
{
0
,
x
}. Then (12) can be reformulated as the following minimization problem by replacing ξ
i with {
1
-
y
i
(
w
t
x
i
+
b
)
}
+:
(17)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
{
1
-
y
i
(
w
t
x
i
+
b
)
}
+
2
.The objective function in (17) is not twice differentiable and can be solved by using a fast Newton-Armijo method [25]. Thus the “+” function in SSVM is approximated by a smooth function, p
(
x
,
α
), as follows:
(18)
p
(
x
,
α
)
=
x
+
1
α
log
⁡
(
1
+
e
-
α
x
)
,
α
>
0
,
where α
>
0 is the smooth parameter. 1
/
(
1
+
e
-
α
x
) is the integral of the sigmoid function of neural networks [28]. The p
(
x
,
α
) with a smoothing parameter α is to replace the “+” function of (17) to obtain the following smooth support vector machine (SSVM) with a linear kernel:
(19)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
.For specific data sets, an appropriate nonlinear mappingx
↦
ϕ
(
x
) can be used to embed the original R
n features into a Hilbert feature space F, ϕ
:
R
n
↦
F, with a nonlinear kernel K
(
x
i
,
x
j
)
≡
ϕ
(
x
i
)
t
ϕ
(
x
j
). Thus, (19) can be extended to the SSVM with a nonlinear kernel:
(20)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b
)
}
,
α
)
2
,
where ∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b is the nonlinear SSVM classifier. The coefficient v
j is determined by solving an optimization problem (20) and the data points with corresponding nonzero coefficients.With the principle of SSVM, we can formulate the interval linear regression model as follows:(21)
min
⁡
a
¯
,
c
¯
,
d
¯
1
2
(
a
¯
t
a
¯
+
c
¯
t
c
¯
+
d
¯
t
d
¯
+
b
2
)
min
⁡
a
¯
,
c
¯
,
d
¯
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
s
.
t
.
a
¯
x
i
+
c
¯
|
x
i
|
≤
y
i
+
e
i
s
.
t
.
a
¯
x
i
-
c
¯
|
x
i
|
≥
y
i
-
e
i
s
.
t
.
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
≥
y
i
+
e
i
s
.
t
.
a
¯
x
i
-
c
¯
|
x
i
|
-
d
¯
|
x
i
|
≤
y
i
-
e
i
s
.
t
.
i
=
1,2
,
…
,
m
,
where a
¯, c
¯, and d
¯ are the collections of all a
i, c
i, and d
i, i
=
1,2
,
…
,
m, respectively.Given (21), the corresponding Lagrangian objective function is
(22)
L
≔
1
2
(
a
¯
t
a
¯
+
c
¯
t
c
¯
+
d
¯
t
d
¯
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
-
∑
i
=
1
m
‍
λ
1
i
(
y
i
+
e
i
-
a
¯
x
i
-
c
¯
|
x
i
|
)
-
∑
i
=
1
m
‍
λ
2
i
(
a
¯
x
i
-
c
¯
|
x
i
|
-
y
i
+
e
i
)
-
∑
i
=
1
m
‍
λ
3
i
(
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
-
y
i
-
e
i
)
-
∑
i
=
1
m
‍
λ
4
i
(
y
i
-
e
i
-
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
)
,
where L is Lagrangian and λ
1
i, λ
2
i, λ
3
i, and λ
4
i are Lagrange multipliers. The idea to construct a Lagrange function from the objective function and the corresponding constraints is to introduce a dual set of variables. It can be shown that the Lagrangian function has a saddle point with respect to the primal and dual variables in the solution [29].The Karush-Kuhn-Tucker (KKT) conditions that the partial derivatives ofL with respect to the primal variables (
a
¯
,
c
¯
,
d
¯
) for optimality
(23)
∂
L
∂
a
¯
=
0
⟹
a
¯
=
-
∑
i
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
x
i
,
∂
L
∂
c
¯
=
0
⟹
c
¯
=
-
∑
i
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
|
x
i
|
,
∂
L
∂
d
¯
=
0
⟹
d
¯
=
∑
i
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
|
x
i
|
.Substituting (23) in (22) yields the following optimization problem:
(24)
max
⁡
1
2
(
∑
i
,
j
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
max
⁡
1
2
000000
×
(
λ
1
j
-
λ
2
j
-
λ
3
j
+
λ
4
j
)
x
i
t
x
j
max
⁡
1
2
000
+
∑
i
,
j
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
max
⁡
1
2
000000000
×
(
λ
1
j
+
λ
2
j
-
λ
3
j
-
λ
4
j
)
|
x
i
|
t
|
x
j
|
max
⁡
1
2
000
-
∑
i
,
j
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
(
λ
3
j
+
λ
4
j
)
|
x
i
|
t
|
x
j
|
+
b
2
)
max
⁡
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
3
i
,
λ
4
i
≥
0
.Similarly, we can obtain the interval nonlinear regression model by mappingx
↦
ϕ
(
x
) to embed the original R
n features into a Hilbert feature space F, ϕ
:
R
n
↦
F, with a nonlinear kernel K
(
x
i
,
x
j
)
≡
ϕ
(
x
i
)
t
ϕ
(
x
j
) as discussed in Section 2.2. Then we obtain the optimization problem as (25) by replacing x
i
t
x
j and |
x
i
|
t
|
x
j
| in (24) with K
(
x
i
,
x
j
) and K
(
|
x
i
|
,
|
x
j
|
), respectively:
(25)
max
⁡
1
2
(
∑
i
,
j
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
max
⁡
1
2
000000
×
(
λ
1
j
-
λ
2
j
-
λ
3
j
+
λ
4
j
)
K
(
x
i
,
x
j
)
max
⁡
1
2
000
+
∑
i
,
j
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
max
⁡
1
2
00000000
×
(
λ
1
j
+
λ
2
j
-
λ
3
j
-
λ
4
j
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
1
2
000
-
∑
i
,
j
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
max
⁡
1
2
0000000000
×
(
λ
3
j
+
λ
4
j
)
K
(
|
x
i
|
,
|
x
j
|
)
+
b
2
∑
i
,
j
=
1
m
‍
)
max
⁡
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b
)
}
,
α
)
2
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
3
i
,
λ
4
i
≥
0
.


## 3.1. Soft Margin

In a conventional SVM, the sign function is used as the decision-making function. The separation threshold of the sign function is 0, which results in an excursion of separation margin for unbalanced data sets. The aim of the hard-margin separation margin is to find a hyperplane with the largest distance to the nearest training data. However, the limitations of the hard-margin formulation are as follows:(1)
there is no separating hyperplane for certain training data;
(2)
complete separation with zero training error will lead to suboptimal prediction error;
(3)
it is difficult to deal with the gray zone between classes.Thus, the soft margin method is proposed to modify the excursion of separation margin and to be effective in the gray zone. The soft margin is defined as(10)
f
-
(
δ
)
=
arctan
(
-
δ
·
s
+
ϑ
·
s
)
π
+
0.5
,
f
+
(
δ
)
=
arctan
(
δ
·
s
-
ϑ
·
s
)
π
+
0.5
,
where δ is the decision value. ϑ and s are offset parameter and scale parameter which need to be estimated using statistical method.With the soft margin as shown in Figure1, the predication of the class labels can be determined as follows:
(11)
y
(
x
)
=
{
-
1
,
if
(
ν
r
<
f
-
(
δ
)
,
δ
<
ϑ
)
or
(
ν
r
>
f
+
(
δ
)
,
δ
>
ϑ
)
+
1
,
if
(
ν
r
>
f
-
(
δ
)
,
δ
<
ϑ
)
or
(
ν
r
<
f
+
(
δ
)
,
δ
>
ϑ
)
,
where ν
r is a random number between 0 and 1.Figure 1
Soft margin.


## 3.2. Interval Regression with SSVM

The main idea of smooth support vector machine (SSVM) is solved by a fast Newton-Armijo algorithm [25] and has been extended to nonlinear separation surfaces by using a nonlinear kernel technology [24].Suppose thatm training data {
x
i
,
y
i
}, i
=
1,2
,
…
,
m, are given, where x
i
∈
R
n are the input patterns and y
i
∈
{
-
1,1
} are the related target values of two-class pattern classification case. Then the standard support vector machine with a linear kernel [14] is
(12)
min
⁡
w
,
b
,
ξ
1
2
∥
w
∥
2
+
C
∑
i
=
1
m
‍
ξ
i
2
s
.
t
.
y
i
(
w
t
x
i
+
b
)
≥
1
-
ξ
i
s
.
t
.
ξ
i
≥
0
,
i
=
1,2
,
…
,
m
,
where b is the location of hyperplane relative to the origin. The regularization constant C is a positive parameter to control the tradeoff between the training error and the part of maximizing the margin that is achieved by minimizing ∥
w
∥
2. ξ
i is the slack variable with weight C
/
2. ∥
w
∥ is the Euclidean norm of w which is the normal to the following hyperplanes:
(13)
w
t
x
i
+
b
=
+
1
,
for
y
i
=
+
1
,
(14)
w
t
x
i
+
b
=
-
1
,
for
y
i
=
-
1
.The first hyperplane (13) bounds the class {
+
1
} and the second hyperplane (14) bounds the class {
-
1
}. The linear separating hyperplane is
(15)
w
t
x
i
+
b
=
0
.In Lee and Mangasarian’s approach [24], b
2
/
2 is added to the objective function of (12). This is equivalent to adding a constant feature to the training data and finding a separating hyperplane through the origin. Consider
(16)
min
⁡
w
,
b
,
ξ
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
ξ
i
2
s
.
t
.
0
y
i
(
w
t
x
i
+
b
)
≥
1
-
ξ
i
s
.
t
.
0
ξ
i
≥
0
,
i
=
1,2
,
…
,
m
,
where ξ
i
=
{
1
-
y
i
(
w
t
x
i
+
b
)
}
+ for all i and the “+” function is defined as x
+
=
max
⁡
{
0
,
x
}. Then (12) can be reformulated as the following minimization problem by replacing ξ
i with {
1
-
y
i
(
w
t
x
i
+
b
)
}
+:
(17)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
{
1
-
y
i
(
w
t
x
i
+
b
)
}
+
2
.The objective function in (17) is not twice differentiable and can be solved by using a fast Newton-Armijo method [25]. Thus the “+” function in SSVM is approximated by a smooth function, p
(
x
,
α
), as follows:
(18)
p
(
x
,
α
)
=
x
+
1
α
log
⁡
(
1
+
e
-
α
x
)
,
α
>
0
,
where α
>
0 is the smooth parameter. 1
/
(
1
+
e
-
α
x
) is the integral of the sigmoid function of neural networks [28]. The p
(
x
,
α
) with a smoothing parameter α is to replace the “+” function of (17) to obtain the following smooth support vector machine (SSVM) with a linear kernel:
(19)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
.For specific data sets, an appropriate nonlinear mappingx
↦
ϕ
(
x
) can be used to embed the original R
n features into a Hilbert feature space F, ϕ
:
R
n
↦
F, with a nonlinear kernel K
(
x
i
,
x
j
)
≡
ϕ
(
x
i
)
t
ϕ
(
x
j
). Thus, (19) can be extended to the SSVM with a nonlinear kernel:
(20)
min
⁡
w
,
b
1
2
(
∥
w
∥
2
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b
)
}
,
α
)
2
,
where ∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b is the nonlinear SSVM classifier. The coefficient v
j is determined by solving an optimization problem (20) and the data points with corresponding nonzero coefficients.With the principle of SSVM, we can formulate the interval linear regression model as follows:(21)
min
⁡
a
¯
,
c
¯
,
d
¯
1
2
(
a
¯
t
a
¯
+
c
¯
t
c
¯
+
d
¯
t
d
¯
+
b
2
)
min
⁡
a
¯
,
c
¯
,
d
¯
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
s
.
t
.
a
¯
x
i
+
c
¯
|
x
i
|
≤
y
i
+
e
i
s
.
t
.
a
¯
x
i
-
c
¯
|
x
i
|
≥
y
i
-
e
i
s
.
t
.
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
≥
y
i
+
e
i
s
.
t
.
a
¯
x
i
-
c
¯
|
x
i
|
-
d
¯
|
x
i
|
≤
y
i
-
e
i
s
.
t
.
i
=
1,2
,
…
,
m
,
where a
¯, c
¯, and d
¯ are the collections of all a
i, c
i, and d
i, i
=
1,2
,
…
,
m, respectively.Given (21), the corresponding Lagrangian objective function is
(22)
L
≔
1
2
(
a
¯
t
a
¯
+
c
¯
t
c
¯
+
d
¯
t
d
¯
+
b
2
)
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
-
∑
i
=
1
m
‍
λ
1
i
(
y
i
+
e
i
-
a
¯
x
i
-
c
¯
|
x
i
|
)
-
∑
i
=
1
m
‍
λ
2
i
(
a
¯
x
i
-
c
¯
|
x
i
|
-
y
i
+
e
i
)
-
∑
i
=
1
m
‍
λ
3
i
(
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
-
y
i
-
e
i
)
-
∑
i
=
1
m
‍
λ
4
i
(
y
i
-
e
i
-
a
¯
x
i
+
c
¯
|
x
i
|
+
d
¯
|
x
i
|
)
,
where L is Lagrangian and λ
1
i, λ
2
i, λ
3
i, and λ
4
i are Lagrange multipliers. The idea to construct a Lagrange function from the objective function and the corresponding constraints is to introduce a dual set of variables. It can be shown that the Lagrangian function has a saddle point with respect to the primal and dual variables in the solution [29].The Karush-Kuhn-Tucker (KKT) conditions that the partial derivatives ofL with respect to the primal variables (
a
¯
,
c
¯
,
d
¯
) for optimality
(23)
∂
L
∂
a
¯
=
0
⟹
a
¯
=
-
∑
i
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
x
i
,
∂
L
∂
c
¯
=
0
⟹
c
¯
=
-
∑
i
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
|
x
i
|
,
∂
L
∂
d
¯
=
0
⟹
d
¯
=
∑
i
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
|
x
i
|
.Substituting (23) in (22) yields the following optimization problem:
(24)
max
⁡
1
2
(
∑
i
,
j
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
max
⁡
1
2
000000
×
(
λ
1
j
-
λ
2
j
-
λ
3
j
+
λ
4
j
)
x
i
t
x
j
max
⁡
1
2
000
+
∑
i
,
j
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
max
⁡
1
2
000000000
×
(
λ
1
j
+
λ
2
j
-
λ
3
j
-
λ
4
j
)
|
x
i
|
t
|
x
j
|
max
⁡
1
2
000
-
∑
i
,
j
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
(
λ
3
j
+
λ
4
j
)
|
x
i
|
t
|
x
j
|
+
b
2
)
max
⁡
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
w
t
x
i
+
b
)
}
,
α
)
2
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
3
i
,
λ
4
i
≥
0
.Similarly, we can obtain the interval nonlinear regression model by mappingx
↦
ϕ
(
x
) to embed the original R
n features into a Hilbert feature space F, ϕ
:
R
n
↦
F, with a nonlinear kernel K
(
x
i
,
x
j
)
≡
ϕ
(
x
i
)
t
ϕ
(
x
j
) as discussed in Section 2.2. Then we obtain the optimization problem as (25) by replacing x
i
t
x
j and |
x
i
|
t
|
x
j
| in (24) with K
(
x
i
,
x
j
) and K
(
|
x
i
|
,
|
x
j
|
), respectively:
(25)
max
⁡
1
2
(
∑
i
,
j
=
1
m
‍
(
λ
1
i
-
λ
2
i
-
λ
3
i
+
λ
4
i
)
max
⁡
1
2
000000
×
(
λ
1
j
-
λ
2
j
-
λ
3
j
+
λ
4
j
)
K
(
x
i
,
x
j
)
max
⁡
1
2
000
+
∑
i
,
j
=
1
m
‍
(
λ
1
i
+
λ
2
i
-
λ
3
i
-
λ
4
i
)
max
⁡
1
2
00000000
×
(
λ
1
j
+
λ
2
j
-
λ
3
j
-
λ
4
j
)
K
(
|
x
i
|
,
|
x
j
|
)
max
⁡
1
2
000
-
∑
i
,
j
=
1
m
‍
(
λ
3
i
+
λ
4
i
)
max
⁡
1
2
0000000000
×
(
λ
3
j
+
λ
4
j
)
K
(
|
x
i
|
,
|
x
j
|
)
+
b
2
∑
i
,
j
=
1
m
‍
)
max
⁡
+
C
2
∑
i
=
1
m
‍
p
(
{
1
-
y
i
(
∑
j
=
1
m
‍
v
j
K
(
x
i
,
x
j
)
+
b
)
}
,
α
)
2
s
.
t
.
00
λ
1
i
,
λ
2
i
,
λ
3
i
,
λ
4
i
≥
0
.


## 4. Numerical Example

To illustrate the methods developed in Section3, the following example is presented.Example. To illustrate the proposed methods dealing with big data sets, we use the data sets from Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) [26] which included the highest, lowest, and closed data and the ranges are from 01
/
02
/
2012 to 12
/
28
/
2012, from 01
/
02
/
2011 to 12
/
28
/
2012, from 01
/
02
/
2010 to 12
/
28
/
2012, and from 01
/
02
/
2009 to 12
/
28
/
2012, respectively. For these data sets, the Gaussian kernel [10] is used where σ
=
2.5 and the regularization constant C
=
300. The results are illustrated from Figure 2 to Figure 5.Figure 2
TAIEX [26] from 01
/
02
/
2012 to 12
/
28
/
2012.Figure 3
TAIEX [26] from 01
/
02
/
2011 to 12
/
28
/
2012.Figure 4
TAIEX [26] from 01
/
02
/
2010 to 12
/
28
/
2012.Figure 5
TAIEX [26] from 01
/
02
/
2009 to 12
/
28
/
2012.The comparison is shown by using the measure of fitness [15] as (26), which defines how closely the possibility output for the jth input approximates the necessity output for the jth input. Consider
(26)
φ
Y
(
x
i
)
=
1
q
∑
j
=
1
q
‍
c
0
+
∑
i
=
1
n
c
i
|
x
i
j
|
c
0
+
∑
i
=
1
n
c
i
|
x
i
j
|
+
d
0
+
∑
i
=
1
n
d
i
|
x
i
j
|
,
where q is a sample size and 0
≤
φ
Y
≤
1.Table1 presents the proposed methods with a Gaussian kernel along with the results computed by Tanaka and Lee [15], Hong and Hwang [16], and Huang [21]. We can find that the proposed methods are more efficient than other methods.Table 1
Comparison results of the measure of fitness.
Tanaka and Lee [15]
Hong and Hwang [16]
Huang [21]
Proposed methods
φ
Y (Figure 2)
0.1404
0.1395
0.1412
0.1354
φ
Y (Figure 3)
0.1573
0.1562
0.1581
0.1429
φ
Y (Figure 4)
0.1694
0.1658
0.1706
0.1583
φ
Y (Figure 5)
0.1714
0.1695
0.1723
0.1609


## 5. Conclusions

In this paper, we collaborate interval regression with SSVM to analyze big data. In addition, the soft margin method is proposed to modify the excursion of separation margin and to be effective in the gray zone. The main idea of SSVM is solved by a fast Newton-Armijo algorithm and has been extended to nonlinear separation surfaces by using a nonlinear kernel technology. The experiment results show that the proposed methods are more efficient than existing methods. In this study, we estimate the interval regression model with crisp inputs and interval output. In future works, both interval inputs-interval output and fuzzy inputs-fuzzy output will be considered.



---

*Source: 243921-2014-07-20.xml*