# A Novel Soft Ensemble Model for Financial Distress Prediction with Different Sample Sizes

Authors: Wei Xu; Hongyong Fu; Yuchen Pan
Journal: Mathematical Problems in Engineering
 (2019)

---

## Abstract

This work presents a novel soft ensemble model (ANSEM) for financial distress prediction with different sample sizes. It integrates qualitative classifiers (expert system method, ES) and quantitative classifiers (convolutional neural network, CNN) based on the uni-int decision making method of soft set theory (UI). We introduce internet searches indices as new variables for financial distress prediction. By constructing a soft set representation of each classifier and then using the optimal decision on soft sets to identify the financial status of firms, ANSEM inherits advantages of ES, CNN, and UI. Empirical experiments with the real data set of Chinese listed firms demonstrate that the proposed ANSEM has superior predicting performance for financial distress on accuracy and stability with different sample sizes. Further discussions also show that internet searches indices can offer additional information to improve predicting performance.


---

## Body

## 1. Introduction

Financial distress prediction, which has been used to identify the financial status of firms in the future, is an essential work in helping investors assess their investment risks [1]. It is a good practical tool for distinguishing firms in financial distress from the healthy ones [2]. Generally, it is believed that symptoms of financial distress can be perceived before encountering a failure or crisis [3]. Many researchers and practitioners have worked on this subject with great interest over decades [4]. However, financial distress prediction practice is still a major challenge for its complexity and rapid variations, especially under the urge of big data. There are three tasks involved in predicting financial distress, as shown in Figure 1.Figure 1
The process of financial distress prediction.First of all, we need to analyze the research object. Financial distress prediction of different objects may take different methods for the heterogeneity. This is why there are various literatures focused on some specific fields, such as American firms [5, 6], Chinese firms [7, 8], and so on [9]. In this paper, we are interested in the financial distress prediction of Chinese listed firms from the Shanghai Stock Exchange and Shenzhen Stock Exchange.Secondly, we should select optimal variables for financial distress prediction according to the object. Financial ratios have been widely used in prior literatures since Beaver [10] first adopted 6 financial ratios for bankruptcy prediction [11]. As we all know, financial ratios are historical data [12]. It is hard to reflect the finance difference of firms timely. Especially, when the operation environment changed quickly, the role of financial ratios on predicting financial distress will probably diminish [13]. Some literatures have introduced nonfinancial variables to improve the predicting performance, including country characteristics [14] and industry factors [15]. Though internet searches indices have been proved to be important variables for predicting under the era of big data [16], they have been rarely involved in financial distress prediction. Therefore, here we try to expand variables for financial distress prediction by integrating internet searches indices and other predicting variables.Finally, we focus on the predicting model for financial distress according to the object and variables. Predicting models, especially individual classifiers, have significant effect on predicting performance [17]. It motives researchers to explore predicting methods. Various statistical methods have been proposed for financial distress prediction in prior literatures, including discriminant analysis (DA) [5] and logistic regression (LR) [6] [12]. However, statistical models have disadvantages on the stringent model assumptions. After that, artificial intelligence methods have been widely applied to predict financial distress, including case-based reasoning (CBR) [18], neural networks (NN) [19], genetic algorithm (GA) [20], rough set (RS) [21], decision tree [22], and support vector machine (SVM) [23] [22]. Each individual method has advantages and disadvantages in predicting financial distress [24]. Therefore, more and more researchers have tried to employ individual methods as classifiers to construct ensemble models for financial distress prediction [25]. The main purpose is to improve predicting performance by taking full advantages of classifiers and in the same time minimizing their disadvantages. A lot of ensemble models have been proposed for financial distress prediction [25, 26]. Recently, Alaka et al. [27] investigated a systematic review of financial distress prediction models. However, there are still some deficiencies to be improved for distress prediction practice.(1)
Those models above may obtain a satisfactory performance with large sample sizes and financial ratios. However, in failure predicting practice, especially under the urge of big data, predicting financial distress with different sample sizes and other nonfinancial variables becomes increasingly frequent.
(2)
For most ensemble models, it is a key point to compute the weight of individual classifiers. Meanwhile, it is still a world widely challenge task since Bates [28] first discussed the ensemble forecasting method.
(3)
With classifiers increase, in most ensemble predicting models may exist serious overfitting issue and poor generalization issue. It is a contradiction that more classifiers will bring additional information to improve predicting performance [4].
(4)
Qualitative methods have been rarely employed to construct ensemble predicting models. Meanwhile, some literatures have found out that qualitative approaches can play a valid role in predicting financial distress [29].Therefore, to address the real practice circumstances, the aim of this paper is to improve a novel soft ensemble model (ANSEM) for financial distress prediction with different sample sizes. Both qualitative methods and quantitative methods are employed as classifiers to take full use of their advantages. We choose the expert system method (ES) as the qualitative classifier for its advantages [29]. For the same reason, convolutional neural network (CNN) is employed as the quantitative classifier [19]. Then the novel uni-int decision making method of soft set theory (UI), initiated by Çağman and Enginoğlu [30], is applied to integrate forecasts of each classifier. The UI has been proved theoretically as a superior nonparametric method for dealing with high dimensional and different sample sizes data [31]. In such a way, the ANSEM inherits the efficiency and flexibility of UI and takes advantages of ES and CNN at the same time.For performance comparison, individual ES, CNN, the ensemble models with ES and CNN based on equal weights (EMEW), convolutional neural network (EMNN), rough set theory and Dempster-Shafer evidence theory (EMRD) [32] are comparative models included in this work. To demonstrate effects of different sample sizes on predicting performance of each model, we divide all real sample data from Chinese listed firms into the training data set and the testing data set randomly for percentages (25%, 75%), (50%, 50%), and (75%, 25%).The remainder of this paper is organized as follows. Section2 briefly reviews the classical uni-int decision making method of soft set theory and mainly introduces the proposed soft ensemble model for financial distress prediction. Section 3 provides an empirical experiment with real data sets from Chinese listed firms. Section 4 presents the empirical results and makes a comparison and discussion. In Section 5 we conclude this paper and discuss further research.


## 2. The Soft Ensemble Predicting Model

### 2.1. The Uni-Int Decision Making Method of Soft Set Theory

Soft set theory, originated by Molodtsov [33], is a novel mathematic theory for uncertain information [34]. Assuming U is an initial universe of objects, E is a parameters set to objects, P
(
U
) is the power set of U, and A is a parameters subset of E (A
⊆
E). A soft set F
A can be defined by the set of ordered pairs [35], shown as (1)
F
A
=
x
,
f
A
x
:
x
∈
E
,
f
A
x
∈
P
Uwhere f
A
:
E
→
P
(
U
) such that f
A
x
=
∅ if x
∉
A. f
A is an approximate function of F
A.Based on the definition above, Çağman and Enginoğlu [30] redefined the product operation of soft sets as the binary operation to take full information of soft sets as follows. Assuming ∧
(
U
) is the set of all ∧
p
r
o
d
u
c
t
s (and products) of soft sets over U. If F
A
∧
F
B
∈
∧
(
U
), the uni-int operation denoted by u
n
i
x
i
n
t
y and u
n
i
y
i
n
t
x are defined, respectively, as (2)
u
n
i
x
i
n
t
y
:
∧
U
→
P
U
,
u
n
i
x
i
n
t
y
F
A
∧
F
B
=
⋃
x
∈
A
⋂
y
∈
B
f
A
∧
B
x
,
y
(3)
u
n
i
y
i
n
t
x
:
∧
U
→
P
U
,
u
n
i
y
i
n
t
x
F
A
∧
F
B
=
⋃
y
∈
B
⋂
x
∈
A
f
A
∧
B
x
,
y
F
A
∧
F
B is a new soft set defined by the function f
A
∧
B
:
A
×
B
→
P
U, f
A
∧
B
(
x
,
y
)
=
f
A
(
x
)
∩
f
B
(
y
). The uni-int decision set is the union of two uni-int operation sets, as (4)
u
n
i
-
i
n
t
F
A
∧
F
B
=
u
n
i
x
i
n
t
y
F
A
∧
F
B
∪
u
n
i
y
i
n
t
x
F
A
∧
F
BThe uni-int decision making method of soft set theory (UI) is an effective integrated tool to exploit information of soft sets [11]. However, researchers working in this field mainly focused on theoretical researches. It is rarely applied to practice. This paper contributes to UI by filling this gap. We take UI as a novel integrated method for financial distress prediction to achieve a better performance.


### 2.2. The Soft Ensemble Predicting Model

Assume there aren
(
n
=
1
,
…
,
N
) samples and  m
(
m
=
(
1
,
…
,
M
) predicting classifiers. Y is the original status matrix of samples. Y
m is the individual predicting results matrix of the m
t
h classifier about the n
t
h samples, Y
u is the integrated results matrix of y
n
m using UI. Y, Y
m, and Y
u are defined as (5)
Y
=
y
1
⋮
y
n
⋮
y
N
,
Y
m
=
y
11
⋯
y
1
m
⋯
y
1
M
⋮
y
n
1
⋮
⋱
y
n
m
⋱
⋮
y
n
M
⋮
y
N
1
⋯
y
N
m
⋯
y
N
M
,
Y
u
=
y
1
u
⋮
y
n
u
⋮
y
N
uwhere(6)
y
n
=
1
0
,
y
n
m
=
1
0
,
y
n
u
=
1
0
y
n
=
0 represents thenth sample is in the actually financial distress status; y
n
=
1 means thenth sample is in the actually financial normal status. y
n
m
=
0 represents themth predicting classifier predicts thenth sample will be in the financial distress status; y
n
m
=
1 means themth predicting classifier predicts thenth sample will be in the financial normal status. y
n
u is the integrated result using UI for thenth sample. y
n
u
=
0 represents thenth sample will be in the financial distress status; y
n
u
=
1 means thenth sample will be in the financial normal status. A concise illustration of the novel soft ensemble model (ANSEM) for financial distress prediction can be shown as in Figure 2. Obviously, three key points are involved in constructing the novel soft ensemble model: the individual classifier, the integrated method, and the algorithm.Figure 2
The principle of ANSEM. Three key points for the novel soft ensemble model: the individual classifier, the integrated method, and the algorithm.


#### 2.2.1. Individual Predicting Classifiers

Individual predicting classifiers play a significant role in performance of ensemble predicting models [28]. As mentioned above, many qualitative methods and quantitative methods, which can be used as classifiers, have been proposed for financial distress prediction in prior literatures [36]. While each classifier has advantages and disadvantages, according to the research object and variables, we need to select some appropriate classifiers from them. On one side, we want to employ more classifiers as components of ANSEM to take advantages of them. On the other side, it is a contradictory because the complexity of ANSEM will be a seriously problem. The computing power will decrease. According to prior literatures, two classifiers may bring a nice balance of the performance and the complexity [8]. We also want to employ both qualitative classifiers and quantitative classifiers to construct ANSEM. Therefore, expert system method (ES) and convolutional neural network (CNN) are employed as individual classifiers for their advantages. ES is a good qualitative classifier for financial distress prediction [29]; CNN is an excellent quantitative classifier too [37].


#### 2.2.2. Integrated Method

The proposed ANSEM for financial distress prediction can be redefined as(7)
y
n
u
=
g
y
n
mwhere g
(
·
) is the integrated method. To overcome disadvantages of ensemble models on measuring weight coefficients, here we take the novel uni-int decision making method of soft set theory (UI) as the integrated method. Soft set theory is a parameterized family of subsets of the universe U [33]. The increase of E will let UI perform better. That means that contradictions of the complexity and the performance can be well solved. It also does not need measure the weight coefficient of each individual classifier. Therefore, we use UI as the integrated method.However, the UI is an operation rule of soft sets. First of all, we need to represent the process of each predicting classifier in the style of soft set theory. Each classifier translates into a typical soft set. Then we are able to run the uni-int operation and find the uni-int decision set. The integration process can be briefly showed as in Figure3. Details are clearly illustrated as follows.Figure 3
The integration process of ANSEM. There are two key points in the integration process: soft set representation and uni-int operation.Step 1.
We can take the samples set as the nonempty initial universeU of soft sets. The variable set can be treated as the parameters set E to objects of U.Step 2.
The approximate mapping functionf of soft sets can be each classifier for financial distress prediction. Then we get m (the number of individual classifiers) different soft sets, shown as (8)
F
m
=
x
,
f
m
x
:
x
∈
E
m
,
f
m
x
∈
P
Uwhere U is the set of samples, E
m is the set of variables for the m
t
h classifier, x is a variable of E, and f
m is each classifier for financial distress prediction. Here, the parameter set E
m
(
m
=
1,2
,
…
,
M
) may include the same or different variables.Step 3.
Based on soft setsF
m, we can run the uni-int operation and find the uni-int decision set using (2)–(4). The uni-int decision set is the finally predicting results.


#### 2.2.3. The Algorithm of ANSEM

The algorithm of ANSEM for financial distress prediction can be briefly shown as in Figure4. Each step is clearly illustrated as follows.Figure 4
The algorithm of ANSEM.Step 1 (data preprocessing).
The measuring unit of variables may be different. We need to normalize all collected data to decrease the difference at first. The function is showed as(9)
x
n
i
′
=
x
n
i
-
m
i
n
i
m
a
x
i
-
m
i
n
iwhere x
n
i is the i
t
h variable’s original value for the n
t
h sample, and m
i
n
i, m
a
x
i are the minimal value and maximal value of the i
t
h variables for all samples, respectively.Step 2 (classifiers training).
For expert system method, unlike previous literatures, we use real financial institutions as experts, especially the security firms in China. It is a common feature for those institutions to issue research reports periodically to give investment recommendations for listed firms’ stock, such as “buy”, “keep” or “sell”. “Buy” means the firm may have a good financial development in future and bring investors an impressive return, in the institution’s opinion. “Keep” presents the opinion that the firm may not suffer financial distress in future and bring investors a normal return. “Sell” means that the firm may suffer financial distress in future and bring investors a loss. Recommendations are made based on professional analysis. It is useful to improve predicting performance.To increase the reliability, we employj
(
j
=
1
,
…
,
J
) financial institutions as experts. J is an odd number and is bigger than 1. Predicting results can be shown as (10)
Y
e
=
y
1
J
⋮
y
n
J
⋮
y
N
J
=
y
11
⋯
y
1
j
⋯
y
1
J
⋮
y
n
1
⋮
⋮
⋱
y
n
j
⋱
⋮
⋮
y
n
J
⋮
y
N
1
⋯
y
N
j
⋯
y
N
Jwhere y
n
j is the latest recommendation of the j
t
h expert on the n
t
h sample. y
n
J is the final predicting result about the n
t
h sample. Y
e is the predicting results matrix. y
n
j equals to “buy”, “keep” or “sell”. Here we code “buy” as 1, “keep” as 0, and “sell” as -1. In other words, y
n
J can be defined as (11)
y
n
J
=
1
,
∑
j
=
1
J
y
n
j
≥
0
0
,
∑
j
=
1
J
y
n
j
<
0That means that if more institutions recommend buying than selling, the expert system method predicts the firm will not suffer the financial distress.For convolutional neural network, it is a key point to choose an appropriate structure [19]. Here, according to prior literatures, we employ the basic convolutional neural network as CNN algorithm. One can refer the literature of Hosaka [24] for details about CNN.Step 3 (soft set representing).
As demonstrated in Section2.2.2, we obtain two soft sets: F
e
s. and F
n
n. F
e
s is the soft set of ES. F
n
n is the soft set of CNN. The U is the same initial universe set of samples for both F
e
s and F
n
n. E
e
s and E
n
n are parameter sets for F
e
s and F
n
n, respectively. They may include the same or different variables. E
e
s, E
n
n
⊆
E. F
e
s and F
n
n can be shown as (12)
F
e
s
=
x
,
f
e
s
x
:
x
∈
E
e
s
,
f
e
s
x
∈
P
U
(13)
F
n
n
=
x
,
f
n
n
x
′
:
x
′
∈
E
n
n
,
f
n
n
x
′
∈
P
UStep 4 (uni-int operations).
Based on soft setsF
e
s and F
n
n, (2) and (3) can be represented as (14) and (15).(14)
u
n
i
x
i
n
t
x
′
:
∧
U
→
P
U
,
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
=
⋃
x
∈
E
e
s
⋂
x
′
∈
E
n
n
f
e
s
∧
n
n
x
,
x
′
(15)
u
n
i
x
′
i
n
t
x
:
∧
U
→
P
U
,
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
n
=
⋃
x
′
∈
E
n
n
⋂
x
∈
E
e
s
f
e
s
∧
n
n
x
,
x
′The uni-int decision set can be represented as (16)
u
n
i
-
i
n
t
F
e
s
∧
F
n
n
=
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
∪
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
nStep 5 (final results).
Applying ANSEM to real data, we can obtain final predicting results.


## 2.1. The Uni-Int Decision Making Method of Soft Set Theory

Soft set theory, originated by Molodtsov [33], is a novel mathematic theory for uncertain information [34]. Assuming U is an initial universe of objects, E is a parameters set to objects, P
(
U
) is the power set of U, and A is a parameters subset of E (A
⊆
E). A soft set F
A can be defined by the set of ordered pairs [35], shown as (1)
F
A
=
x
,
f
A
x
:
x
∈
E
,
f
A
x
∈
P
Uwhere f
A
:
E
→
P
(
U
) such that f
A
x
=
∅ if x
∉
A. f
A is an approximate function of F
A.Based on the definition above, Çağman and Enginoğlu [30] redefined the product operation of soft sets as the binary operation to take full information of soft sets as follows. Assuming ∧
(
U
) is the set of all ∧
p
r
o
d
u
c
t
s (and products) of soft sets over U. If F
A
∧
F
B
∈
∧
(
U
), the uni-int operation denoted by u
n
i
x
i
n
t
y and u
n
i
y
i
n
t
x are defined, respectively, as (2)
u
n
i
x
i
n
t
y
:
∧
U
→
P
U
,
u
n
i
x
i
n
t
y
F
A
∧
F
B
=
⋃
x
∈
A
⋂
y
∈
B
f
A
∧
B
x
,
y
(3)
u
n
i
y
i
n
t
x
:
∧
U
→
P
U
,
u
n
i
y
i
n
t
x
F
A
∧
F
B
=
⋃
y
∈
B
⋂
x
∈
A
f
A
∧
B
x
,
y
F
A
∧
F
B is a new soft set defined by the function f
A
∧
B
:
A
×
B
→
P
U, f
A
∧
B
(
x
,
y
)
=
f
A
(
x
)
∩
f
B
(
y
). The uni-int decision set is the union of two uni-int operation sets, as (4)
u
n
i
-
i
n
t
F
A
∧
F
B
=
u
n
i
x
i
n
t
y
F
A
∧
F
B
∪
u
n
i
y
i
n
t
x
F
A
∧
F
BThe uni-int decision making method of soft set theory (UI) is an effective integrated tool to exploit information of soft sets [11]. However, researchers working in this field mainly focused on theoretical researches. It is rarely applied to practice. This paper contributes to UI by filling this gap. We take UI as a novel integrated method for financial distress prediction to achieve a better performance.


## 2.2. The Soft Ensemble Predicting Model

Assume there aren
(
n
=
1
,
…
,
N
) samples and  m
(
m
=
(
1
,
…
,
M
) predicting classifiers. Y is the original status matrix of samples. Y
m is the individual predicting results matrix of the m
t
h classifier about the n
t
h samples, Y
u is the integrated results matrix of y
n
m using UI. Y, Y
m, and Y
u are defined as (5)
Y
=
y
1
⋮
y
n
⋮
y
N
,
Y
m
=
y
11
⋯
y
1
m
⋯
y
1
M
⋮
y
n
1
⋮
⋱
y
n
m
⋱
⋮
y
n
M
⋮
y
N
1
⋯
y
N
m
⋯
y
N
M
,
Y
u
=
y
1
u
⋮
y
n
u
⋮
y
N
uwhere(6)
y
n
=
1
0
,
y
n
m
=
1
0
,
y
n
u
=
1
0
y
n
=
0 represents thenth sample is in the actually financial distress status; y
n
=
1 means thenth sample is in the actually financial normal status. y
n
m
=
0 represents themth predicting classifier predicts thenth sample will be in the financial distress status; y
n
m
=
1 means themth predicting classifier predicts thenth sample will be in the financial normal status. y
n
u is the integrated result using UI for thenth sample. y
n
u
=
0 represents thenth sample will be in the financial distress status; y
n
u
=
1 means thenth sample will be in the financial normal status. A concise illustration of the novel soft ensemble model (ANSEM) for financial distress prediction can be shown as in Figure 2. Obviously, three key points are involved in constructing the novel soft ensemble model: the individual classifier, the integrated method, and the algorithm.Figure 2
The principle of ANSEM. Three key points for the novel soft ensemble model: the individual classifier, the integrated method, and the algorithm.


### 2.2.1. Individual Predicting Classifiers

Individual predicting classifiers play a significant role in performance of ensemble predicting models [28]. As mentioned above, many qualitative methods and quantitative methods, which can be used as classifiers, have been proposed for financial distress prediction in prior literatures [36]. While each classifier has advantages and disadvantages, according to the research object and variables, we need to select some appropriate classifiers from them. On one side, we want to employ more classifiers as components of ANSEM to take advantages of them. On the other side, it is a contradictory because the complexity of ANSEM will be a seriously problem. The computing power will decrease. According to prior literatures, two classifiers may bring a nice balance of the performance and the complexity [8]. We also want to employ both qualitative classifiers and quantitative classifiers to construct ANSEM. Therefore, expert system method (ES) and convolutional neural network (CNN) are employed as individual classifiers for their advantages. ES is a good qualitative classifier for financial distress prediction [29]; CNN is an excellent quantitative classifier too [37].


### 2.2.2. Integrated Method

The proposed ANSEM for financial distress prediction can be redefined as(7)
y
n
u
=
g
y
n
mwhere g
(
·
) is the integrated method. To overcome disadvantages of ensemble models on measuring weight coefficients, here we take the novel uni-int decision making method of soft set theory (UI) as the integrated method. Soft set theory is a parameterized family of subsets of the universe U [33]. The increase of E will let UI perform better. That means that contradictions of the complexity and the performance can be well solved. It also does not need measure the weight coefficient of each individual classifier. Therefore, we use UI as the integrated method.However, the UI is an operation rule of soft sets. First of all, we need to represent the process of each predicting classifier in the style of soft set theory. Each classifier translates into a typical soft set. Then we are able to run the uni-int operation and find the uni-int decision set. The integration process can be briefly showed as in Figure3. Details are clearly illustrated as follows.Figure 3
The integration process of ANSEM. There are two key points in the integration process: soft set representation and uni-int operation.Step 1.
We can take the samples set as the nonempty initial universeU of soft sets. The variable set can be treated as the parameters set E to objects of U.Step 2.
The approximate mapping functionf of soft sets can be each classifier for financial distress prediction. Then we get m (the number of individual classifiers) different soft sets, shown as (8)
F
m
=
x
,
f
m
x
:
x
∈
E
m
,
f
m
x
∈
P
Uwhere U is the set of samples, E
m is the set of variables for the m
t
h classifier, x is a variable of E, and f
m is each classifier for financial distress prediction. Here, the parameter set E
m
(
m
=
1,2
,
…
,
M
) may include the same or different variables.Step 3.
Based on soft setsF
m, we can run the uni-int operation and find the uni-int decision set using (2)–(4). The uni-int decision set is the finally predicting results.


### 2.2.3. The Algorithm of ANSEM

The algorithm of ANSEM for financial distress prediction can be briefly shown as in Figure4. Each step is clearly illustrated as follows.Figure 4
The algorithm of ANSEM.Step 1 (data preprocessing).
The measuring unit of variables may be different. We need to normalize all collected data to decrease the difference at first. The function is showed as(9)
x
n
i
′
=
x
n
i
-
m
i
n
i
m
a
x
i
-
m
i
n
iwhere x
n
i is the i
t
h variable’s original value for the n
t
h sample, and m
i
n
i, m
a
x
i are the minimal value and maximal value of the i
t
h variables for all samples, respectively.Step 2 (classifiers training).
For expert system method, unlike previous literatures, we use real financial institutions as experts, especially the security firms in China. It is a common feature for those institutions to issue research reports periodically to give investment recommendations for listed firms’ stock, such as “buy”, “keep” or “sell”. “Buy” means the firm may have a good financial development in future and bring investors an impressive return, in the institution’s opinion. “Keep” presents the opinion that the firm may not suffer financial distress in future and bring investors a normal return. “Sell” means that the firm may suffer financial distress in future and bring investors a loss. Recommendations are made based on professional analysis. It is useful to improve predicting performance.To increase the reliability, we employj
(
j
=
1
,
…
,
J
) financial institutions as experts. J is an odd number and is bigger than 1. Predicting results can be shown as (10)
Y
e
=
y
1
J
⋮
y
n
J
⋮
y
N
J
=
y
11
⋯
y
1
j
⋯
y
1
J
⋮
y
n
1
⋮
⋮
⋱
y
n
j
⋱
⋮
⋮
y
n
J
⋮
y
N
1
⋯
y
N
j
⋯
y
N
Jwhere y
n
j is the latest recommendation of the j
t
h expert on the n
t
h sample. y
n
J is the final predicting result about the n
t
h sample. Y
e is the predicting results matrix. y
n
j equals to “buy”, “keep” or “sell”. Here we code “buy” as 1, “keep” as 0, and “sell” as -1. In other words, y
n
J can be defined as (11)
y
n
J
=
1
,
∑
j
=
1
J
y
n
j
≥
0
0
,
∑
j
=
1
J
y
n
j
<
0That means that if more institutions recommend buying than selling, the expert system method predicts the firm will not suffer the financial distress.For convolutional neural network, it is a key point to choose an appropriate structure [19]. Here, according to prior literatures, we employ the basic convolutional neural network as CNN algorithm. One can refer the literature of Hosaka [24] for details about CNN.Step 3 (soft set representing).
As demonstrated in Section2.2.2, we obtain two soft sets: F
e
s. and F
n
n. F
e
s is the soft set of ES. F
n
n is the soft set of CNN. The U is the same initial universe set of samples for both F
e
s and F
n
n. E
e
s and E
n
n are parameter sets for F
e
s and F
n
n, respectively. They may include the same or different variables. E
e
s, E
n
n
⊆
E. F
e
s and F
n
n can be shown as (12)
F
e
s
=
x
,
f
e
s
x
:
x
∈
E
e
s
,
f
e
s
x
∈
P
U
(13)
F
n
n
=
x
,
f
n
n
x
′
:
x
′
∈
E
n
n
,
f
n
n
x
′
∈
P
UStep 4 (uni-int operations).
Based on soft setsF
e
s and F
n
n, (2) and (3) can be represented as (14) and (15).(14)
u
n
i
x
i
n
t
x
′
:
∧
U
→
P
U
,
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
=
⋃
x
∈
E
e
s
⋂
x
′
∈
E
n
n
f
e
s
∧
n
n
x
,
x
′
(15)
u
n
i
x
′
i
n
t
x
:
∧
U
→
P
U
,
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
n
=
⋃
x
′
∈
E
n
n
⋂
x
∈
E
e
s
f
e
s
∧
n
n
x
,
x
′The uni-int decision set can be represented as (16)
u
n
i
-
i
n
t
F
e
s
∧
F
n
n
=
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
∪
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
nStep 5 (final results).
Applying ANSEM to real data, we can obtain final predicting results.


## 2.2.1. Individual Predicting Classifiers

Individual predicting classifiers play a significant role in performance of ensemble predicting models [28]. As mentioned above, many qualitative methods and quantitative methods, which can be used as classifiers, have been proposed for financial distress prediction in prior literatures [36]. While each classifier has advantages and disadvantages, according to the research object and variables, we need to select some appropriate classifiers from them. On one side, we want to employ more classifiers as components of ANSEM to take advantages of them. On the other side, it is a contradictory because the complexity of ANSEM will be a seriously problem. The computing power will decrease. According to prior literatures, two classifiers may bring a nice balance of the performance and the complexity [8]. We also want to employ both qualitative classifiers and quantitative classifiers to construct ANSEM. Therefore, expert system method (ES) and convolutional neural network (CNN) are employed as individual classifiers for their advantages. ES is a good qualitative classifier for financial distress prediction [29]; CNN is an excellent quantitative classifier too [37].


## 2.2.2. Integrated Method

The proposed ANSEM for financial distress prediction can be redefined as(7)
y
n
u
=
g
y
n
mwhere g
(
·
) is the integrated method. To overcome disadvantages of ensemble models on measuring weight coefficients, here we take the novel uni-int decision making method of soft set theory (UI) as the integrated method. Soft set theory is a parameterized family of subsets of the universe U [33]. The increase of E will let UI perform better. That means that contradictions of the complexity and the performance can be well solved. It also does not need measure the weight coefficient of each individual classifier. Therefore, we use UI as the integrated method.However, the UI is an operation rule of soft sets. First of all, we need to represent the process of each predicting classifier in the style of soft set theory. Each classifier translates into a typical soft set. Then we are able to run the uni-int operation and find the uni-int decision set. The integration process can be briefly showed as in Figure3. Details are clearly illustrated as follows.Figure 3
The integration process of ANSEM. There are two key points in the integration process: soft set representation and uni-int operation.Step 1.
We can take the samples set as the nonempty initial universeU of soft sets. The variable set can be treated as the parameters set E to objects of U.Step 2.
The approximate mapping functionf of soft sets can be each classifier for financial distress prediction. Then we get m (the number of individual classifiers) different soft sets, shown as (8)
F
m
=
x
,
f
m
x
:
x
∈
E
m
,
f
m
x
∈
P
Uwhere U is the set of samples, E
m is the set of variables for the m
t
h classifier, x is a variable of E, and f
m is each classifier for financial distress prediction. Here, the parameter set E
m
(
m
=
1,2
,
…
,
M
) may include the same or different variables.Step 3.
Based on soft setsF
m, we can run the uni-int operation and find the uni-int decision set using (2)–(4). The uni-int decision set is the finally predicting results.


## 2.2.3. The Algorithm of ANSEM

The algorithm of ANSEM for financial distress prediction can be briefly shown as in Figure4. Each step is clearly illustrated as follows.Figure 4
The algorithm of ANSEM.Step 1 (data preprocessing).
The measuring unit of variables may be different. We need to normalize all collected data to decrease the difference at first. The function is showed as(9)
x
n
i
′
=
x
n
i
-
m
i
n
i
m
a
x
i
-
m
i
n
iwhere x
n
i is the i
t
h variable’s original value for the n
t
h sample, and m
i
n
i, m
a
x
i are the minimal value and maximal value of the i
t
h variables for all samples, respectively.Step 2 (classifiers training).
For expert system method, unlike previous literatures, we use real financial institutions as experts, especially the security firms in China. It is a common feature for those institutions to issue research reports periodically to give investment recommendations for listed firms’ stock, such as “buy”, “keep” or “sell”. “Buy” means the firm may have a good financial development in future and bring investors an impressive return, in the institution’s opinion. “Keep” presents the opinion that the firm may not suffer financial distress in future and bring investors a normal return. “Sell” means that the firm may suffer financial distress in future and bring investors a loss. Recommendations are made based on professional analysis. It is useful to improve predicting performance.To increase the reliability, we employj
(
j
=
1
,
…
,
J
) financial institutions as experts. J is an odd number and is bigger than 1. Predicting results can be shown as (10)
Y
e
=
y
1
J
⋮
y
n
J
⋮
y
N
J
=
y
11
⋯
y
1
j
⋯
y
1
J
⋮
y
n
1
⋮
⋮
⋱
y
n
j
⋱
⋮
⋮
y
n
J
⋮
y
N
1
⋯
y
N
j
⋯
y
N
Jwhere y
n
j is the latest recommendation of the j
t
h expert on the n
t
h sample. y
n
J is the final predicting result about the n
t
h sample. Y
e is the predicting results matrix. y
n
j equals to “buy”, “keep” or “sell”. Here we code “buy” as 1, “keep” as 0, and “sell” as -1. In other words, y
n
J can be defined as (11)
y
n
J
=
1
,
∑
j
=
1
J
y
n
j
≥
0
0
,
∑
j
=
1
J
y
n
j
<
0That means that if more institutions recommend buying than selling, the expert system method predicts the firm will not suffer the financial distress.For convolutional neural network, it is a key point to choose an appropriate structure [19]. Here, according to prior literatures, we employ the basic convolutional neural network as CNN algorithm. One can refer the literature of Hosaka [24] for details about CNN.Step 3 (soft set representing).
As demonstrated in Section2.2.2, we obtain two soft sets: F
e
s. and F
n
n. F
e
s is the soft set of ES. F
n
n is the soft set of CNN. The U is the same initial universe set of samples for both F
e
s and F
n
n. E
e
s and E
n
n are parameter sets for F
e
s and F
n
n, respectively. They may include the same or different variables. E
e
s, E
n
n
⊆
E. F
e
s and F
n
n can be shown as (12)
F
e
s
=
x
,
f
e
s
x
:
x
∈
E
e
s
,
f
e
s
x
∈
P
U
(13)
F
n
n
=
x
,
f
n
n
x
′
:
x
′
∈
E
n
n
,
f
n
n
x
′
∈
P
UStep 4 (uni-int operations).
Based on soft setsF
e
s and F
n
n, (2) and (3) can be represented as (14) and (15).(14)
u
n
i
x
i
n
t
x
′
:
∧
U
→
P
U
,
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
=
⋃
x
∈
E
e
s
⋂
x
′
∈
E
n
n
f
e
s
∧
n
n
x
,
x
′
(15)
u
n
i
x
′
i
n
t
x
:
∧
U
→
P
U
,
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
n
=
⋃
x
′
∈
E
n
n
⋂
x
∈
E
e
s
f
e
s
∧
n
n
x
,
x
′The uni-int decision set can be represented as (16)
u
n
i
-
i
n
t
F
e
s
∧
F
n
n
=
u
n
i
x
i
n
t
x
′
F
e
s
∧
F
n
n
∪
u
n
i
x
′
i
n
t
x
F
e
s
∧
F
n
nStep 5 (final results).
Applying ANSEM to real data, we can obtain final predicting results.


## 3. Empirical Experiment

### 3.1. Samples and Data Sets

In China, listed firms are divided into two categories in practice. One is the Specially Treated (ST) group. The other is the Not Specially Treated (NST) group. The benchmark is either the negative net profit in recent two years or published financial reports with serious misstatements. In this paper, we take ST listed firms as financial distress samples and NST listed firms as financial normal samples. We randomly selected 100 ST listed firms and 100 NST listed firms in the seven-year period 2011-2017. It means that there are 50 training samples and 150 testing samples for the percentage (25%, 75%). It is the same mean for percentages (50%, 50%) and (75%, 25%). Obviously, we can observe the change of predicting performance with different sample sizes. Data of listed firms can be collected from the CSMAR solution.For expert system method, the data set is collected from Chinese financial institutions in the seven-year period 2011-2017. There are more than 150 professional institutions that have recommended or are recommending for investments. According to the sustainability in recommendation, we randomly employed 3(
j
=
3
) institutions as experts with similar backgrounds and abilities. They are “HAITONG Securities”, “CITIC Securities”, and “SHENWAN&HONGYUAN Securities”. We can download their research reports from the “WIND” database. Furthermore, each expert may publish two or more reports in a year. We choose the latest report as the outcome of experts.


### 3.2. Variables Selection

A lot of variables have been proposed for financial distress prediction. The widely popular variables for financial distress prediction are listed in Table1 [6]. To get the optimal traditional variables, the stepwise logistic regression is applied to the training data set of the year (
t
-
1
) to select variables from Table 1. We obtain 6 variables: x
4, x
6, x
7, x
11, x
14, and x
15.Table 1
The most popular variables for financial distress prediction.
No.
Variables
No.
Variables
x
1
Current ratio
x
2
Cash flow / total assets
x
3
Cash flow / total debt
x
4
Cash flow / sales
x
5
Debt ratio
x
6
Market value equity / total debt
x
7
Working capital / total asset
x
8
Working capital / sales
x
9
Quick asset / total asset
x
10
Quick asset / sales
x
11
Current debt / sales
x
12
Current assets / total asset
x
13
No-credit interval
x
14
Net income / total asset
x
15
Retained earnings / total asset
x
16
Sales / total asset
x
17
log(total assets / GNP price-level index)
x
18
Earnings before interest and taxes / total assetBesides, we are trying to expand predicting variables in following two ways. First of all, we introduce internet searches indices of samples for financial distress prediction. Since January 1, 2011, we can collect the daily, weekly, monthly, and annual searches data for sample’s name from “Baidu” website. Many literatures have demonstrated the significance of internet searches data on forecasting. We mark the annual searches data for sample’s name asx
19. Secondly, we use the recommendations as the outcomes of the expert system method. Thus, we employ the variable “recommendation” for expert system method and mark it as x
20. All variables selected for ANSEM are listed in Table 2.Table 2
All variables selected for ANSEM.
No.
Variables
No.
Variables
x
7
Working capital/ total asset
x
4
Cash flow/ sales
x
11
Current debt/ sales
x
6
Market value equity/ total debt
x
15
Retained earnings/ total asset
x
14
Net income/ total asset
x
19
The annual searches data for sample’s name
x
20
recommendation


### 3.3. Experiment Design

Li and Sun [38] have found out that financial distress prediction of the year t using data sets of the year (
t
-
2
) or (
t
-
3
) is more difficult than using data sets of the year (
t
-
1
). Here, we tackle the challenge. The framework of the empirical experiment is briefly shown in Figure 5. Details are clearly illustrated as follows.Figure 5
The framework of the empirical experiment. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Step 1 (collect samples and data sets).
We randomly divide samples into three data sets by ten times’ split method. One is the training data set; the other one is the testing data set and the last one is the hold-out data set.Step 2 (select variables).
We apply the stepwise logistic regression to the training data set of the year(
t
-
1
) to select optimal traditional variables from Table 1. Then we integrate optimal traditional variables, recommendation, and the annual number of searches for sample’s name into the variable set of ANSEM.Step 3 (obtain the predicting results).
We employ ES, NN, EMEW, EMNN, EMRD, and ANSEM to the testing data sets of the year(
t
-
2
) or (
t
-
3
) to obtain the predicting results with the 5-fold cross validation method.Step 4.
Compare and discuss the predicting performance.


## 3.1. Samples and Data Sets

In China, listed firms are divided into two categories in practice. One is the Specially Treated (ST) group. The other is the Not Specially Treated (NST) group. The benchmark is either the negative net profit in recent two years or published financial reports with serious misstatements. In this paper, we take ST listed firms as financial distress samples and NST listed firms as financial normal samples. We randomly selected 100 ST listed firms and 100 NST listed firms in the seven-year period 2011-2017. It means that there are 50 training samples and 150 testing samples for the percentage (25%, 75%). It is the same mean for percentages (50%, 50%) and (75%, 25%). Obviously, we can observe the change of predicting performance with different sample sizes. Data of listed firms can be collected from the CSMAR solution.For expert system method, the data set is collected from Chinese financial institutions in the seven-year period 2011-2017. There are more than 150 professional institutions that have recommended or are recommending for investments. According to the sustainability in recommendation, we randomly employed 3(
j
=
3
) institutions as experts with similar backgrounds and abilities. They are “HAITONG Securities”, “CITIC Securities”, and “SHENWAN&HONGYUAN Securities”. We can download their research reports from the “WIND” database. Furthermore, each expert may publish two or more reports in a year. We choose the latest report as the outcome of experts.


## 3.2. Variables Selection

A lot of variables have been proposed for financial distress prediction. The widely popular variables for financial distress prediction are listed in Table1 [6]. To get the optimal traditional variables, the stepwise logistic regression is applied to the training data set of the year (
t
-
1
) to select variables from Table 1. We obtain 6 variables: x
4, x
6, x
7, x
11, x
14, and x
15.Table 1
The most popular variables for financial distress prediction.
No.
Variables
No.
Variables
x
1
Current ratio
x
2
Cash flow / total assets
x
3
Cash flow / total debt
x
4
Cash flow / sales
x
5
Debt ratio
x
6
Market value equity / total debt
x
7
Working capital / total asset
x
8
Working capital / sales
x
9
Quick asset / total asset
x
10
Quick asset / sales
x
11
Current debt / sales
x
12
Current assets / total asset
x
13
No-credit interval
x
14
Net income / total asset
x
15
Retained earnings / total asset
x
16
Sales / total asset
x
17
log(total assets / GNP price-level index)
x
18
Earnings before interest and taxes / total assetBesides, we are trying to expand predicting variables in following two ways. First of all, we introduce internet searches indices of samples for financial distress prediction. Since January 1, 2011, we can collect the daily, weekly, monthly, and annual searches data for sample’s name from “Baidu” website. Many literatures have demonstrated the significance of internet searches data on forecasting. We mark the annual searches data for sample’s name asx
19. Secondly, we use the recommendations as the outcomes of the expert system method. Thus, we employ the variable “recommendation” for expert system method and mark it as x
20. All variables selected for ANSEM are listed in Table 2.Table 2
All variables selected for ANSEM.
No.
Variables
No.
Variables
x
7
Working capital/ total asset
x
4
Cash flow/ sales
x
11
Current debt/ sales
x
6
Market value equity/ total debt
x
15
Retained earnings/ total asset
x
14
Net income/ total asset
x
19
The annual searches data for sample’s name
x
20
recommendation


## 3.3. Experiment Design

Li and Sun [38] have found out that financial distress prediction of the year t using data sets of the year (
t
-
2
) or (
t
-
3
) is more difficult than using data sets of the year (
t
-
1
). Here, we tackle the challenge. The framework of the empirical experiment is briefly shown in Figure 5. Details are clearly illustrated as follows.Figure 5
The framework of the empirical experiment. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Step 1 (collect samples and data sets).
We randomly divide samples into three data sets by ten times’ split method. One is the training data set; the other one is the testing data set and the last one is the hold-out data set.Step 2 (select variables).
We apply the stepwise logistic regression to the training data set of the year(
t
-
1
) to select optimal traditional variables from Table 1. Then we integrate optimal traditional variables, recommendation, and the annual number of searches for sample’s name into the variable set of ANSEM.Step 3 (obtain the predicting results).
We employ ES, NN, EMEW, EMNN, EMRD, and ANSEM to the testing data sets of the year(
t
-
2
) or (
t
-
3
) to obtain the predicting results with the 5-fold cross validation method.Step 4.
Compare and discuss the predicting performance.


## 4. Results and Discussion

### 4.1. Empirical Results

In this paper, the 5-fold cross validation method is employed to perform the empirical experiment [39]. MATLAB (2016) is used to obtain optimal variables and predicting results of each model with different sample sizes.Financial distress prediction is a typical two-class classification problem. The predicting outputs are usually marked as either positive (P) or negative (N). More specifically, the outputs of financial distress prediction include four different results. One is the true positive (TP). TP means that the predicting output is positive and the real status is positive too. One is the false positive (FP). FP means that the predicting output is positive, but the real status is negative. The other one is the true negative (TN). TN means that the predicting output is negative and the real status is also negative. The last one is false negative (FN). FN means that the predicting output is negative, but the real status is positive. Thus, we can use an index called accuracy of correct classification (ACC) to measure the predicting accuracy of each method. The definition of ACC is shown as(17)
A
C
C
=
T
P
+
T
N
P
+
N


#### 4.1.1. Empirical Results with Different Sample Sizes

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using all variables and data sets of the year(
t
-
2
) and (
t
-
3
) for percentages (25%, 75%), (50%, 50%), and (75%, 25%) are respectively listed in Tables 3–8.Table 3
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.733
0.700
0.667
0.867
0.967
2
0.933
0.967
0.967
0.933
0.867
0.867
3
0.867
0.933
0.933
0.800
0.967
0.867
4
0.767
0.733
0.867
0.967
0.733
0.800
5
0.900
0.933
0.800
0.800
0.900
0.933
Mean
0.853
0.860
0.853
0.833
0.867
0.887
Variance
0.005
0.014
0.011
0.014
0.007
0.004
Coefficient of variation
0.006
0.016
0.013
0.017
0.008
0.005Table 4
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.950
0.950
0.950
2
0.850
0.750
0.800
0.900
0.700
0.800
3
0.900
0.950
0.800
0.700
0.950
0.950
4
0.800
0.650
0.650
0.800
0.800
0.750
5
0.750
0.900
0.950
0.600
0.750
0.900
Mean
0.840
0.830
0.820
0.790
0.830
0.870
Variance
0.004
0.016
0.013
0.021
0.013
0.008
Coefficient of variation
0.005
0.019
0.016
0.026
0.016
0.009Table 5
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
1.000
0.900
0.900
0.900
0.900
1.000
2
0.800
0.700
0.700
0.600
0.800
0.900
3
0.900
0.900
0.900
0.800
0.900
0.900
4
0.800
0.900
0.900
0.800
0.600
0.900
5
0.800
0.600
0.700
0.600
0.700
0.700
Mean
0.860
0.800
0.820
0.740
0.780
0.880
Variance
0.008
0.020
0.012
0.018
0.017
0.012
Coefficient of variation
0.009
0.025
0.015
0.024
0.022
0.014Table 6
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.800
0.933
0.833
2
0.700
0.933
0.833
0.700
0.800
0.900
3
0.833
0.733
0.867
0.900
0.833
0.800
4
0.667
0.667
0.633
0.567
0.633
0.833
5
0.667
0.667
0.567
0.600
0.733
0.633
Mean
0.733
0.780
0.740
0.713
0.787
0.800
Variance
0.006
0.016
0.017
0.019
0.013
0.010
Coefficient of variation
0.008
0.021
0.024
0.027
0.016
0.013Table 7
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.900
0.700
0.900
2
0.750
0.850
0.900
0.700
0.900
0.700
3
0.800
0.650
0.750
0.650
0.850
0.850
4
0.700
0.800
0.650
0.900
0.800
0.800
5
0.650
0.600
0.600
0.600
0.600
0.650
Mean
0.740
0.760
0.740
0.750
0.770
0.780
Variance
0.004
0.017
0.014
0.020
0.015
0.011
Coefficient of variation
0.006
0.022
0.019
0.027
0.019
0.014Table 8
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.600
0.900
0.900
2
0.700
0.600
0.600
0.800
0.800
0.900
3
0.700
0.800
0.800
0.500
0.500
0.600
4
0.700
0.500
0.600
0.400
0.600
0.800
5
0.800
0.600
0.600
0.900
0.700
0.800
Mean
0.760
0.680
0.700
0.640
0.700
0.800
Variance
0.008
0.027
0.020
0.043
0.025
0.015
Coefficient of variation
0.011
0.040
0.029
0.067
0.036
0.019


#### 4.1.2. Empirical Results with Different Variables

To investigate whether the internet searches index can improve financial distress prediction performance under the era of big data, we run the empirical experiment again with selected variables except the internet searches index. And we employ data sets for the percentage (50%, 50%) as experiment data sets. Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM with all variables except the internet searches index and data sets of the year(
t
-
2
) and (
t
-
3
) are listed in Tables 9 and 10.Table 9
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.850
0.850
0.800
0.950
0.700
2
0.850
0.950
0.900
0.750
0.700
0.950
3
0.900
0.850
0.950
0.850
0.950
0.950
4
0.800
0.550
0.600
0.500
0.650
0.750
5
0.750
0.800
0.750
0.950
0.800
0.850
Mean
0.840
0.800
0.810
0.770
0.810
0.840
Variance
0.004
0.023
0.019
0.028
0.019
0.013
Coefficient of variation
0.005
0.028
0.024
0.037
0.024
0.015Table 10
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.700
0.750
0.850
0.750
0.750
2
0.750
0.900
0.900
0.950
0.900
0.850
3
0.800
0.600
0.650
0.600
0.600
0.650
4
0.700
0.850
0.800
0.550
0.850
0.900
5
0.650
0.500
0.500
0.550
0.550
0.600
Mean
0.740
0.710
0.720
0.700
0.730
0.750
Variance
0.004
0.028
0.023
0.035
0.023
0.016
Coefficient of variation
0.006
0.039
0.032
0.050
0.032
0.022


### 4.2. Comparison and Discussion

In this paper, we employ three statistical indices of ACC from the 5-fold cross validation procedure to evaluate the performance of each model. The three statistical indices are mean, variance, and coefficient of variation. The mean is critical on evaluating the predicting accuracy of each model. The variance and coefficient of variation are critical on evaluating the predicting stability of each model.


#### 4.2.1. Results Comparison and Discussion with Different Sample Sizes

Based on Tables3–8, we can demonstrate the mean clearly in Figure 6. Meanwhile, we can illustrate the variance and coefficient of variance in Figures 7 and 8.Figure 6
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 7
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 8
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables3–8 and Figure 6, we can find out that the proposed novel soft ensemble model (ANSEM) for financial distress prediction has the highest mean accuracy no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is, the predicting accuracy of ANSEM does not have a lot of changes. However, other predicting models are different, especially the NN and EWNN. With changes in sample sizes, means of NN and EWNN are significantly different. Because NN is not good at dealing with small sample sizes [29], the mean of ES also does not change a lot. Because ESs are real practitioners, they have to pay attention on the risk.Moreover, all predicting methods except ES have a worse predicting accuracy using data sets of the year(
t
-
3
) than those using data sets of the year (
t
-
2
). Predicting financial distress on a long term is much more complicated than a short term prediction.Similar as the conclusion of predicting accuracy, from Tables3–8 and Figures 7-8, we can find out that ANSEM has the best predicting stability no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is employed, ANSEM has the lowest variance and coefficient of variation of ACC. Besides, ES also has an excellent performance on the predicting stability. This is because ESs are real practitioners. They have to pay attention to the risk. Other predicting models are different; especially the NN and EWNN have the worst predicting stability.Without a surprise, predicting stability of models using data sets of the year(
t
-
2
) outperforms that using data sets of the year (
t
-
3
).


#### 4.2.2. Results Comparison and Discussion with Different Variables

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using different variables and data sets of the year(
t
-
2
) and (
t
-
3
) for the percentage (50%, 50%) are listed in Tables 4, 7, 9, and 10. From Tables 4, 7, 9, and 10, we can compare the mean of ACC clearly in Figure 9. Meanwhile, we can compare the variance and coefficient of variance in Figures 10 and 11.Figure 9
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 10
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 11
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables4, 7, 9, and 10 and Figures 9–11, we can find out that the predicting model with all variables uniformly performs better no matter which year of data sets is used for predicting. It has a higher predicting accuracy comparing to those with all variables except the internet searches index. Also, it has a lower variance and coefficient of variation. That means the predicting model with all variables has a better predicting stability. Therefore, it is a good try to introduce the internet searches index for financial distress prediction to improve the predicting performance under the era of big data. Also, no matter which year of data sets is used for predicting, ANSEM always obtains the best performance with all variables except the internet searches indices too.


### 4.3. Summary

Empirical results indicate that the proposed ANSEM can improve the predicting performance (accuracy and stability) of financial distress with different sample sizes, especially with the small sample sizes. This is because we use the novel uni-int decision making method of soft set theory as the integrated method and we employ real financial institutions as experts to improve the ES classifier. As a result, ANSEM has fewer restrictions and can make full use of more information when it is used to predicting practice. In other words, ANSEM can improve the practice performance.Also, empirical results indicate that the internet searches index can offer additional information to improve the predicting performance under the era of big data because internet searches indices mainly reflect the potential demands and concerns of general public [40].


## 4.1. Empirical Results

In this paper, the 5-fold cross validation method is employed to perform the empirical experiment [39]. MATLAB (2016) is used to obtain optimal variables and predicting results of each model with different sample sizes.Financial distress prediction is a typical two-class classification problem. The predicting outputs are usually marked as either positive (P) or negative (N). More specifically, the outputs of financial distress prediction include four different results. One is the true positive (TP). TP means that the predicting output is positive and the real status is positive too. One is the false positive (FP). FP means that the predicting output is positive, but the real status is negative. The other one is the true negative (TN). TN means that the predicting output is negative and the real status is also negative. The last one is false negative (FN). FN means that the predicting output is negative, but the real status is positive. Thus, we can use an index called accuracy of correct classification (ACC) to measure the predicting accuracy of each method. The definition of ACC is shown as(17)
A
C
C
=
T
P
+
T
N
P
+
N


### 4.1.1. Empirical Results with Different Sample Sizes

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using all variables and data sets of the year(
t
-
2
) and (
t
-
3
) for percentages (25%, 75%), (50%, 50%), and (75%, 25%) are respectively listed in Tables 3–8.Table 3
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.733
0.700
0.667
0.867
0.967
2
0.933
0.967
0.967
0.933
0.867
0.867
3
0.867
0.933
0.933
0.800
0.967
0.867
4
0.767
0.733
0.867
0.967
0.733
0.800
5
0.900
0.933
0.800
0.800
0.900
0.933
Mean
0.853
0.860
0.853
0.833
0.867
0.887
Variance
0.005
0.014
0.011
0.014
0.007
0.004
Coefficient of variation
0.006
0.016
0.013
0.017
0.008
0.005Table 4
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.950
0.950
0.950
2
0.850
0.750
0.800
0.900
0.700
0.800
3
0.900
0.950
0.800
0.700
0.950
0.950
4
0.800
0.650
0.650
0.800
0.800
0.750
5
0.750
0.900
0.950
0.600
0.750
0.900
Mean
0.840
0.830
0.820
0.790
0.830
0.870
Variance
0.004
0.016
0.013
0.021
0.013
0.008
Coefficient of variation
0.005
0.019
0.016
0.026
0.016
0.009Table 5
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
1.000
0.900
0.900
0.900
0.900
1.000
2
0.800
0.700
0.700
0.600
0.800
0.900
3
0.900
0.900
0.900
0.800
0.900
0.900
4
0.800
0.900
0.900
0.800
0.600
0.900
5
0.800
0.600
0.700
0.600
0.700
0.700
Mean
0.860
0.800
0.820
0.740
0.780
0.880
Variance
0.008
0.020
0.012
0.018
0.017
0.012
Coefficient of variation
0.009
0.025
0.015
0.024
0.022
0.014Table 6
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.800
0.933
0.833
2
0.700
0.933
0.833
0.700
0.800
0.900
3
0.833
0.733
0.867
0.900
0.833
0.800
4
0.667
0.667
0.633
0.567
0.633
0.833
5
0.667
0.667
0.567
0.600
0.733
0.633
Mean
0.733
0.780
0.740
0.713
0.787
0.800
Variance
0.006
0.016
0.017
0.019
0.013
0.010
Coefficient of variation
0.008
0.021
0.024
0.027
0.016
0.013Table 7
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.900
0.700
0.900
2
0.750
0.850
0.900
0.700
0.900
0.700
3
0.800
0.650
0.750
0.650
0.850
0.850
4
0.700
0.800
0.650
0.900
0.800
0.800
5
0.650
0.600
0.600
0.600
0.600
0.650
Mean
0.740
0.760
0.740
0.750
0.770
0.780
Variance
0.004
0.017
0.014
0.020
0.015
0.011
Coefficient of variation
0.006
0.022
0.019
0.027
0.019
0.014Table 8
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.600
0.900
0.900
2
0.700
0.600
0.600
0.800
0.800
0.900
3
0.700
0.800
0.800
0.500
0.500
0.600
4
0.700
0.500
0.600
0.400
0.600
0.800
5
0.800
0.600
0.600
0.900
0.700
0.800
Mean
0.760
0.680
0.700
0.640
0.700
0.800
Variance
0.008
0.027
0.020
0.043
0.025
0.015
Coefficient of variation
0.011
0.040
0.029
0.067
0.036
0.019


### 4.1.2. Empirical Results with Different Variables

To investigate whether the internet searches index can improve financial distress prediction performance under the era of big data, we run the empirical experiment again with selected variables except the internet searches index. And we employ data sets for the percentage (50%, 50%) as experiment data sets. Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM with all variables except the internet searches index and data sets of the year(
t
-
2
) and (
t
-
3
) are listed in Tables 9 and 10.Table 9
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.850
0.850
0.800
0.950
0.700
2
0.850
0.950
0.900
0.750
0.700
0.950
3
0.900
0.850
0.950
0.850
0.950
0.950
4
0.800
0.550
0.600
0.500
0.650
0.750
5
0.750
0.800
0.750
0.950
0.800
0.850
Mean
0.840
0.800
0.810
0.770
0.810
0.840
Variance
0.004
0.023
0.019
0.028
0.019
0.013
Coefficient of variation
0.005
0.028
0.024
0.037
0.024
0.015Table 10
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.700
0.750
0.850
0.750
0.750
2
0.750
0.900
0.900
0.950
0.900
0.850
3
0.800
0.600
0.650
0.600
0.600
0.650
4
0.700
0.850
0.800
0.550
0.850
0.900
5
0.650
0.500
0.500
0.550
0.550
0.600
Mean
0.740
0.710
0.720
0.700
0.730
0.750
Variance
0.004
0.028
0.023
0.035
0.023
0.016
Coefficient of variation
0.006
0.039
0.032
0.050
0.032
0.022


## 4.1.1. Empirical Results with Different Sample Sizes

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using all variables and data sets of the year(
t
-
2
) and (
t
-
3
) for percentages (25%, 75%), (50%, 50%), and (75%, 25%) are respectively listed in Tables 3–8.Table 3
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.733
0.700
0.667
0.867
0.967
2
0.933
0.967
0.967
0.933
0.867
0.867
3
0.867
0.933
0.933
0.800
0.967
0.867
4
0.767
0.733
0.867
0.967
0.733
0.800
5
0.900
0.933
0.800
0.800
0.900
0.933
Mean
0.853
0.860
0.853
0.833
0.867
0.887
Variance
0.005
0.014
0.011
0.014
0.007
0.004
Coefficient of variation
0.006
0.016
0.013
0.017
0.008
0.005Table 4
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.950
0.950
0.950
2
0.850
0.750
0.800
0.900
0.700
0.800
3
0.900
0.950
0.800
0.700
0.950
0.950
4
0.800
0.650
0.650
0.800
0.800
0.750
5
0.750
0.900
0.950
0.600
0.750
0.900
Mean
0.840
0.830
0.820
0.790
0.830
0.870
Variance
0.004
0.016
0.013
0.021
0.013
0.008
Coefficient of variation
0.005
0.019
0.016
0.026
0.016
0.009Table 5
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
1.000
0.900
0.900
0.900
0.900
1.000
2
0.800
0.700
0.700
0.600
0.800
0.900
3
0.900
0.900
0.900
0.800
0.900
0.900
4
0.800
0.900
0.900
0.800
0.600
0.900
5
0.800
0.600
0.700
0.600
0.700
0.700
Mean
0.860
0.800
0.820
0.740
0.780
0.880
Variance
0.008
0.020
0.012
0.018
0.017
0.012
Coefficient of variation
0.009
0.025
0.015
0.024
0.022
0.014Table 6
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (25%, 75%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.800
0.933
0.833
2
0.700
0.933
0.833
0.700
0.800
0.900
3
0.833
0.733
0.867
0.900
0.833
0.800
4
0.667
0.667
0.633
0.567
0.633
0.833
5
0.667
0.667
0.567
0.600
0.733
0.633
Mean
0.733
0.780
0.740
0.713
0.787
0.800
Variance
0.006
0.016
0.017
0.019
0.013
0.010
Coefficient of variation
0.008
0.021
0.024
0.027
0.016
0.013Table 7
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.900
0.800
0.900
0.700
0.900
2
0.750
0.850
0.900
0.700
0.900
0.700
3
0.800
0.650
0.750
0.650
0.850
0.850
4
0.700
0.800
0.650
0.900
0.800
0.800
5
0.650
0.600
0.600
0.600
0.600
0.650
Mean
0.740
0.760
0.740
0.750
0.770
0.780
Variance
0.004
0.017
0.014
0.020
0.015
0.011
Coefficient of variation
0.006
0.022
0.019
0.027
0.019
0.014Table 8
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (75%, 25%).
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.900
0.900
0.600
0.900
0.900
2
0.700
0.600
0.600
0.800
0.800
0.900
3
0.700
0.800
0.800
0.500
0.500
0.600
4
0.700
0.500
0.600
0.400
0.600
0.800
5
0.800
0.600
0.600
0.900
0.700
0.800
Mean
0.760
0.680
0.700
0.640
0.700
0.800
Variance
0.008
0.027
0.020
0.043
0.025
0.015
Coefficient of variation
0.011
0.040
0.029
0.067
0.036
0.019


## 4.1.2. Empirical Results with Different Variables

To investigate whether the internet searches index can improve financial distress prediction performance under the era of big data, we run the empirical experiment again with selected variables except the internet searches index. And we employ data sets for the percentage (50%, 50%) as experiment data sets. Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM with all variables except the internet searches index and data sets of the year(
t
-
2
) and (
t
-
3
) are listed in Tables 9 and 10.Table 9
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
2
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.900
0.850
0.850
0.800
0.950
0.700
2
0.850
0.950
0.900
0.750
0.700
0.950
3
0.900
0.850
0.950
0.850
0.950
0.950
4
0.800
0.550
0.600
0.500
0.650
0.750
5
0.750
0.800
0.750
0.950
0.800
0.850
Mean
0.840
0.800
0.810
0.770
0.810
0.840
Variance
0.004
0.023
0.019
0.028
0.019
0.013
Coefficient of variation
0.005
0.028
0.024
0.037
0.024
0.015Table 10
Predicting results of 5-fold cross-validation and summaries of ACC on mean, variance, and variance coefficient using data sets of the year(
t
-
3
) for the percentage (50%, 50%) and all variables except the internet searches index.
ES
CNN
EMEW
EMNN
EMRD
ANSEM
1
0.800
0.700
0.750
0.850
0.750
0.750
2
0.750
0.900
0.900
0.950
0.900
0.850
3
0.800
0.600
0.650
0.600
0.600
0.650
4
0.700
0.850
0.800
0.550
0.850
0.900
5
0.650
0.500
0.500
0.550
0.550
0.600
Mean
0.740
0.710
0.720
0.700
0.730
0.750
Variance
0.004
0.028
0.023
0.035
0.023
0.016
Coefficient of variation
0.006
0.039
0.032
0.050
0.032
0.022


## 4.2. Comparison and Discussion

In this paper, we employ three statistical indices of ACC from the 5-fold cross validation procedure to evaluate the performance of each model. The three statistical indices are mean, variance, and coefficient of variation. The mean is critical on evaluating the predicting accuracy of each model. The variance and coefficient of variation are critical on evaluating the predicting stability of each model.


### 4.2.1. Results Comparison and Discussion with Different Sample Sizes

Based on Tables3–8, we can demonstrate the mean clearly in Figure 6. Meanwhile, we can illustrate the variance and coefficient of variance in Figures 7 and 8.Figure 6
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 7
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 8
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables3–8 and Figure 6, we can find out that the proposed novel soft ensemble model (ANSEM) for financial distress prediction has the highest mean accuracy no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is, the predicting accuracy of ANSEM does not have a lot of changes. However, other predicting models are different, especially the NN and EWNN. With changes in sample sizes, means of NN and EWNN are significantly different. Because NN is not good at dealing with small sample sizes [29], the mean of ES also does not change a lot. Because ESs are real practitioners, they have to pay attention on the risk.Moreover, all predicting methods except ES have a worse predicting accuracy using data sets of the year(
t
-
3
) than those using data sets of the year (
t
-
2
). Predicting financial distress on a long term is much more complicated than a short term prediction.Similar as the conclusion of predicting accuracy, from Tables3–8 and Figures 7-8, we can find out that ANSEM has the best predicting stability no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is employed, ANSEM has the lowest variance and coefficient of variation of ACC. Besides, ES also has an excellent performance on the predicting stability. This is because ESs are real practitioners. They have to pay attention to the risk. Other predicting models are different; especially the NN and EWNN have the worst predicting stability.Without a surprise, predicting stability of models using data sets of the year(
t
-
2
) outperforms that using data sets of the year (
t
-
3
).


### 4.2.2. Results Comparison and Discussion with Different Variables

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using different variables and data sets of the year(
t
-
2
) and (
t
-
3
) for the percentage (50%, 50%) are listed in Tables 4, 7, 9, and 10. From Tables 4, 7, 9, and 10, we can compare the mean of ACC clearly in Figure 9. Meanwhile, we can compare the variance and coefficient of variance in Figures 10 and 11.Figure 9
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 10
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 11
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables4, 7, 9, and 10 and Figures 9–11, we can find out that the predicting model with all variables uniformly performs better no matter which year of data sets is used for predicting. It has a higher predicting accuracy comparing to those with all variables except the internet searches index. Also, it has a lower variance and coefficient of variation. That means the predicting model with all variables has a better predicting stability. Therefore, it is a good try to introduce the internet searches index for financial distress prediction to improve the predicting performance under the era of big data. Also, no matter which year of data sets is used for predicting, ANSEM always obtains the best performance with all variables except the internet searches indices too.


## 4.2.1. Results Comparison and Discussion with Different Sample Sizes

Based on Tables3–8, we can demonstrate the mean clearly in Figure 6. Meanwhile, we can illustrate the variance and coefficient of variance in Figures 7 and 8.Figure 6
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 7
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 8
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables3–8 and Figure 6, we can find out that the proposed novel soft ensemble model (ANSEM) for financial distress prediction has the highest mean accuracy no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is, the predicting accuracy of ANSEM does not have a lot of changes. However, other predicting models are different, especially the NN and EWNN. With changes in sample sizes, means of NN and EWNN are significantly different. Because NN is not good at dealing with small sample sizes [29], the mean of ES also does not change a lot. Because ESs are real practitioners, they have to pay attention on the risk.Moreover, all predicting methods except ES have a worse predicting accuracy using data sets of the year(
t
-
3
) than those using data sets of the year (
t
-
2
). Predicting financial distress on a long term is much more complicated than a short term prediction.Similar as the conclusion of predicting accuracy, from Tables3–8 and Figures 7-8, we can find out that ANSEM has the best predicting stability no matter which year of data sets or which percentage of data sets is employed for predicting. No matter how small or big the sample size is employed, ANSEM has the lowest variance and coefficient of variation of ACC. Besides, ES also has an excellent performance on the predicting stability. This is because ESs are real practitioners. They have to pay attention to the risk. Other predicting models are different; especially the NN and EWNN have the worst predicting stability.Without a surprise, predicting stability of models using data sets of the year(
t
-
2
) outperforms that using data sets of the year (
t
-
3
).


## 4.2.2. Results Comparison and Discussion with Different Variables

Predicting results of ES, NN, EMEW, EMNN, EMRD, and ANSEM using different variables and data sets of the year(
t
-
2
) and (
t
-
3
) for the percentage (50%, 50%) are listed in Tables 4, 7, 9, and 10. From Tables 4, 7, 9, and 10, we can compare the mean of ACC clearly in Figure 9. Meanwhile, we can compare the variance and coefficient of variance in Figures 10 and 11.Figure 9
Mean of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 10
Variance of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.Figure 11
Coefficient of variation of ACC. ES is the expert system method. CNN is the convolutional neural network. EMEW is the ensemble model based on equal weight. EMNN is the ensemble model based on the convolutional neural network. EMRD is the ensemble model based on the rough set theory and evidence theory. ANSEM is the soft ensemble model based on the uni-int decision making method.From Tables4, 7, 9, and 10 and Figures 9–11, we can find out that the predicting model with all variables uniformly performs better no matter which year of data sets is used for predicting. It has a higher predicting accuracy comparing to those with all variables except the internet searches index. Also, it has a lower variance and coefficient of variation. That means the predicting model with all variables has a better predicting stability. Therefore, it is a good try to introduce the internet searches index for financial distress prediction to improve the predicting performance under the era of big data. Also, no matter which year of data sets is used for predicting, ANSEM always obtains the best performance with all variables except the internet searches indices too.


## 4.3. Summary

Empirical results indicate that the proposed ANSEM can improve the predicting performance (accuracy and stability) of financial distress with different sample sizes, especially with the small sample sizes. This is because we use the novel uni-int decision making method of soft set theory as the integrated method and we employ real financial institutions as experts to improve the ES classifier. As a result, ANSEM has fewer restrictions and can make full use of more information when it is used to predicting practice. In other words, ANSEM can improve the practice performance.Also, empirical results indicate that the internet searches index can offer additional information to improve the predicting performance under the era of big data because internet searches indices mainly reflect the potential demands and concerns of general public [40].


## 5. Conclusions

In this paper, we extended the model research for financial distress prediction with different sample sizes by proposing a novel soft ensemble model including qualitative classifiers (ES) and quantitative classifiers (CNN) based on the uni-int decision making method of soft set theory (UI). It constructs a soft set representation of each classifier then uses the optimal decision on soft sets to identify the status of firms. It inherits the advantage of ES, CNN, and UI. This lets ANSEM have fewer restrictions and can make full use of more information when it is used to practice. Compared with ES, CNN, EMEW, EMNN, and EMRD, our method ANSEM has demonstrated superior predicting performance for financial distress on accuracy and stability with different sample sizes. We also extend the research of predicting variables under the era of big data by introducing the internet searches index for financial distress prediction. It offers some new information to improve performance of financial distress prediction.Though the empirical result is satisfactory, there are some inadequacies in this work. First of all, we use the annual number of searches for sample’s name on “Baidu” website as the internet searches index. However, there is more than one searching engine in the world. We need to collect more internet searching data from different searching engines. Secondly, individual classifiers are simply discussed in the process of constructing the novel soft ensemble model. Because the key part of this paper is the integrated method, more attention should be paid on individual classifiers to get better performance. Finally, the proposed predicting method and predicting variables are applied to the data set of Chinese listed firms. There are no samples from unlisted firms and others regions. More samples should be included to evaluate the predicting performance.



---

*Source: 3085247-2019-04-04.xml*