# An Ensemble Learning for Predicting Breakdown Field Strength of Polyimide Nanocomposite Films

Authors: Hai Guo; Jinghua Yin; Jingying Zhao; Lei Yao; Xu Xia; Hao Luo
Journal: Journal of Nanomaterials
 (2015)

---

## Abstract

Using the method of Stochastic Gradient Boosting, ten SMO-SVR are constructed into a strong prediction model (SGBS model) that is efficient in predicting the breakdown field strength. Adopting the method of in situ polymerization, thirty-two samples of nanocomposite films with different percentage compositions, components, and thicknesses are prepared. Then, the breakdown field strength is tested by using voltage test equipment. From the test results, the correlation coefficient (CC), the mean absolute error (MAE), the root mean squared error (RMSE), the relative absolute error (RAE), and the root relative squared error (RRSE) are 0.9664, 14.2598, 19.684, 22.26%, and 25.01% with SGBS model. The result indicates that the predicted values fit well with the measured ones. Comparisons between models such as linear regression, BP, GRNN, SVR, and SMO-SVR have also been made under the same conditions. They show that CC of the SGBS model is higher than those of other models. Nevertheless, the MAE, RMSE, RAE, and RRSE of the SGBS model are lower than those of other models. This demonstrates that the SGBS model is better than other models in predicting the breakdown field strength of polyimide nanocomposite films.


---

## Body

## 1. Introduction

As one of material products that have been developed for a long time, polyimide film (PI film) has been mainly applied in high and new technology industries such as aerospace, machinery, electrical and electronics engineering, optical communication, LCD, automobile, precision instrument, gas separation, and microelectronics [1]. With the development of nanotechnology, nanoparticles of different sizes, percentage compositions, and components have been mixed with PI by more and more researchers to produce high-quality polyimide nanocomposite films [2, 3].Breakdown field strength is an important characteristic parameter to characterize polyimide nanocomposite films. It can be calculated by (breakdown voltage)/(film thickness). Many researchers have already made studies and analysis on the breakdown field strength of nanocomposite films [4–6]. There are many factors that can impact the breakdown field strength, including the type of nanoparticle, dielectric constant, electric conductivity, coefficient of thermal conductivity, composition, nanoparticles’ size and specific area, and composite film thickness. In order to establish the knowledge base for the material property regarding breakdown field strength of polyimide nanocomposite films, large quantities of experiments have to be prepared and measure related characteristics. Nevertheless, it is also well known that getting properties data is very costly in terms of time and materials. It is for this reason that developing a fast and efficiency method to predict the breakdown field strength of polyimide nanocomposite films is very much in demand.Intelligent computing and neuronal network have been widely applied in performance prediction, identification, and optimization of nanocomposite films. Yang et al. used a generalized regression neural network (GRNN) to predict the friction coefficient ofCr1-xAlxC film [7]. Cho et al. optimized the characteristics of ITO/Al/ITO multilayer films by advantages of neural network and genetic algorithm [8]. Bahramian made use of an artificial neural network to predict the growth rate of TiO2 nanostructured film [9]. Ensemble learning has now become a new hotspot of intelligent computing [10–13]. By using ensemble, several weak classifiers can be constructed into a strong classifier. Some common methods of ensemble learning include boosting, bagging, and stacking. Boosting [14] keeps upgrading weights during data extraction and revises the weights of data set that have been classified wrong. In the end, several weak classifiers are obtained and can be constructed into a strong classifier. Bagging [15] is used the reiterative training to get several classifiers based on a training set. Stacking [16] contains two layers. In the first layer, different algorithms are employed to generate several weak classifiers. At the same time, a new data set with the same size of the original data set is also generated. Then, the new data set together with a new algorithm can be used to construct the classifier of the second layer.The purpose of this paper is to develop a Stochastic Gradient Boosting + SMO-SVR model (SGBS model) to predict the breakdown field strength of polyimide nanocomposite films. In what follows, film preparation and prediction model are introduced first. The experimental details for sputtering systems and materials are described next. Then, the experimental results are described and the establishment of SGBS model with 10-fold cross validation results is carried out. Comparison experiments between linear regression, BP neural network (BP), general regression neural network (GRNN), SVR (support vector regression), and SMO-SVR models are conducted.


## 2. Nanocomposite Film Preparation and Prediction Model

### 2.1. Preparation of Nanocomposite Films

The method of in situ polymerization is used to prepare the polyimide matrix inorganic nanocomposite film. Experimental materials include 4,4′-diaminodiphenyl ether (ODA), pyromellitic dianhydride (PMDA), dimethylacetamide (DMAc), Al2O3, rutile TiO2, BaTiO3, SiO2, and ethanol. The detailed purchase information of the above experimental materials is shown in Table 1. Firstly, put PMDA into the solution of ODA in DMAc to produce an amount of polyamide acid of certain viscosity. Secondly, add in different nanoparticles. Finally, let the mixture go through paving membrane heat treatment and imidization transform.Table 1
The detailed purchase information of experimental materials.
Material
Companies
4,4′-Diaminodiphenyl ether
Sinopharm Chemical Reagent Co., Ltd.
Pyromellitic dianhydride
Sinopharm Chemical Reagent Co., Ltd.
Dimethylacetamide
Tianjin Fuyu Fine Chemical Co., Ltd.
Al2O3
Beijing DK Nano Technology Co., Ltd.
Rutile TiO2
Beijing DK Nano Technology Co., Ltd.
BaTiO3
Beijing DK Nano Technology Co., Ltd.
SiO2
Beijing DK Nano Technology Co., Ltd.
Ethanol
Tianjin Fuyu Fine Chemical Co., Ltd.To verify our SGBS model, we have designed and prepared in this paper nanoparticle samples with different types, sizes, ratios, and thicknesses. The SEM image of surface appearance of pure PI is given in Figure1(a). The surface of film is smooth and its tightness is well. Figure 1(b) shows the SEM image of surface appearance of PI/BaTiO3 composite film doped with content of 60 wt%. There are large amounts of BaTiO3 nanoparticles exposing on the surface of film with the size from 100 nm to 300 nm. Comparing with the other films, the surface appearance of film has been changed to be rough due to the dopant of BaTiO3 nanoparticles. Its density has also been decreased. Figure 1(c) shows the SEM image of surface appearance of PI/TiO2 composite film doped with content of 5 wt%. TiO2 can absorb ultraviolet ray to change the color of film into brown. The surface appearance of film is smooth compared with pure PI film. Figure 1(d) shows the SEM image of surface appearance of PI/Al2O3 composite film doped with content of 20 wt%. There are some nanoparticles appearing in the surface of film, uniformly, where the diameter of nanoparticles is observed to be 70 nm and some particles are not cluster. In conclusion, the prepared hybrid PI particles are evenly distributed.Figure 1
SEM image of the nanocomposite film. (a) The SEM image of pure PI film; (b) the SEM image of surface appearance of PI/BaTiO3 composite film doped with content of 60 wt%; (c) the SEM image of surface appearance of PI/TiO2 composite film doped with content of 5 wt%; (d) the SEM image of surface appearance of PI/Al2O3 composite film doped with content of 20 wt%.
(a)
(b)
(c)
(d)


### 2.2. Standard SMO-SVR Model

SVM (support vector machine) [17] is one of the most typical machine learning methods in field of statistical learning theory. It has a very good learning ability in a small sampling space. Compared with BP, GRNN, and other neural networks, it has advantages such as structural risk minimization, global optimization solution, high dimensional space, and being linearly separable. It has been widely used for recognitions and regressions.The core of support vector regression (SVR) is to find out a hyperplane, namely, to a minimized linear function with inequality constraints, giving a training sample(1)S=x1,y1,…,xn,yn,where x means input and y means output. The training sample needs to be turned into a linearly separable problem when it is a linear inseparability. Generally, the sample space can be mapped from a low-dimensional space into a high dimensional space by using a kernel function. The nonlinear regression function (estimation) is (2)fx=ωφx+b,where b is a threshold, ω is a weight, and φx is the nonlinear mapping function. The loss function is shown as follows:(3)cx,y,f=y-fxε.By importing a Lagrange multiplier, the minimization of the objective function can be expressed as(4)min12ω2+C∑i=1nξi+ξi′subject  toyi-ωφxi-b≤ε+ξiωφxi+b-yi≤ε+ξi′ξi,ξi′≥0,i=1,2,…,n.Transform (4) into a dual problem, and we get (5)max⁡-12∑i,jnai∗-aiaj∗-aiKxi,xj-ε∑i=1nai∗+ai+∑i=1nyiai-ai∗,where C is a penalty factor and ai∗ and ai are Lagrangians; then the regression function is (6)fx=∑i=1Nai-ai∗Kxi,xj+b,where N is the number of support vectors in (6) [18]. In this paper, a normalized polynomial kernel has been taken as the kernel function.John Platt, from Microsoft Research, proposed the SMO (sequential minimal optimization) algorithm in order to shorten the training time of support vector regression in 1998 [19]. SMO can optimize the α value of two samples at one time. Through the loop iteration with a given times M, the α value of all samples can be optimized, as shown in Figure 2. For the data in Table 2, the method of 10-fold cross validation is used to make the model training on the standard SVR and SMO-SVR. The time of modeling SVR and SMO-SVR is 0.26 s and 0.03 s, respectively. SMO can promote the training efficiency of support vector regression while reducing the training time of the model.Table 2
Data of breakdown field strengths of nanocomposite films.
Samples
Input
Output
Type of nanoparticles
Dielectric constant
Thickness of films (μm)
Thermal conductivity (W/cm·k)
Electrical resistivity (Ω·m)
Ratio of nanoparticles (wt%)
Size of nanoparticles (nm)
Specific area of nanoparticles (m2/g)
Breakdown field strength of the hybrid PI (kV/mm)
1
BaTiO3
1400
25
0.50
1 × 105
10
100
12
157
2
BaTiO3
1400
25
0.50
1 × 105
15
100
12
137
3
BaTiO3
1400
25
0.50
1 × 105
20
100
12
125
4
BaTiO3
1400
25
0.50
1 × 105
25
100
12
119
5
BaTiO3
1400
25
0.50
1 × 105
30
100
12
105
6
BaTiO3
1400
25
0.50
1 × 105
50
100
12
72
7
BaTiO3
1400
25
0.50
1 × 105
60
100
12
54
8
BaTiO3
1400
25
0.50
1 × 105
70
100
12
41
9
Rutile TiO2
100
25
0.40
9 × 107
0
35
70
230
10
Rutile TiO2
100
25
0.63
9 × 107
1
35
70
240
11
Rutile TiO2
100
25
0.63
9 × 107
3
35
70
210
12
Rutile TiO2
100
25
0.63
9 × 107
4
35
70
177
13
Rutile TiO2
100
25
0.63
9 × 107
5
35
70
170
14
Rutile TiO2
100
25
0.63
9 × 107
7
35
70
160
15
SiO2
1.56
25
160
1 × 1016
5
40
300
313
16
SiO2
1.56
25
160
1 × 1016
10
40
300
307
17
SiO2
1.56
25
160
1 × 1016
15
40
300
327
18
SiO2
1.56
25
160
1 × 1016
20
40
300
252
19
SiO2
1.56
25
160
1 × 1016
25
40
300
240
20
αAl2O3
10
30
4.10
1 × 1014
4
30
25
275
21
αAl2O3
10
30
4.10
1 × 1014
8
30
25
263
22
αAl2O3
10
30
4.10
1 × 1014
12
30
25
234
23
αAl2O3
10
30
4.10
1 × 1014
16
30
25
212
24
αAl2O3
10
30
4.10
1 × 1014
20
30
25
235
25
αAl2O3
10
30
4.10
1 × 1014
24
30
25
203
26
Al2O3
8
30
29.31
1 × 1014
15
13
100
190
27
Al2O3
8
30
29.31
1 × 1014
20
13
100
189
28
Al2O3
8
30
29.31
1 × 1014
25
13
100
178
29
SiO2
1.56
30
160
1 × 1016
10
7
350
307
30
SiO2
1.56
30
160
1 × 1016
15
7
350
327
31
SiO2
1.56
30
160
1 × 1016
20
7
350
252
32
SiO2
1.56
30
160
1 × 1016
25
7
350
240Figure 2
Flowchart of the SMO optimization algorithm.


### 2.3. Promoting the Model by Stochastic Gradient Boosting

Boosting [20], as one of the most important ensemble learning methods, is to obtain a predictive function by construction and ensemble of a series of predictive functions. The core idea of gradient boosting, proposed by Friedman [21], is to construct an ensemble learning machine by calculating a loss function and letting the function descend along its gradient. In other words, it is to calculate the loss function of previous model, so as to build a new model along the descending gradient direction of the loss function. Eventually, a regression model can be generated through N times of iterations. In this paper, we use the Stochastic Gradient Boosting to turn ten SMO-SVR models into a strong predictive model (SGBS model). The algorithmic steps of the SGBS model are shown as follows:(1)
{X,Y} is a data sample of breakdown field strengths of different nanocomposite thin films, in which X=(x1,x2,…,xn) is input and Y=(y1,y2,…,yn) is output. The loss function is L(yi,F(xi)),i∈{1,2,…,n}, and the prediction model of SMO-SVR is hX.(2)
Initialize the model(7)F0x=argminβ⁡∑i=1NLyi,β.(3)
Calculate the upper limit(M) of iterations by cross validation.(4)
Calculate the gradient direction(8)y~i=-∂Lyi,Fxi∂FxiFx=Fm-1x,i=1,…,N.(5)
Fit the SMO-SVR by using least squares to obtain the fitting modelhxi,αm.(6)
Figure out new step length based on the loss function(9)βm=argmin∑i=1NLyi,Fm-1xi+βhxi,αm.(7)
Generate a new model(10)Fmx=Fm-1x+βmhx,αm.(8)
Stochastically extractf% of the training sample to fit the SMO-SVR during every time of iteration; then we can get the SGBS model after M times of iterations:(11)Fx=∑m=1Mβmhx,αm.


## 2.1. Preparation of Nanocomposite Films

The method of in situ polymerization is used to prepare the polyimide matrix inorganic nanocomposite film. Experimental materials include 4,4′-diaminodiphenyl ether (ODA), pyromellitic dianhydride (PMDA), dimethylacetamide (DMAc), Al2O3, rutile TiO2, BaTiO3, SiO2, and ethanol. The detailed purchase information of the above experimental materials is shown in Table 1. Firstly, put PMDA into the solution of ODA in DMAc to produce an amount of polyamide acid of certain viscosity. Secondly, add in different nanoparticles. Finally, let the mixture go through paving membrane heat treatment and imidization transform.Table 1
The detailed purchase information of experimental materials.
Material
Companies
4,4′-Diaminodiphenyl ether
Sinopharm Chemical Reagent Co., Ltd.
Pyromellitic dianhydride
Sinopharm Chemical Reagent Co., Ltd.
Dimethylacetamide
Tianjin Fuyu Fine Chemical Co., Ltd.
Al2O3
Beijing DK Nano Technology Co., Ltd.
Rutile TiO2
Beijing DK Nano Technology Co., Ltd.
BaTiO3
Beijing DK Nano Technology Co., Ltd.
SiO2
Beijing DK Nano Technology Co., Ltd.
Ethanol
Tianjin Fuyu Fine Chemical Co., Ltd.To verify our SGBS model, we have designed and prepared in this paper nanoparticle samples with different types, sizes, ratios, and thicknesses. The SEM image of surface appearance of pure PI is given in Figure1(a). The surface of film is smooth and its tightness is well. Figure 1(b) shows the SEM image of surface appearance of PI/BaTiO3 composite film doped with content of 60 wt%. There are large amounts of BaTiO3 nanoparticles exposing on the surface of film with the size from 100 nm to 300 nm. Comparing with the other films, the surface appearance of film has been changed to be rough due to the dopant of BaTiO3 nanoparticles. Its density has also been decreased. Figure 1(c) shows the SEM image of surface appearance of PI/TiO2 composite film doped with content of 5 wt%. TiO2 can absorb ultraviolet ray to change the color of film into brown. The surface appearance of film is smooth compared with pure PI film. Figure 1(d) shows the SEM image of surface appearance of PI/Al2O3 composite film doped with content of 20 wt%. There are some nanoparticles appearing in the surface of film, uniformly, where the diameter of nanoparticles is observed to be 70 nm and some particles are not cluster. In conclusion, the prepared hybrid PI particles are evenly distributed.Figure 1
SEM image of the nanocomposite film. (a) The SEM image of pure PI film; (b) the SEM image of surface appearance of PI/BaTiO3 composite film doped with content of 60 wt%; (c) the SEM image of surface appearance of PI/TiO2 composite film doped with content of 5 wt%; (d) the SEM image of surface appearance of PI/Al2O3 composite film doped with content of 20 wt%.
(a)
(b)
(c)
(d)


## 2.2. Standard SMO-SVR Model

SVM (support vector machine) [17] is one of the most typical machine learning methods in field of statistical learning theory. It has a very good learning ability in a small sampling space. Compared with BP, GRNN, and other neural networks, it has advantages such as structural risk minimization, global optimization solution, high dimensional space, and being linearly separable. It has been widely used for recognitions and regressions.The core of support vector regression (SVR) is to find out a hyperplane, namely, to a minimized linear function with inequality constraints, giving a training sample(1)S=x1,y1,…,xn,yn,where x means input and y means output. The training sample needs to be turned into a linearly separable problem when it is a linear inseparability. Generally, the sample space can be mapped from a low-dimensional space into a high dimensional space by using a kernel function. The nonlinear regression function (estimation) is (2)fx=ωφx+b,where b is a threshold, ω is a weight, and φx is the nonlinear mapping function. The loss function is shown as follows:(3)cx,y,f=y-fxε.By importing a Lagrange multiplier, the minimization of the objective function can be expressed as(4)min12ω2+C∑i=1nξi+ξi′subject  toyi-ωφxi-b≤ε+ξiωφxi+b-yi≤ε+ξi′ξi,ξi′≥0,i=1,2,…,n.Transform (4) into a dual problem, and we get (5)max⁡-12∑i,jnai∗-aiaj∗-aiKxi,xj-ε∑i=1nai∗+ai+∑i=1nyiai-ai∗,where C is a penalty factor and ai∗ and ai are Lagrangians; then the regression function is (6)fx=∑i=1Nai-ai∗Kxi,xj+b,where N is the number of support vectors in (6) [18]. In this paper, a normalized polynomial kernel has been taken as the kernel function.John Platt, from Microsoft Research, proposed the SMO (sequential minimal optimization) algorithm in order to shorten the training time of support vector regression in 1998 [19]. SMO can optimize the α value of two samples at one time. Through the loop iteration with a given times M, the α value of all samples can be optimized, as shown in Figure 2. For the data in Table 2, the method of 10-fold cross validation is used to make the model training on the standard SVR and SMO-SVR. The time of modeling SVR and SMO-SVR is 0.26 s and 0.03 s, respectively. SMO can promote the training efficiency of support vector regression while reducing the training time of the model.Table 2
Data of breakdown field strengths of nanocomposite films.
Samples
Input
Output
Type of nanoparticles
Dielectric constant
Thickness of films (μm)
Thermal conductivity (W/cm·k)
Electrical resistivity (Ω·m)
Ratio of nanoparticles (wt%)
Size of nanoparticles (nm)
Specific area of nanoparticles (m2/g)
Breakdown field strength of the hybrid PI (kV/mm)
1
BaTiO3
1400
25
0.50
1 × 105
10
100
12
157
2
BaTiO3
1400
25
0.50
1 × 105
15
100
12
137
3
BaTiO3
1400
25
0.50
1 × 105
20
100
12
125
4
BaTiO3
1400
25
0.50
1 × 105
25
100
12
119
5
BaTiO3
1400
25
0.50
1 × 105
30
100
12
105
6
BaTiO3
1400
25
0.50
1 × 105
50
100
12
72
7
BaTiO3
1400
25
0.50
1 × 105
60
100
12
54
8
BaTiO3
1400
25
0.50
1 × 105
70
100
12
41
9
Rutile TiO2
100
25
0.40
9 × 107
0
35
70
230
10
Rutile TiO2
100
25
0.63
9 × 107
1
35
70
240
11
Rutile TiO2
100
25
0.63
9 × 107
3
35
70
210
12
Rutile TiO2
100
25
0.63
9 × 107
4
35
70
177
13
Rutile TiO2
100
25
0.63
9 × 107
5
35
70
170
14
Rutile TiO2
100
25
0.63
9 × 107
7
35
70
160
15
SiO2
1.56
25
160
1 × 1016
5
40
300
313
16
SiO2
1.56
25
160
1 × 1016
10
40
300
307
17
SiO2
1.56
25
160
1 × 1016
15
40
300
327
18
SiO2
1.56
25
160
1 × 1016
20
40
300
252
19
SiO2
1.56
25
160
1 × 1016
25
40
300
240
20
αAl2O3
10
30
4.10
1 × 1014
4
30
25
275
21
αAl2O3
10
30
4.10
1 × 1014
8
30
25
263
22
αAl2O3
10
30
4.10
1 × 1014
12
30
25
234
23
αAl2O3
10
30
4.10
1 × 1014
16
30
25
212
24
αAl2O3
10
30
4.10
1 × 1014
20
30
25
235
25
αAl2O3
10
30
4.10
1 × 1014
24
30
25
203
26
Al2O3
8
30
29.31
1 × 1014
15
13
100
190
27
Al2O3
8
30
29.31
1 × 1014
20
13
100
189
28
Al2O3
8
30
29.31
1 × 1014
25
13
100
178
29
SiO2
1.56
30
160
1 × 1016
10
7
350
307
30
SiO2
1.56
30
160
1 × 1016
15
7
350
327
31
SiO2
1.56
30
160
1 × 1016
20
7
350
252
32
SiO2
1.56
30
160
1 × 1016
25
7
350
240Figure 2
Flowchart of the SMO optimization algorithm.


## 2.3. Promoting the Model by Stochastic Gradient Boosting

Boosting [20], as one of the most important ensemble learning methods, is to obtain a predictive function by construction and ensemble of a series of predictive functions. The core idea of gradient boosting, proposed by Friedman [21], is to construct an ensemble learning machine by calculating a loss function and letting the function descend along its gradient. In other words, it is to calculate the loss function of previous model, so as to build a new model along the descending gradient direction of the loss function. Eventually, a regression model can be generated through N times of iterations. In this paper, we use the Stochastic Gradient Boosting to turn ten SMO-SVR models into a strong predictive model (SGBS model). The algorithmic steps of the SGBS model are shown as follows:(1)
{X,Y} is a data sample of breakdown field strengths of different nanocomposite thin films, in which X=(x1,x2,…,xn) is input and Y=(y1,y2,…,yn) is output. The loss function is L(yi,F(xi)),i∈{1,2,…,n}, and the prediction model of SMO-SVR is hX.(2)
Initialize the model(7)F0x=argminβ⁡∑i=1NLyi,β.(3)
Calculate the upper limit(M) of iterations by cross validation.(4)
Calculate the gradient direction(8)y~i=-∂Lyi,Fxi∂FxiFx=Fm-1x,i=1,…,N.(5)
Fit the SMO-SVR by using least squares to obtain the fitting modelhxi,αm.(6)
Figure out new step length based on the loss function(9)βm=argmin∑i=1NLyi,Fm-1xi+βhxi,αm.(7)
Generate a new model(10)Fmx=Fm-1x+βmhx,αm.(8)
Stochastically extractf% of the training sample to fit the SMO-SVR during every time of iteration; then we can get the SGBS model after M times of iterations:(11)Fx=∑m=1Mβmhx,αm.


## 3. Experiment and Result Analysis

### 3.1. Construction of the Experimental Sample

Test the breakdown field strength by using a withstanding voltage tester (type: CS2674C). Voltage range is 0–50 kV; test error is ±5%; leakage current measurement range is 0.5–20 mA; measurement error is ±5%. Take polymethylphenylsiloxane fluid with high insulation strength (which can be up to 16 kV/mm) as the medium, with the rate of voltage rise of 500 V/s. Test the breakdown field strength of the nanocomposite films prepared in Section2.1. Select 20 samples as a group for testing each hybrid PI, and take the mean value of the 10 middle breakdown field strengths as the breakdown field strength of the group. Then, calculate the standard deviation. After the experiment we get the data of breakdown field strengths as shown in Table 2. The input X includes type (X1), doping ratio (X2), electrical resistivity (X3), dielectric constant (X4), thermal conductivity (X5), size (X6), and specific area (X7) of nanoparticles and thickness of the film (X8). The output Y is the breakdown field strength of the hybrid PI.


### 3.2. Evaluation Indicators of the Prediction Performance

There are many indicators for evaluating the prediction performance, such as MAPE (mean absolute percentage error), MAE (mean absolute error), RMSE (root mean squared error), CC (correlation coefficient), RAE (relative absolute error), and RRSE (root relative squared error). In this paper, CC, MSE, RMSE, RSE, and RRSE are taken as evaluation indicators.CC (correlation coefficient) represents the relevancy of the linear regression relationship betweenfi (independent variable) and yi (response variable) of the model. It is a coefficient of a simple linear correlation between yi and its estimated values. CC is larger than zero and ranges from 0 to 1. A larger CC means a more relevant linear regression relationship.MSE (mean absolute error) is to judge the different degree between predicted values and real ones. It is inversely proportional to prediction accuracy. A smaller MSE means a better effect the predictor can be with. It can be expressed as(12)MAE=1n∑i=1nfi-yi=1n∑i=1nei,where fi represents predicted values and yi represents real values.RMSE (root mean squared error) is the square root of the ratio of the quadratic sum of deviations between predicted values and real values to the times(n) of predictions. It is sensitive to maximum or minimal errors of a group of predicted values and therefore can well reflect prediction accuracy. RMSE is inversely proportional to the prediction accuracy. The smaller the RMSE is, the more accurate the predictor can be. It can be expressed as follows:(13)RMSE=∑i=1nfi-yi2n.RSE (relative absolute error) is the different degree between the absolute deviation obtained from the prediction model and the absolute deviation obtained by directly speculating the training sample. It is inversely proportional to prediction accuracy. The smaller the RSE is, the higher the prediction accuracy can be:(14)RSE=∑i=1nfi-yi∑i=1nf-i-yi.RRSE (root relative squared error) can be calculated as follows:(15)RRSE=∑i=1nfi-yi2∑i=1nf-i-yi2.RRSE is also inversely proportional to prediction accuracy. The smaller the RRSE is, the higher the prediction accuracy can be [22].


### 3.3. Experimental Results and Analysis

To verify the SGBS prediction model, the type, dielectric constant, electrical resistivity, thermal conductivity, size and specific area of nanoparticles, and the thickness of films in Table2 are taken as the input X, and the breakdown field strength of hybrid PI is taken as the output Y. In this paper, we use Macbook Pro (CPU: Intel I7-2640 M; memory: 16G) as the hardware for experiments and use Matlab 2012a to program prediction model.Experiment 1.
Use the method of 10-fold cross validation to fit the data in Table2. Main parameters of the SGBS model include kernel function, C and S, of which C is the penalty factor of SMO-SVR and S is the extracting ratio when training the sample. First of all, normalize the sample data. Three kernel functions—normalized polynomial kernel, polynomial kernel, and RBF kernel—are employed to test the sample. The results are listed in Table 3. From the table we know that the normalized polynomial kernel has the highest correlation coefficient but lowest mean absolute error, root mean squared error, relative absolute error, and root relative squared error. Therefore, we choose it as the kernel function for the SGBS model.
For optimization function, the penalty factor of outliers is selected by experience as well as by experiments. Figure3 shows the comparison on prediction performances of SGBS model when C varies from 1 to 1.8. Figure 3(a) demonstrates that the correlation coefficient reaches its peak values when C=1.6, 1.7, and 1.8, which are chosen as the value of C, for correlation coefficient is proportional to prediction accuracy. By analyzing Figures 3(b) and 3(d) we know that the mean absolute error and relative absolute error reach their minimal values when C=1.7, only larger than the values when C=1.6 and 1.8. Therefore, they reach optimal performance when C=1.7, for mean absolute error and relative absolute error are inversely proportional to prediction accuracy.
By analyzing Figures3(c) and 3(d) we know that the smallest root mean squared error and root relative squared error appear at C=1.6, followed by C=1.7 and C=1.8. Based on the above analysis, this paper takes C=1.7 as the optimum value.
S is the specify shrinkage rate in Stochastic Gradient Boosting, namely, the proportion of the stochastically extracted training sample. Figure 4 shows the impact of variation of S on prediction performance when C=1.7. When S=1, correlation coefficient reaches its peak value, and the prediction attains the best performance as shown in Figure 4(a). By analyzing Figures 4(b)–4(e) we know that when S=1, mean absolute error, root mean squared error, relative absolute error, and root relative squared error reach their minimal values, signifying the best prediction performance.
According to the above analysis we know that whenC=1.7 and S=1 are taken as the optimum values of the SGBS model, the prediction performance is the best. Figure 5 shows the degree of fitting between predicted values and real ones. Figure 6 shows the absolute error ratio of prediction. The tables and figures demonstrate that the error ratios of sample 8 (70 wt%, 100 nm, BaTiO3), sample 17 (15 wt%, 40 nm, SiO2), and sample 30 (15 wt%, 7 nm, SiO2) are all larger than 15%, indicating an ordinary fitting, while the error ratios of other samples are less than 15%, indicating a better fitting. For all the six multicomponent TiO2 film samples with the thickness of 35 μm, the prediction errors are not larger than 15%. For the eight BaTiO3 film samples, there is one sample of which the prediction errors are larger than 15%. For all the nine multicomponent Al2O3 film samples, the prediction errors are not larger than 15%, while for the nine multicomponent SiO2 film samples, there are also two samples of which the prediction errors are larger than 15 wt%. In this model, the prediction performance of the multicomponent Al2O3 and rutile TiO2 with different thicknesses is better than that of the BaTiO3 and SiO2 film samples. For the thirty-two nanocomposite films with different components, mixtures, and thicknesses, there are twenty-nine film samples of which the prediction errors are lower than 15%, proving that the model is of practical value in actual engineering works.Table 3
Comparison on prediction performances of three kernel functions.
Indicators
Normalized  polynomial
Polynomial
RBF
Correlation coefficient
0.9603
0.9236
0.8901
Mean absolute error
16.8162
23.5927
29.6448
Root mean squared error
21.2266
29.2044
34.7791
Relative absolute error
26.255%
36.8351%
46.2842%
Root relative squared error
26.9779%
37.1173%
44.2024%Figure 3
The impacts of variation of penalty factor (C) on prediction performance. (a) The correlation coefficient with different penalty factors (C); (b) the mean absolute error with different penalty factors (C); (c) the mean absolute error with different penalty factors (C); (d) the root mean squared error with different penalty factors (C); (e) the root relative squared error with different penalty factor (C).
(a)
(b)
(c)
(d)
(e)Figure 4
Impact ofS on prediction performance. (a) Correlation coefficient relative to shrinkage rate (S) of training samples; (b) mean absolute error relative to shrinkage rate (S) of training samples; (c) root mean squared error relative to shrinkage rate (S) of training samples; (d) relative absolute error relative to shrinkage rate (S) of training samples; (e) root relative squared error relative to shrinkage rate (S) of training samples.
(a)
(b)
(c)
(d)
(e)Figure 5
Comparison between real values and predicted values of the SGBS mode.Figure 6
Analysis on sample errors.Experiment 2.
In order to further verify the SGBS model, it needs to be compared to other models, namely, the linear regression, BP neural network, GRNN neural network, SVR (support vector regression), and SMO-SVR, under the same conditions. Comparison results of the prediction performance of these models are shown in Table4. The correlation coefficient of SGBS model is 0.962, larger than that of the models of linear regression [23], BP neural network [24], GRNN neural network [7], SMO-SVR [25], and SVR [26], proving that the linear regression relationship of the SGBS model is better than that of the other five models. In the SGBS model, the root mean squared error and root mean squared error are 20.8668 and 26.520%, respectively, lower than those of the other models, certifying its better prediction performance.Table 4
Comparisons of models’ prediction performance.
Indicators
SGBS
Linear regression
BP
GRNN
SVR
SMO-SVR
CC
0.962
0.9438
0.960
0.957
0.459
0.917
RMSE
20.866
25.235
21.350
21.932
79.662
30.401
RRSE
26.520%
32.072%
27.135%
27.874%
101.246%
38.639%Experiment 3.
In order to validate the generosity of the model, the actual measurement results conducted by Shi et al. in [27] are chosen to do the prediction. In this preference, the particle size is 30 μm, and the doping ratio is 2 wt% with PI/nano-Al2O3 composite films. The thickness of film is 30 μm. Its breakdown field strength is 233 kV/mm. Adopting the samples in Table 2 as the training set of SGBS model. For the parameters of the model, C=1.7, S=1. Using the 2 wt%-PI/nano-Al2O3 composite films in [27] as testing sample, the results could be obtained as shown in Table 5. For the test result of breakdown field strength that is 233 kV/mm, its prediction value is 225.9667 kV/mm by SGBS model. The error ratio is 3.0185%. The MAE, RMSE, RAE, and RRSE are 7.0334, 7.0334, 24.5976, and 24.5976. The prediction and actual data are in good agreement. This method could predict the PI/nano-Al2O3 composite films effectively.Table 5
The comparison of the predicted and tested values of  breakdown field strength.
Measured value [27] (kV/mm)
Predicted value (kV/mm)
MAE
RMSE
RAE (%)
RRSE (%)
Error ratio (%)
233
225.9667
7.0334
7.0334
24.5976
24.5976
3.0185


## 3.1. Construction of the Experimental Sample

Test the breakdown field strength by using a withstanding voltage tester (type: CS2674C). Voltage range is 0–50 kV; test error is ±5%; leakage current measurement range is 0.5–20 mA; measurement error is ±5%. Take polymethylphenylsiloxane fluid with high insulation strength (which can be up to 16 kV/mm) as the medium, with the rate of voltage rise of 500 V/s. Test the breakdown field strength of the nanocomposite films prepared in Section2.1. Select 20 samples as a group for testing each hybrid PI, and take the mean value of the 10 middle breakdown field strengths as the breakdown field strength of the group. Then, calculate the standard deviation. After the experiment we get the data of breakdown field strengths as shown in Table 2. The input X includes type (X1), doping ratio (X2), electrical resistivity (X3), dielectric constant (X4), thermal conductivity (X5), size (X6), and specific area (X7) of nanoparticles and thickness of the film (X8). The output Y is the breakdown field strength of the hybrid PI.


## 3.2. Evaluation Indicators of the Prediction Performance

There are many indicators for evaluating the prediction performance, such as MAPE (mean absolute percentage error), MAE (mean absolute error), RMSE (root mean squared error), CC (correlation coefficient), RAE (relative absolute error), and RRSE (root relative squared error). In this paper, CC, MSE, RMSE, RSE, and RRSE are taken as evaluation indicators.CC (correlation coefficient) represents the relevancy of the linear regression relationship betweenfi (independent variable) and yi (response variable) of the model. It is a coefficient of a simple linear correlation between yi and its estimated values. CC is larger than zero and ranges from 0 to 1. A larger CC means a more relevant linear regression relationship.MSE (mean absolute error) is to judge the different degree between predicted values and real ones. It is inversely proportional to prediction accuracy. A smaller MSE means a better effect the predictor can be with. It can be expressed as(12)MAE=1n∑i=1nfi-yi=1n∑i=1nei,where fi represents predicted values and yi represents real values.RMSE (root mean squared error) is the square root of the ratio of the quadratic sum of deviations between predicted values and real values to the times(n) of predictions. It is sensitive to maximum or minimal errors of a group of predicted values and therefore can well reflect prediction accuracy. RMSE is inversely proportional to the prediction accuracy. The smaller the RMSE is, the more accurate the predictor can be. It can be expressed as follows:(13)RMSE=∑i=1nfi-yi2n.RSE (relative absolute error) is the different degree between the absolute deviation obtained from the prediction model and the absolute deviation obtained by directly speculating the training sample. It is inversely proportional to prediction accuracy. The smaller the RSE is, the higher the prediction accuracy can be:(14)RSE=∑i=1nfi-yi∑i=1nf-i-yi.RRSE (root relative squared error) can be calculated as follows:(15)RRSE=∑i=1nfi-yi2∑i=1nf-i-yi2.RRSE is also inversely proportional to prediction accuracy. The smaller the RRSE is, the higher the prediction accuracy can be [22].


## 3.3. Experimental Results and Analysis

To verify the SGBS prediction model, the type, dielectric constant, electrical resistivity, thermal conductivity, size and specific area of nanoparticles, and the thickness of films in Table2 are taken as the input X, and the breakdown field strength of hybrid PI is taken as the output Y. In this paper, we use Macbook Pro (CPU: Intel I7-2640 M; memory: 16G) as the hardware for experiments and use Matlab 2012a to program prediction model.Experiment 1.
Use the method of 10-fold cross validation to fit the data in Table2. Main parameters of the SGBS model include kernel function, C and S, of which C is the penalty factor of SMO-SVR and S is the extracting ratio when training the sample. First of all, normalize the sample data. Three kernel functions—normalized polynomial kernel, polynomial kernel, and RBF kernel—are employed to test the sample. The results are listed in Table 3. From the table we know that the normalized polynomial kernel has the highest correlation coefficient but lowest mean absolute error, root mean squared error, relative absolute error, and root relative squared error. Therefore, we choose it as the kernel function for the SGBS model.
For optimization function, the penalty factor of outliers is selected by experience as well as by experiments. Figure3 shows the comparison on prediction performances of SGBS model when C varies from 1 to 1.8. Figure 3(a) demonstrates that the correlation coefficient reaches its peak values when C=1.6, 1.7, and 1.8, which are chosen as the value of C, for correlation coefficient is proportional to prediction accuracy. By analyzing Figures 3(b) and 3(d) we know that the mean absolute error and relative absolute error reach their minimal values when C=1.7, only larger than the values when C=1.6 and 1.8. Therefore, they reach optimal performance when C=1.7, for mean absolute error and relative absolute error are inversely proportional to prediction accuracy.
By analyzing Figures3(c) and 3(d) we know that the smallest root mean squared error and root relative squared error appear at C=1.6, followed by C=1.7 and C=1.8. Based on the above analysis, this paper takes C=1.7 as the optimum value.
S is the specify shrinkage rate in Stochastic Gradient Boosting, namely, the proportion of the stochastically extracted training sample. Figure 4 shows the impact of variation of S on prediction performance when C=1.7. When S=1, correlation coefficient reaches its peak value, and the prediction attains the best performance as shown in Figure 4(a). By analyzing Figures 4(b)–4(e) we know that when S=1, mean absolute error, root mean squared error, relative absolute error, and root relative squared error reach their minimal values, signifying the best prediction performance.
According to the above analysis we know that whenC=1.7 and S=1 are taken as the optimum values of the SGBS model, the prediction performance is the best. Figure 5 shows the degree of fitting between predicted values and real ones. Figure 6 shows the absolute error ratio of prediction. The tables and figures demonstrate that the error ratios of sample 8 (70 wt%, 100 nm, BaTiO3), sample 17 (15 wt%, 40 nm, SiO2), and sample 30 (15 wt%, 7 nm, SiO2) are all larger than 15%, indicating an ordinary fitting, while the error ratios of other samples are less than 15%, indicating a better fitting. For all the six multicomponent TiO2 film samples with the thickness of 35 μm, the prediction errors are not larger than 15%. For the eight BaTiO3 film samples, there is one sample of which the prediction errors are larger than 15%. For all the nine multicomponent Al2O3 film samples, the prediction errors are not larger than 15%, while for the nine multicomponent SiO2 film samples, there are also two samples of which the prediction errors are larger than 15 wt%. In this model, the prediction performance of the multicomponent Al2O3 and rutile TiO2 with different thicknesses is better than that of the BaTiO3 and SiO2 film samples. For the thirty-two nanocomposite films with different components, mixtures, and thicknesses, there are twenty-nine film samples of which the prediction errors are lower than 15%, proving that the model is of practical value in actual engineering works.Table 3
Comparison on prediction performances of three kernel functions.
Indicators
Normalized  polynomial
Polynomial
RBF
Correlation coefficient
0.9603
0.9236
0.8901
Mean absolute error
16.8162
23.5927
29.6448
Root mean squared error
21.2266
29.2044
34.7791
Relative absolute error
26.255%
36.8351%
46.2842%
Root relative squared error
26.9779%
37.1173%
44.2024%Figure 3
The impacts of variation of penalty factor (C) on prediction performance. (a) The correlation coefficient with different penalty factors (C); (b) the mean absolute error with different penalty factors (C); (c) the mean absolute error with different penalty factors (C); (d) the root mean squared error with different penalty factors (C); (e) the root relative squared error with different penalty factor (C).
(a)
(b)
(c)
(d)
(e)Figure 4
Impact ofS on prediction performance. (a) Correlation coefficient relative to shrinkage rate (S) of training samples; (b) mean absolute error relative to shrinkage rate (S) of training samples; (c) root mean squared error relative to shrinkage rate (S) of training samples; (d) relative absolute error relative to shrinkage rate (S) of training samples; (e) root relative squared error relative to shrinkage rate (S) of training samples.
(a)
(b)
(c)
(d)
(e)Figure 5
Comparison between real values and predicted values of the SGBS mode.Figure 6
Analysis on sample errors.Experiment 2.
In order to further verify the SGBS model, it needs to be compared to other models, namely, the linear regression, BP neural network, GRNN neural network, SVR (support vector regression), and SMO-SVR, under the same conditions. Comparison results of the prediction performance of these models are shown in Table4. The correlation coefficient of SGBS model is 0.962, larger than that of the models of linear regression [23], BP neural network [24], GRNN neural network [7], SMO-SVR [25], and SVR [26], proving that the linear regression relationship of the SGBS model is better than that of the other five models. In the SGBS model, the root mean squared error and root mean squared error are 20.8668 and 26.520%, respectively, lower than those of the other models, certifying its better prediction performance.Table 4
Comparisons of models’ prediction performance.
Indicators
SGBS
Linear regression
BP
GRNN
SVR
SMO-SVR
CC
0.962
0.9438
0.960
0.957
0.459
0.917
RMSE
20.866
25.235
21.350
21.932
79.662
30.401
RRSE
26.520%
32.072%
27.135%
27.874%
101.246%
38.639%Experiment 3.
In order to validate the generosity of the model, the actual measurement results conducted by Shi et al. in [27] are chosen to do the prediction. In this preference, the particle size is 30 μm, and the doping ratio is 2 wt% with PI/nano-Al2O3 composite films. The thickness of film is 30 μm. Its breakdown field strength is 233 kV/mm. Adopting the samples in Table 2 as the training set of SGBS model. For the parameters of the model, C=1.7, S=1. Using the 2 wt%-PI/nano-Al2O3 composite films in [27] as testing sample, the results could be obtained as shown in Table 5. For the test result of breakdown field strength that is 233 kV/mm, its prediction value is 225.9667 kV/mm by SGBS model. The error ratio is 3.0185%. The MAE, RMSE, RAE, and RRSE are 7.0334, 7.0334, 24.5976, and 24.5976. The prediction and actual data are in good agreement. This method could predict the PI/nano-Al2O3 composite films effectively.Table 5
The comparison of the predicted and tested values of  breakdown field strength.
Measured value [27] (kV/mm)
Predicted value (kV/mm)
MAE
RMSE
RAE (%)
RRSE (%)
Error ratio (%)
233
225.9667
7.0334
7.0334
24.5976
24.5976
3.0185


## 4. Conclusions

This paper presents an ensemble learning method for predicting breakdown field strength of polyimide nanocomposite films. By using the method of Stochastic Gradient Boosting, ten SMO-SVR prediction models are constructed into a strong prediction model (SGBS model) that is efficient in predicting the breakdown field strength. Through analyzing the experiment data we obtain following conclusions:(1)
In prediction of thirty-two nanocomposite films of different components, particles, and thicknesses (25–30μm) by using the method of 10-fold cross validation, there are twenty-nine samples of which the prediction errors are lower than 15%, proving that the SGBS model is efficient in predicting the breakdown field strength of polyimide nanocomposite films.(2)
Comparisons show that the SGBS model has a larger correlation coefficient than that of linear regression, BP, GRNN, SVR, and SMO-SVR models but smaller root mean squared error and root relative squared error. Hence, prediction performance of the SGBS model is better than that of the other five models.(3)
The SGBS model shows a better prediction on Al2O3 and rutile TiO2 films than on BaTiO3 and SiO2 films.Next, some other ensemble learning methods will be employed to predict the corona resistance, dielectric constant, dielectric constant, and thermal properties of polyimide nanocomposite films.



---

*Source: 950943-2015-10-25.xml*