# A Review of Data Mining with Big Data towards Its Applications in the Electronics Industry
## 4 Data Mining with Big Data Applications in the Electronics Industry
### Data Handling  
Data preparation is the initial step of DM to collect the necessary data recording the feature values directly from the experimental data and historical observations or indirectly from the simulation results [4], in which the experimental data are the records of full factorials or fractional factorials while historical observations can be obtained either through online measurements or from historical accumulated records. The data preparation from the reviewed literature is summarized in Table 6.  
Through Table 6, we can see that the data for the verification of product design improvement and manufacturing process optimization were mainly based on experimental observation and historical records. The DM application in the production process monitoring and control for the tasks of fault detection and classification (FDC), run to run (R2R), statistical process control (SPC), and so on, worked mainly on the data obtained through online measurements while DM in production and quality management for the tasks such as scheduling, yield/cost/cycle time prediction, and so forth was conducted mainly based on historical records from ERP and MES along with some process simulation. The task of SCM and CRM is conducted mainly based on interactions and transaction records accumulated in the system of SCM, OMS, and CRM. Figure 3: The distribution of selected articles in different years and different stages. Data preprocessing techniques used in the selected applications are summarized in Table 7, from which we can see that most of the cleaning techniques were used for the observational data sets. Some imputation techniques such as the missing values-patient rule induction method (m-PRLM) [30], k-NN [84; 87], syndromes imputation [109] and so on were developed for filling in missing values. SVM [54], moving average smoothing [90], King-move neighborhood [93], Winter's exponentials smoothing [126; 127], and so on were employed for noise smoothing. Meanwhile, the methods of box plot [79; 88], PCA [97], clustering [122], and so forth were applied for outliers detection. However, the missing values, noise, outliers, and redundant data were omitted directly in most cases. \begin{table}
\begin{tabular}{c c c c} \hline
**Preprocess** & **Functions** & **Methods** & **Reference** \\ \hline \multirow{8}{*}{Data cleaning} & & m-PRLM & [30] \\  & & Delete & [37; 52; 81; 88; 96; 117; 123; 124; 130] \\  & Filling in missing values & Manually fill & [48] \\  & & k-NN & [84; 87] \\  & & Omit/replace & [85; 130] \\  & & Missing syndromes & [109] \\  & & SVM & [54] \\  & & Delete & [79; 82; 114; 120] \\  & Smoothing out noise & Moving average & [90] \\  & & King-move neighborhood & [93] \\  & & Winter’s exponentials & [126; 127] \\  & & Box plot & [79; 88] \\  & Handling outliers & Delete & [84; 96] \\  & & Online PCA & [97] \\  & & Clustering & [122] \\  & Handling redundant data & Detecting and removing & [81; 96] \\ \hline \multirow{8}{*}{Data transformation} & & Variance scaling & [35; 63] \\  & & Normalization & [37; 39; 46; 81; 55; 56; 57; 58; 60; 62; 64; 78; 80; 82; 88; 94; 98; 102; 104; 105; 108; 111; 113] \\  & & Text mining & [42; 123] \\  & & Fisher Z & [44; 47] \\  & & Box-Cox & [84] \\  & & Numerical into binary & [85] \\  & & Binary vector & [91; 93; 117] \\  & & Spreadsheet format & [95] \\ \hline \end{tabular}
\end{table} 
Table 7: The preprocessing techniques used in the reviewed literature.