# Input Selection Methods for Soft Sensor Design: A Survey
## 2 SS Design Stages
However, due to the complex processes occurring in industrial plants and given the great amount of collected data, the use of gray- or black-box data-driven identification approaches can be a good choice, since satisfactory results can be achieved with reasonable computational and time efforts. Such data-driven models are only based on the empirical observations of the process and generally require slight knowledge of the system. A great work of data processing is needed anyway. Since it is hard to find a general solution equally satisfactory for any case, any plant experts' knowledge must still be taken into account. It can regard the input variables importance, the system order, the operating range, time delay, degree of nonlinearity or sampling times. Linear models are usually preferable as they are computationally easier, since the numerical procedures and the design of a controller are simpler. Such approximation is possible when certain conditions are met, like small variations around the nominal working point, a small degree of nonlinearity of the process or what degree of approximation is needed for the model. If a linear model does not show good results, a nonlinear identification approach is needed. In the industrial field, parametric structures such as FIR, ARX or ARMAX are widely used in both the linear and nonlinear (NFIR, NARX, NARMAX, respectively) cases [1,62; 63], as well as static models. Industrial SSs are generally designed with Artificial Neural Networks (ANN), mainly with the Multi-Layer Perceptron (MLP) structure [7; 11; 63; 64], Convolutional Neural Networks (CNN) [65; 66], Generative Adversarial Networks (GAN) [67; 68], Radial Basis Networks [69], Wavelet Networks [70],Hinging Hyperplanes [71], Deep Belief Networks [72; 73], Stacked Autoencoders [74], Long Short-Term Memory Networks [75], Support Vector Regression [76], Gaussian Processes Regression [77], Extreme Learning Machines [78], Fuzzy Systems and Neuro-Fuzzy Systems [79; 80], just to mention a few [5; 81; 82; 83; 84; 85; 86; 87; 88]. In some cases, the designer can choose to create more linear or nonlinear models for the same system, each one for a different working point, instead of a single one covering all of the system dynamics. In such cases, the models are aggregated by a suitable algorithm such as fuzzy logic or neural stacking. Such approaches are called ensemble methods [88]. In some cases, the designer could deal with systems showing finite time delay between the input variables and the process output, sometimes caused by the measurement process. Several approaches are proposed in literature. In [57], a FIR model along with an Expectation-Maximization (EM)-based algorithm is used to estimate the model parameters and the time delays. In [79], a Takagi-Sugeno-fuzzy model is tuned using a genetic-algorithm-based approach to identify delays. Genetic Algorithms have also been used to estimate the time delays as in [89; 90], where they are used to minimize the Joint Conditional Entropy between the input and output variables. In [91], the problem of selecting input time-lags is treated as a variable selection problem with a multidimensional mutual information estimator. Mutual Information is also applied to delay selection in the design of a SS in [92; 93], where the delay is estimated by using the Normalized Mutual Information and delayed replicas of the inputs. In [10], delays are estimated through the learning phase of a Deep Belief Network. Once the structure has been chosen, the whole preprocessed and selected data set should be partitioned in subsets for the last two design steps, as:  
* Identification data
* Validation data  
The first allows to identify the candidate models and empirically estimate their unknown parameters. Finally, the validation step exploits validation data to verify whether the model is able to adequately represent the system and perform generalization to new samples.