# Proposing Enhanced Feature Engineering and a Selection Model for Machine Learning Processes
## 3 Groundwork of eFES
### Theory
As discussed earlier, LG and GG functions are developed to regulate the accuracy and thus validity of the classifier is measured initially based on accuracy metric. Table 2 shows the probability of local and global error limits based on probability function (Figure 7) in terms of local (g) and global gain function (G). If we assume the fact of \(\{\Delta S_{1},\Delta S_{2}\), \(\Delta S_{3},\ldots,\Delta S_{n}\in I\}\), such that \(d\) (\(\Delta S_{i}\)) \(\
eq\) 0 and \(d\) (\(\Delta S_{i+1}\)) \(\
eq\) 0, where '\(I\)' is the global input of testing data. We also confirm the relevance of the feature in the set using objective Function construct in distribution '\(d\)', thus:  
\[ObF\left(d,\ I\right)=\frac{leg\left(Gin\left(I,F\left(\alpha_{s,2}\right) \right)\right)}{\left(err\left[\alpha_{s,1}\right],err\left[\alpha_{s,2}\right] \right)}\ \ \left|\ d\left(\Delta S_{i}\right)\ \
eq\ 0\ \right|\ \ for\ every\ F_{i}\ in\ group \tag{17}\]  
Then, Using Equations (14)-(17), we can finally get  
\[F.Eng(x,y,z)=\ \frac{1}{\left(k\times d\right)}\sum\limits_{i=1}^{K}\prod \limits_{l=-k}^{M}ObF\left(d,\ I\right)\ \times MH_{t}) \tag{18}\]  
\[F.Grp(x,y,z)=\ F.Eng(x,0,0)+\ F.Eng(0,y,0)-\ F.Eng(0,0,z) \tag{19}\]  
Figure 8 Illustration of Feature Engineering and Feature Group as constructed in the mathematical model and governed by the Algorithms 1 and 2, defined later. Metrics API is available from eMLEE package. The white, yellow, and red orbital shapes indicate the local gain progression through 3D space. The little 3D shapes (\(x,y,\) and \(z\)) in the accepted feature space in grouping indicates several (theoretically unlimited) instances of the optimized values as the quantization progresses. \begin{table}
\begin{tabular}{c c c c c} \hline  & **g (max)** & **G (min)** & **G (max)** & **g (min)** \\ \hline P (err) & 0.25 & 0.45 & 0.31 & 0.56 \\ P (Err) & 0.32 & 0.41 & 0.49 & 0.59 \\ \hline \end{tabular}
\end{table} 
Table 2: Typical observations. Figure 8: Illustration of Feature Engineering and Feature Group as constructed in the mathematical model. Figure 7: Illustration on probability of LG and GG function. will guarantee the optimum minimum set of features for acceptable classification accuracy. We use linear regression to compute the value of features to detect the non-linearity relationship between features, we thus implement a function, \(Func(U(t))=a+b*t\). Where a, and b are two test features and values can be determined by using linear regression techniques, so \(b=\frac{\sum_{t=1}^{T}\left(t-\overline{t}\right)\left(U(t)-\overline{u} \right)}{\sum_{t=1}^{T}\left(t-\overline{t}\right)^{2}}\). Where, \(a=\overline{u}-b*\overline{t}\), \(\overline{u}=\frac{1}{T}\sum\limits_{t=1}^{T}U\left(t\right)\), \(\overline{t}=\frac{1}{T}\sum\limits_{t=1}^{T}t\). These equations also minimize the squared error. To compute weighted function, we use feature ranking technique [12]. In this method, we will score each feature, based on quality measure such as information gain. Eventually, the large feature set will be reduced to a small feature set that is usable. The Feature Selection can be enhanced in several ways such as pre-processing, calculating information gain, error estimation, redundant feature or terms removal, and determining outlier's quantification, etc. The information gain can be determined as:  
\[Gain_{1}(w)=-\sum\limits_{j=1}^{M}P(M_{j}).\ \log P(M_{j})+P(w)\sum\limits_{j=1}^{M}P \left(M_{j}\mid w\right).\log P(M_{j}\mid w)+P\left(\overline{w}\right)\sum \limits_{j=1}^{M}P(M_{j}\mid\overline{w})\ \log\ P\left(M_{j}\mid\overline{w}\right) \tag{22}\]  
'\(M\)' shows the number of classes and '\(P\)' is the probability. '\(W\)' is the term that it contains as a feature. \(P\left(M_{j}\mid w\right)\) is the conditional probability. In practice, the gain is normalized using Entropy, such as  
\[Norm.Gain_{1}(w)=\frac{\left\{Gain_{1}(w)\right\}}{\left\{-\frac{w\mid w\mid w \mid}{\pi^{2}}\log\frac{w\mid w\mid}{\pi^{2}}\right\}} \tag{23}\]  
Here we apply conventional variance-mean techniques.