# Proposing Enhanced Feature Engineering and a Selection Model for Machine Learning Processes
## 3 Groundwork of eFES
### Theory  
eFES model manifests itself into specialized optimization goals of the features in the datasets. The most crucial step of all is the Extended Feature Engineering (EFE) that we refer when we build upon existing EF techniques. These five definitions help build the technical mechanics of the proposed model of eFES unit. **Definition 1**. : _Let there be a Logical Table (LT) module that regulates the ML process during eFES constructions. Let LT have 3D coordinates as x, y, and z to track, parallelize, and update the x - overfit(0:1), y - underfit(0:1), z - optimumfit(-1:+1). Let there be two functions, Feature Adder as +F, and Feature Remover as -F, based on linearity of the classifier for each feature under test for which the RoOpF (Rule. 1) is valid. Let Lt. RoOpF > 0.5 to be considered of acceptable predictive value._  
eFES LT module builds very important functions at initial layers for adding a good fit feature and removing a bad fit feature from the set of features available to it, especially when algorithm blend is being engineered. Clearly, not all features will have an optimum predictive value and thus identifying them will count towards optimization. The feature adder function is built as:  
\[+\mathbb{F}(x,y,z)=+\mathbb{F}_{F_{n}}=\ \ (F_{n}\cup\ F_{n+1})\ \sum_{i=1}^{Z}(LT.score\ (i))+\ \sum_{j,k=1}^{x,y}\ (LT. score\ (j,k)) \tag{1}\]  
The feature remover function is built as:  
\[-\mathbb{F}(x,y,z)=-\mathbb{F}_{F_{n}}=\ \ (F_{n}\cap\ F_{n+1})\ \sum_{j,k=1}^{x,y}(LT.score\ (j,k))-\ \sum_{i=1}^{z}\ (LT. score(i)) \tag{2}\]  
Very similar to \(k\)-means clustering [12] concept, that is highly used in unsupervised learning, LT implements feature weights mechanism (FWM) so it can report a feature with high relevancy score and non-redundant in a quantized form. Thus, we define:  
\[FWM(X,Y,Z)=\sum_{x=1}^{X}\sum_{y=1}^{Y}\sum_{z=1}^{Z}\ (u_{x}w_{x}.\ u_{y}w_{y}.\ u_{z}w_{z})\ (\Delta(x,y,z)) \tag{3}\]  
Let us define a Continuous Random Vector \(CRV\in Q^{N}\), and Discrete Random Variable \(DRV\in H=\{h_{1}h_{2}h_{3},\ldots\ldots\ \ldots,J_{n}\}\). The density function of the random vector based on cumulative probability is \(P(CRV)=\sum_{i=1}^{N}P_{H}(h_{i})p\)\(CRV\)\(|DRV\), \(P_{H}(h_{i})\) being a priori probability of class. Figure 4: Illustration of the conceptual view of LT Modules in 3D space. Figure 5: **(a)** This test shows the variance of the LT module for the cost function for all three co-ordinates and then z (optimum-fitness). This is the ideal behavior; **(b)** This test shows the real (experimental) behavior; **(c)** This shows the ideal shift of all 3 coordinates in space while they are tested by the model in parallel. Each coordinate (\(x,y,z\)) lies on the black lines in each direction. Then, based on the scoring reported by LT object (and cost function), they either sit on positive point or negative as shown; **(d)** This shows the ideal spread of each point, when z is optimized with the lowest cost function. Figure 6: Illustration of probability-based feature binary classification. Overlapped matrices for _Red.F_ and _Irr.F_, for which the probability scope resulted in acceptable errors by stepping into vector space, for which 0.8 \(>\) err \(>\) 0.2. As we observe that the higher error limit (e) (err, green line, round symbol) and lower error limit (E), (Err, blue line, square symbol) bound the feature correlation in this process. Our aim is to spread the distribution in z-dimension for optimum fitting as features are added. The red line (diamond symbol) that separates the binary distribution of Redundant Feature (Red.F) and Irrelevant Features (Irr.F) based on error bounds. The green and red lines define the upper and lower limit of the error, in which all features correlate. Here, we build a mutual information (MI) function [14] so we can quantify the relevance of a feature upon other in the random set and this information is used to build the construct for _Irr.F_, since once our classifier learns, it will mature the _Irr.F_ learning module as defined in the Algorithm 1.