## 3 Class Imbalance Problem
### Genetic Algorithm Architecture for Class Imbalance Problem
#### 3.4.2 Genetic Operators
\begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{3}{c}{**INPUT**} \\ \hline Input: & \(\overrightarrow{a}=(a_{1},\ldots,a_{n})\) & \(\dim(\overrightarrow{a})=N\) & (I.1) \\ \hline \multicolumn{3}{c}{**MIDDLE (HIDDEN) LAYER**} \\ \hline Input: & \(\overrightarrow{b}=U\overrightarrow{a}\) & \(\dim(\overrightarrow{b})=M\) & (I.2) \\ Output: & \(\overrightarrow{c}=f(b-\overrightarrow{s})\) & \(\dim(\overrightarrow{c})=M\) & (I.3) \\ U: & \(M\times N\) weight matrix & \\ \(\underline{f}\): & hidden layer activation function & \\ \(\overrightarrow{s}\): & thresholds & \\ \hline \multicolumn{3}{c}{**OUTPUT LAYER**} \\ \hline Input: & \(\overrightarrow{d}=W\overrightarrow{c}\) & \(\dim(\overrightarrow{d})=K\) & (I.4) \\ Output: & \(\overrightarrow{e}=g(d-\overrightarrow{t})\) & \(\dim(\overrightarrow{e})=1\) & (I.5) \\ W: & \(1\times M\) weight matrix & \\ \(\underline{g}\): & output layer activation function & \\ \(\underline{t}\): & thresholds & \\ \hline \multicolumn{3}{c}{**ERROR CORRECTION**} \\ \hline MSE: & \(\underline{E}=1/2(\overrightarrow{p}-\overrightarrow{c})^{2}\) & (I.6) \\ \(\Delta W_{ij}\) & \(=-\alpha\partial E/\partial W_{ij}=\alpha\delta_{i}c_{j}\) & (I.6) \\ \(\Delta t_{i}\) & \(=\alpha\delta_{i}\) & (I.7) \\ \(\Delta U_{ji}\) & \(=-\beta\partial E/\partial U_{ji}\) & (I.8) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of Neural network structure for every image category used in this work. ### Class Finalization through Genetic Classifier Comity Learning

For GCCL based classification, majority voting rule (MVR) is used in our implementation. MVR does not consider any individual behaviour of the weak classifiers, it only counts the largest number of classifiers that agree with each other [21]. If we have a series of weak classifiers \(\{\hat{C}_{i}(x),1\leq i\leq Q\}\), we can represent MVR as [21]:

\[C^{*}(x)=sgn\bigg{\{}\sum_{i}\hat{C}_{i}(X)-\frac{Q-1}{2}\bigg{\}}. \tag{25}\]

For image classification, we combine all the classifiers trained through GA based asymmetric and symmetric bagging procedures resulting in the form of a powerful and stable semantic classifier. Hence, the class of image is one that has a maximum combined association value for SVM and neural network classifiers. In order to show why our bagging procedure works, consider the following scenario [33]: Let \((x,y)\) be a data sample in the training set \(\mathcal{L}\), where \(y\) is the class label of the sample \(x\) and \(L\) is drawn from a probability distribution P. Suppose \(\varphi(x,L)\) is a simple classifier constructed by the asymmetric bagging and symmetric bagging, the aggregated predictor is averaged over \(L\) of \(\varphi(x,L)\), i.e., \(\varphi_{A}(x)=\mathcal{E}_{\mathcal{L}}\varphi(x,L)\) where \(\mathcal{E}_{\mathcal{L}}\) is the expectation over prediction. If \(x\) and \(y\) are fixed then

\[\mathcal{E}_{\mathcal{L}}(y-\varphi(x,L))^{2}=y^{2}-2y\mathcal{E}_{\mathcal{L }}\varphi(x,L)+\mathcal{E}_{\mathcal{L}}\varphi^{2}(x,L). \tag{26}\]

Using \(\mathcal{E}_{\mathcal{L}}\varphi(x,L)=\varphi_{A}(x)\) and the inequality \(\mathcal{E}Z^{2}>(\mathcal{E}Z)^{2}\) in Equation (26) in third term,

\[\mathcal{E}_{\mathcal{L}}(y-\varphi(x,L))^{2}\geq(y-\varphi_{A}(x))^{2}. \tag{27}\]

Therefore from Equations (26) and (27) we draw:

\[y^{2}-2y\mathcal{E}_{\mathcal{L}}\varphi(x,L)+\mathcal{E}_{\mathcal{L}}\varphi ^{2}(x,L)\geq(y-\varphi_{A}(x))^{2}. \tag{28}\]

Integrating both sides of Equation (27) over the sample distribution, the MSE of \(\varphi_{A}(x)\) is smaller than the MSE averaged over \(L\) of \(\varphi(x,L)\) depending on the inequality of both sides,

\[[\mathcal{E}_{\mathcal{L}}\varphi(x,L)]^{2}\leq\mathcal{E}_{\mathcal{L}} \varphi^{2}(x,L). \tag{29}\]

The instability effect is clearly visible from the inequality. Two sides will be nearly equal if \(\varphi(x,L)\) does not change too much, therefore, in this case, aggregation will not help. If \(\varphi(x,L)\) shows high variance then the aggregation will produce more improvement.