## 3 Class Imbalance Problem
### Motivations of Genetic Algorithm for Class Imbalance  
The distance between two feature vectors can be minimized by simple perturbation. Keeping this analogy in mind, GA is used to extend the sample set by generating a uniform feature vector set for skewed category of images. Details of the process are presented in this section. As elaborated in Figure 1, these feature vectors serve as the representation of the virtual images in the skewed category that have the image portions from multiple samples already present in the skewed class and that are generated by the GA at the time of feature extraction. The proposed GA based architecture effectively handles the classifier bias problem and ensures better image retrieval output. Figure 1: Virtual image generation through genetic algorithm (GA). (**a**) and (**b**) are original images; (**c**) is the virtual image generated by combining image regions. Proof of Effectiveness  
Consider the case of regression in which training data can be represented as: \((x^{(i)},y^{(i)})\), where \(x^{(i)}\) are the observations in the form of feature vectors, \(y^{(i)}\) are the corresponding outputs, and \(i=\{1,2,...,m\}\) are number of training examples. For training data, our goal is to generate the optimal parameters \(\theta\) that may reduce the difference between the known output \(y^{(i)}\) and the computed output \(h_{\theta}(x^{(i)})\) based on the feature observations. The cost function \(J(\theta)\) can be represented as:  
\[J(\theta)=min\frac{1}{2}\sum_{i=1}^{m}\bigg{(}h_{\theta}(x^{(i)})-y^{(i)} \bigg{)}^{2}, \tag{1}\]  
where  
\[h_{\theta}(x^{(i)})=\sum_{j=1}^{n}(\theta_{j}x_{j})=\theta^{T}x, \tag{2}\]  
and \(j=\{1,2,...,n\}\) are corresponding features in the feature vector \(x\). The optimal selection of parameters \(\theta\) ensures minimum value of the cost function \(J(\theta)\) in Equation (1). The parameters \(\theta\) can be optimized through the gradient decent algorithm for every \(j\) as:  
\[\theta_{j}:=\theta_{j}+\beta\sum_{i=1}^{m}(y^{(i)}-h_{\theta}(x^{(i)}))x_{j}^ {(i)}, \tag{3}\]  
where \(\beta\) is learning rate and it's value is predefined. The probabilistic interpretation of the phenomena is given by:  
\[P(y^{(i)}|x^{(i)};\theta)=\frac{1}{\sqrt{2\pi}\sigma}exp\bigg{(}-\frac{y^{(i) }-\theta^{T}x^{(i)}}{2\sigma^{2}}\bigg{)}. \tag{4}\]  
Now consider the case of normal CBIR approach where our target is to obtain the top \(k\) most similar images against a query image in terms of distance, i.e., the images that give the minimum distance in terms of features. Therefore, the objective function of CBIR can be expressed as:  
\[J(\hat{y})=min\frac{1}{2}\sum_{i=1}^{k}(x_{q}-y^{(i)})^{2}. \tag{5}\]  
Here \(x_{q}\) is the query image and \(y^{(i)}\) are the images present in the image repository in the form of feature vectors, whereas, \(J(\hat{y})\) represents the objective function of CBIR that targets the \(k\) most similar images against a query image that have lowest aggregate distance. The probabilistic interpretation of Equation (5) can be given as:  
\[P(y^{(i)}|x_{q})=\frac{1}{\sqrt{2\pi}\sigma}exp\bigg{(}-\frac{y^{(i)}-x_{q}}{ 2\sigma^{2}}\bigg{)}. \tag{6}\]  
Equation (6) shows that the semantic output is normally distributed and objective function of Equation (5) is very similar to Equation (1) but parameter \(\theta\) that exists in Equation (2) does not exists in Equation (5). Therefore, there is no way to optimize the usual output that appears on the base of features and depends on the robustness of features; in case of weak features the semantic output cannot be improved. Here it is important to mention that as the parameter \(\theta\) in Equation (1) can be optimized through various algorithms, for example, gradient decent, conjugate gradient, etc., and therefore even the weak features lead to the right output. However, as in Equation (5), there is no way by which we can optimize features, therefore, output of CBIR system may remain inappropriate.