Content‐based image retrieval using Gaussian–Hermite moments and firefly and grey wolf optimization

CAAI Transactions on Intelligence TechnologyVolume 6, Issue 2 p. 135-146 ORIGINAL RESEARCH PAPEROpen Access Content-based image retrieval using Gaussian–Hermite moments and firefly and grey wolf optimization Yasasvy Tadepalli, Corresponding Author Yasasvy Tadepalli tyasasvy@gmail.com orcid.org/0000-0002-7684-5427 Department of ECE, GRIET, Hyderabad, India Correspondence T Yasasvy, Department of ECE, GRIET, Batchupally, Hyderabad, India. Email: tyasasvy@gmail.comSearch for more papers by this authorMeenakshi Kollati, Meenakshi Kollati Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorSwaraja Kuraparthi, Swaraja Kuraparthi Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorPadmavathi Kora, Padmavathi Kora Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorAnil Kumar Budati, Anil Kumar Budati Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorLakshmi Kala Pampana, Lakshmi Kala Pampana Department of ECE, VNR Vignana Jyothi Institute of Engineering and Technology, Hyderabad, IndiaSearch for more papers by this author Yasasvy Tadepalli, Corresponding Author Yasasvy Tadepalli tyasasvy@gmail.com orcid.org/0000-0002-7684-5427 Department of ECE, GRIET, Hyderabad, India Correspondence T Yasasvy, Department of ECE, GRIET, Batchupally, Hyderabad, India. Email: tyasasvy@gmail.comSearch for more papers by this authorMeenakshi Kollati, Meenakshi Kollati Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorSwaraja Kuraparthi, Swaraja Kuraparthi Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorPadmavathi Kora, Padmavathi Kora Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorAnil Kumar Budati, Anil Kumar Budati Department of ECE, GRIET, Hyderabad, IndiaSearch for more papers by this authorLakshmi Kala Pampana, Lakshmi Kala Pampana Department of ECE, VNR Vignana Jyothi Institute of Engineering and Technology, Hyderabad, IndiaSearch for more papers by this author First published: 22 April 2021 https://doi.org/10.1049/cit2.12040Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Rapid growth in the transfer of multimedia information over the Internet requires algorithms to retrieve a queried image from large image database repositories. The proposed content-based image retrieval (CBIR) uses Gaussian–Hermite moments as the low-level features. Later these features are compressed with principal component analysis. The compressed feature set is multiplied with the weight matrix array, which has the same size as the feature vector. Hybrid firefly and grey wolf optimization (FAGWO) is used to prevent the premature convergence of optimization in the firefly algorithm. The retrieval of images in CBIR is carried out in an OpenCV python environment with K-nearest neighbours and random forest algorithm classifiers. The fitness function for FAGWO is the accuracy of the classifier. The FAGWO algorithm derives the optimum weights from a randomly generated initial population. When these optimized weights are applied, the proposed algorithm shows better precision/recall and efficiency than other techniques such as exact legendre moments, Region-based image retrieval, K-means clustering ​and Color descriptor wavelet-based texture descriptor retrieval technique. In terms of optimization, hybrid FAGWO outperformed various optimization techniques (when used alone) like Particle Swarm Optmization, Genetic Algorithm, Grey-Wolf Optimization and FireFly algorithm. 1 INTRODUCTION Digital image acquisition in recent years has increased exponentially from rapid advances in internet and cellular communication, a steady decline in the price of memory, and increased access to technology for everyone. In various applications such as medicine, advertising, education, and mass communication, there is a need for the efficient search and retrieval of images similar to the queried image. These aspects motivated the development of content-based image retrieval (CBIR) to retrieve images relevant to the query [1]. In CBIR, the features are collected from an image database using preprocessing and feature-extraction steps. The objective of image preprocessing is to suppress unwanted noise and improve image quality in subsequent steps. Preprocessing steps include denoising [2], Gaussian low-pass filtering (GLPF) [3], and anisotropic distortion [4]. The features of the queried image are compared with the features of images in the database using a similarity measure. The motivation of CBIR is to retrieve images similar to the image queried. However, when images in the database vary in their translation, rotation, shape, size, or image-related features, efficient image retrieval becomes difficult. Moments-based feature extraction is carried out for CBIR. Moments have already been used in the literature for edge detection [5], object recognition [6], facial recognition [7], and other realms of image processing and computer vision. Moments are scalar quantities used to characterize a function and derive its features [8]. Directly speaking, they are projections onto a polynomial base. There are two types of moments, namely geometrical and orthogonal moments, that differ in terms of stability and computational capabilities, especially in the discrete domain. Powers are used in the computation of geometrical moments, and for small and large values of the exponent, computed values of the moment increase rapidly as order increases. On the other hand, in orthogonal moments, features are captured with more precision due to the representation of moments in recurrent relation when compared with the standard powers used in geometrical moments. Orthogonal Gaussian–Hermite moments (GHMs) are used as features in the proposed algorithm. The orthogonal moment base functions of different orders exhibit different numbers of zero crossings and very different shapes [9]. Hence, orthogonal moments represent better distinct image features based on different modes. These discriminative features are useful in computer vision tasks. There are several orthogonal moments, such as exact Legendre moments and Tchebichef moments. Compared with these two moments, GHMs are powerful and it is mainly due to the zero-crossings function of the GHMs which are distributed more evenly than those of other orthogonal moments such as Legendre and discrete Tchebichef moments [10]. GHMs are more immune to noise and avoid the artifacts introduced by the discontinuity window function. In addition, GHMs are translation and rotation invariant. These factors motivated us to use GHMs in the proposed CBIR framework. The proposed CBIR algorithm's retrieval performance is examined by conducting experimentation on the Corel image database, which contains 10 different categories, as shown in Figure 1. The next step is the feature compression technique using principal component analysis (PCA) [11]. This step removes irrelevant information, and thus only discriminant features would be obtained for classification. Discriminant GHM features are obtained with PCA. The compressed GHM features are multiplied by weights to obtain control for improving retrieval efficiency. The optimization algorithms seek to find the best available solution in a given search space [12]. Reference [13] developed a CBIR framework using exact Legendre Moments, HVS color quantization with DC coefficients and statistical properties such as variance, mean, and skew of the conjugate symmetric sequency complex Hadamard transform. The authors used two optimization techniques, differential evolution and the firefly algorithm (FA), for feature compression. FA was used to derive optimal weights from the random initial population. The algorithm's major drawback was a cascade of optimization algorithms used for feature compression and feature retrieval that drastically increased computational complexity. In CBIR evaluation, the best available solution is to obtain improved retrieval efficiency. The CBIR system's retrieval efficiency is improved by applying optimal weights to the proposed algorithm's feature vectors [13]. These optimal weights are generated from a random initial population of hybrid combinations of two popular nature-inspired optimization algorithms, FA [14, 15] and grey wolf optimization (GWO) [16], known in combination as FAGWO. Non-optimized algorithm performance is low in the case of multimodel databases like Corel 1K. Optimum feature selection increases retrieval efficiency and is achieved with the optimization algorithm. FA has the disadvantage of becoming stuck at local maxima. GWO and FireFly (FF) algorithm hybridization ensures optimal weight selection and maintains the trade-off between the degree of exploitation and exploration in the search space. The exploration and exploitation of the optimization are well balanced in the proposed hybrid FAGWO algorithm. In Section 2, prior works on CBIR, GHM, FF, and GWO are put forward. In Section 3, a concise presentation of orthogonal GHMs and their approximation computation is given. The algorithmic steps of the FAGWO algorithm are described in Section 3, and process implementation is discussed in Section 4. Experimental results are presented in Section 5, and finally, the conclusion is given in Section 6. FIGURE 1Open in figure viewerPowerPoint Different Corel 1K dataset objects 2 LITERATURE REVIEW CBIR finds its applications in medical image analysis, remote sensing, surveillance, security, textile industries etc. Several techniques already exist in the literature for CBIR. Through virtual posing, IBM has developed query by image content [17]. The queries are refined based on various image properties such as color, texture, user-constructed sketches, and drawings [17]. Feature vectors formed from geometric shapes, colors, and textures are used to distinguish one class of objects from another in CBIR. Reference [18] proposed a CBIR framework based on color moments considering the mean, variance, and skewness of three color planes. The color distributions were later classified using quantized histogram statistical texture features combined with distance metrics such as Euclidean and Mahalanobis distances. However, this method is highly granular, as it carries out a detailed comparison of histogram bins of queries and images in the database. Another drawback is that histogram bins are highly susceptible to noise. Computational complexity is another limitation of this method. Texture variations in different images can distinguish image features, especially when images contain different textured regions. The texture features obtained from discrete wavelet transform combined with those for color are used to classify textures [19]. Although this scheme works well on small texture databases, it cannot support geometric invariance. Reference [20] utilized color histogram and texture measures to obtained a CBIR system. Texture features included entropy, smoothness and uniformity. The experiment was performed on a Corel database with Euclidean distance measurement as the retrieval mechanism. Reference [21] presented a CBIR system with color histogram, color moment, co-occurrence matrices, and wavelet moments. A K-means clustering algorithm is employed in the CBIR to form the initial clusters. The fitness function, which was optimized by the particle swarm optimization (PSO) technique, minimized the distance from a cluster point to the cluster centre to improve retrieval efficiency. PSO-based CBIR schemes provide optimum results, yet they suffer from high computational time. Reference [22] incorporates a CBIR framework exploiting interactive genetic algorithms (IGAs) and support vector machine (SVM) to learn the user perception. In this scheme, a SVM classifier is trained using the user feedback. The unlabelled images which are classified as most relevant by the SVM classifier, are added to the IGA training set. However, the main drawback of CBIR systems employing genetic algorithm (GA) optimization is that they suffer from convergence problems. that is, the problem of degeneracy. The famous GWO technique is used in many classification algorithms. GWO was combined with a bag of features for facial recognition [23]. Facial features were obtained using the Speeded up Robust Features descriptor. Optimal features were selected by K-means clustering combined with GWO. It has been proved that GWO produces better convergence precision for standard benchmark problems than other meta-heuristic techniques. Another good optimization technique is FA. Yang first developed FA in 2007 [24]. Reference [15] proposed an adaptive CBIR framework exploiting relevance feedback and Gaussian FA. A firefly alone has a high probability of being trapped in local maxima. The FA uses the principles of attractiveness, randomization, and absorption. Random walk is initialized for each firefly and the Levy flight function propels the fireflies' movement in isometric random directions. Gaussian distribution was used for random walk propagation in CBIR [25]. It has been demonstrated that the FF with clustering algorithm outperformed the GA and PSO. Deep neural networks made inroads in CBIR. Series of convolution layers extract the low-level and high-level features from images. High-level features represent the semantics of the data, which in turn provide greater robustness to intra-class variability. Low-level hallmarks are rich in texture and other features that are unique to each image dataset. Low-level features of CNN require a massive amount of dataset with minor variations within each dataset. Hence, careful selection of the dataset images [26] is needed because mis-classification might occur. In addition, CNNs are computationally intensive and require resource-intensive hardware to run. Moreover, the techniques used for extracting the information from an image are referred to as image analysis [27]. Autonomous segmentation is a very critical step in image analysis. The eventual success or failure of image analysis depends on segmentation. The strategies used in image segmentation are based on two attributes of grey values, discontinuity and similarity. The principal focus in similarity is thresholding, and in most thresholding methods, ground truth image is analyzed by human subjects based on visual inspection. This method is error-prone, as it relies on human observer scores. Reference [28] rectified this by proposing a novel approach to automatically generate ground truth for image binarization by consensus of different thresholding methods. The authors concluded that F-measure, modified Hausdorff distance, and edge mismatch error are non-redundant and distinct indicators and are useful in evaluating the performance of the algorithm on image databases. Very recently, CBIR has been effectively used in material identification and medical imagery. The features used in the material database are grey level co-occurrence matrix (GLCM) and artificial bee colony (ABC) [29]. The texture features were extracted using GLCM and ABC and were used for classification and retrieval of the images. Histopathology image segmentation was carried out using region extraction and curve fitting to extract the nuclei [30]. With tremendous improvements in wireless technology, the transmission of images sent through wireless networks has increased to a huge extent. Recently, researchers are seeking to improve the transmission capacity of wireless channels. The analog/digital communication system consists of an encoder, transmitter, channel, receiver, and decoder. Usually, an encoder is any system that incorporates efficient image compression [31]. Feature reduction techniques such as GHMs and other techniques can help in transforming and reducing the size of the image. Orthogonal frequency division multiplexing (OFDM) is the most used channel waveform for image transmission in both analog and digital systems [32]. Discrete cosine transform, wavelet coding, and embedded block coding along with optimal truncation and scaling were used for image compression in medical images [33]. OFDM was used to transmit the image using multicarrier code [34]. Image transmission happens in packets, but as the image's size increases, more energy is required to transmit the images. Energy is depleted when the image is transmitted. To prevent this, an image transmission scheme using a multihop was proposed based on LEACH. This technique eliminates the energy hole problem when the cluster head node compresses the images [35, 36]. 3-D image processing is gaining significance, as much information is captured in 3-D images. A patch map was generated and used for error detection on the depth map [37]. This patch map not only reduced the transmission error but also enhanced the quality of the 3-D image. Much of the information can be reduced if an effective feature vector is generated from the images. Shape is an important low-level feature of an image that helps in identifying significant image regions. Without the help of visual cues, shapes can recognize variations in the image effectively [38]. Contour-based and region-based shape descriptors are the two important types of shape descriptors. 'Contour' deals with only the boundary, whereas in a region-based descriptor, all the information related to the texture and shape is obtained from the image [39]. Fourier descriptors, polygonal approximations, curvature scale space, and shape signatures are examples of contour-based shape descriptors. Moment invariants form a part of region-based descriptors [40]. 'Invariant' features are insensitive to the various deformations in the image. Deformations are rotation variance, scale variance etc. If the polynomial base is orthogonal, the moments are termed orthogonal moments. Examples of orthogonal moments are Zernike moments (ZMs), exact Legendre moments (ELMs) and GHMs [40]. ZMs are effective in shape recognition but are very computationally intensive. The coefficients of ZM are mapped to a unit circle, which is a limitation while retrieving greyscale images. When the pixels that fall outside the unit circle are omitted, an error is introduced [40, 41]. This motivated many researchers to experiment on ELMs and GHMs. A model for computing ELMs is proposed, and it eliminates the need for numerical approximation. A fast and accurate method improved the performance of ELMs. It was [9] who reported that ELMs could represent an image with a minimum amount of information redundancy. ELMs are invariant to translation and scaling. They show optimum performance in terms of retrieval efficiency and computational intensity. The main drawback of the ELMs is rotation sensitivity. GHMs [41] blend the advantages of ELMs [9] and ZMs, thus providing computational simplicity and performance richness in terms of Image retrieval. The introduction of GHMs and their application is first found in [9]. Later on, the moments were used in fingerprint recognition and classification [43], 3-D face recognition [44], SAR image segmentation [45], and the intelligent license plate recognition system [46]. Drawing motivation from the performance of the GHMs, CBIR using GHMs is taken up in this paper. It is postulated that higher-order Gaussian moments in discrete and continuous implementations are good at global feature representation and image reconstruction [9]. Chandra Mohan et al. [47] demonstrated a fast way to retrieve the Images using GHMs and SVM. In this paper, random forest (RF) and K-nearest neighbours (KNNs) were used as classifiers. RF is intrinsically suited for multiclass problems consisting of a mixture of numerical and categorical features. Less parameterization makes it easy to program and the nature of RF makes them less likely to overfit. KNN is used in this project because many project classes have less linear separability. The KNN algorithm assumes that similar things exist close. This proximity is calculated using the distance metrics. The use of KNN has been both boon and bane in this paper. Without proper optimization, the KNN technique was found to overfit the data points. The same was the case with RF. The Corel database contains images with different backgrounds, different numbers of objects, and different orientations. Non-optimization algorithms solve such multimodal problems poorly. The optimization algorithm must reach the global maxima in search space to increase retrieval efficiency. An optimizer helps to determine the global maxima by selecting the best weight matrix for the feature vector. FF has the disadvantage of becoming struck at local maxima. GWO has better exploration and exploitation tendencies. Hence, a hybrid optimizer combining the FA and GWO technique, FAGWO, is used in the proposed CBIR framework. 3 PRELIMINARIES In this section, Gaussian–Hermite moments, classification algorithms, namely, KNN and RF, and the optimization algorithm, namely, FAGWO, are discussed. 3.1 Gaussian–Hermite moments Gaussian–Hermite moments are a set of moments formed using Gaussian–Hermite polynomials as basis functions. The Hermite and Gaussian polynomials are described as follows: The Hermite polynomials of ith order moment are defined as H i x = − 1 i e x 2 d i d x i e − x 2 (1) The relation of orthogonality is ∫ − ∞ ∞ H i x H j x e − x 2 d x = δ i , j i ! 2 i π (2) δi,j is known as knocker's delta. The orthogonality condition simplifies the reconstruction of the original function from the generated moments. The main reason of introducing orthogonality in Gaussian–Hermite polynomials are stable and fast numerical implementation, avoidance of high dynamic range of moment values that may lead to loss of precision due to overflow or underflow, and a higher robustness to random noise. The Hermite polynomials are orthogonal in the interval of − ∞ , ∞ [8] and therefore not well suited to image-processing tasks. To suit the moments for image processing applications, Wu and Shen [34] proposed orthogonal GHMs G H M i j x , y , f x , y as given in Equation (3): ∫ − ∞ ∞ ∫ − ∞ ∞ g u , v , σ H i u σ H i v σ d u d v (3)that depend on image coordinates (x,y). The g u , v , σ , a Gaussian function, is defined as g u , v , σ = 1 2 π σ 2 e − u 2 + v 2 2 σ 2 (4)where σ is its standard deviation. Hermite polynomials are computed quickly using recurrent relation used in Equation (5): H i + 1 x = 2 x H i x − 2 i H i − 1 x , f o r i > = 1 (5) Generally, GHMs are defined in the continuous domain. In the discrete domain, GHMs are fast and stable due to computation with recurrent relations. High dynamic range in moment values is prevented by confining GHMs to a narrow interval of − 1 , 1 . The following equation is a convenient method for discrete implementation of the moment computation: η i j = 4 K − 1 2 ∑ p = 0 K − 1 ∑ q = 0 K − 1 I p , q H ˆ i p , K ; σ H ˆ j q , K ; σ (6) The number of GHM features obtained for order n is n2. These features are compressed to z features using PCA. For z features, z weight matrices are obtained. The weighed features will be used to train machine learning algorithms for classification. KNN and RF classifiers are used in this paper. 3.2 K-nearest neighbours algorithm KNN [48] is a supervised machine learning technique that classifies objects based on the closest set of points in a vector space containing features. KNN is termed 'lazy learning' where the function works only locally and the computation is carried out until the classification procedure is completed. A simple technique, KNN is widely used when the spatial distribution of the data is unknown. Each data point goes into one of the K clusters by taking the nearest neighbour and it is denoted by factor 'K'. Efficiency in classification is improved when the optimum 'K' value is chosen. 3.3 Random forest classifier RF [49] employs an ensemble method for classification and regression. Classification is carried out in RF by constructing a multitude of trees at the training phase, resulting in a class with the majority vote. In RF, multiple trees are grown and discrimination between the classes is achieved by projecting the training data on a stochastically chosen vector subspace before fitting the model at each tree or node, as shown in Figure 2. The decision at each node is not selected deterministically but randomly. Bagging provides flexibility for each model in the ensemble to give each tree equal weights. To promote model variance, bagging trains each model in the ensemble using a randomly drawn subset of the training set. Therefore, the RF algorithm combines random decision trees with bagging to achieve very high classification efficiency. The RF produces classification results comparable to other classifiers and has the advantages of not overfitting, withstanding more noise and ease of measuring out-of-bag error. FIGURE 2Open in figure viewerPowerPoint Random forest classifier with majority vote 3.4 Grey wolf optimization GWO [16, 23] is a meta-heuristic and biologically inspired technique inspired by wolves' hunting behavior, as they move and hunt in packs. Packs hunt for potential prey, and once they find it, they encircle and attack it. A leader would be present in the pack and all the other wolves in the lower rung of hierarchy obey the leader's instructions. The same principle is applied in the GWO technique. There are four types of variables designated as four types of wolves which are alpha (α), beta (β), delta (δ) and omega (ω). The hunting is guided by α, β, δ respectively, with the α, as the leader. ω follows them in the hunting process. The three main hunting steps, namely searching, encircling, and attacking prey, are implemented in GWO using the following mathematical equations: D = C x p t − x t (7) x t + 1 = x p t − A D (8)where xp is the position vector of the prey and x indicates the grey wolf's position vector and t is the current iteration. A and C are coefficient vectors, A and C vectors can be computed using Equations (9) and (10): A = 2 a r 1 − a (9) C = 2 r 2 (10)where components of a are linearly decreased from 2 to 0, throughout iterations, r1 and r2 are random vectors in [0,1]. α is the best solution. β and δ are the next best solutions. These solutions are saved and other agents are forced to update their position according to the best position. To update the grey wolves position, the following equations are used: D α = C 1 X α − X D β = C 2 X β − X D γ = C 3 X γ − X (11)where C1, C2 and C3 are computed using Equation (10): X 1 = X α − A 1 D α X 2 = X β − A 2 D β X 3 = X γ − A 3 D γ (12) x t + 1 = X 1 + X 2 + X 3 3 (13)where Xα, Xβ, and Xγ are the three best solutions at iteration t, A1, A2, and A3 computed using Equation (9), Dα, Dβ, and Dγ defined in Equation (11), and X(t + 1) indicates the position of wolves in the next iteration. The hunting process is ceased when the prey halts moving and wolves jolt an attack and can be done mathematically by decreasing the value of a during iterations for controlling exploration and exploitation of GWO. 3.5 Firefly and grey wolf optimization The firefly and grey wolf optimization (FAGWO) hybrid optimization algorithm integrates GWO into FA, one of the nature-inspired swarm intelligence-based optimization algorithms that rely on fireflies' flashing light. The FA computes the optimum position of the particles based on the brightness. The fireflies are unisexual and therefore, any firefly can attract another firefly, disregarding their sex. The firefly's attractiveness is directly proportional to one's intensity and attraction between them curtails as the distance increases. In a firefly pair, the brighter one attracts the firefly with lesser brightness and the intensity falls when their mutual distance increases. The firefly's movement is unpredictable and random if no firefly is found brighter than a given firefly. Two vital issues considered in the FA are the variation of light intensity and formulation of the attractiveness. The FA assumes that a firefly's attractiveness depends on brightness associated with the encoded objective function. The light intensity at a certain distance r from the light source obeys the inverse square law as given in Equation (14): I ∝ 1 / r 2 (14) Furthermore, as air absorbs light, the light intensity becomes feeble as the distance r increases. These two combined factors make most fireflies constraint to a limited distance, usually, several hundred meters at night, which is generally good enough for fireflies to communica