# Noise Resilient Local Gradient Orientation for Content-Based Image Retrieval

Authors: Samina Bilquees; Hassan Dawood; Hussain Dawood; Nadeem Majeed; Ali Javed; Muhammad Tariq Mahmood
Journal: International Journal of Optics
 (2021)

---

## Abstract

In a world of multimedia information, where users seek accurate results against search query and demand relevant multimedia content retrieval, developing an accurate content-based image retrieval (CBIR) system is difficult due to the presence of noise in the image. The performance of the CBIR system is impaired by this noise. To estimate the distance between the query and database images, CBIR systems use image feature representation. The noise or artifacts present within the visual data might confuse the CBIR when retrieving relevant results. Therefore, we propose Noise Resilient Local Gradient Orientation (NRLGO) feature representation that overcomes the noise factor within the visual information and strengthens the CBIR to retrieve accurate and relevant results. The proposed NRLGO consists of three steps: estimation and removal of noise to protect the local visual structure; extraction of color, texture, and local contrast features; and, at the end, generation of microstructure for visual representation. The Manhattan distance between the query image and the database image is used to measure their similarity. The proposed technique was tested using the Corel dataset, which contains 10000 images from 100 different categories. The outcomes of the experiment signify that the proposed NRLGO has higher retrieval performance in comparison with state-of-the-art techniques.


---

## Body

## 1. Introduction

The advent of multimedia tools has made it easier to access a wider range of images with a variety of information. CBIR (content-based image retrieval) uses low-level image properties to search a huge database for images that suit the user’s demands. Color, edges, and orientation are considered low-level image attributes [1]. CBIR is widely used in different applications like medical imaging [2], E-commerce [3], and digital libraries [4].Text-based image retrieval (TBIR) and sketch-based image retrieval are two image retrieval methodologies. TBIR search images on the basis of labels and keywords. It is mostly used in Google Images. Therefore, it become difficult to describe whole image content in words and it may show irrelevant content. Manual annotation becomes difficult when there is large database [5]. A sketch-based image retrieval system is used to find images on the basis of sketch content drawn by user [6]. Soft Histogram of Edge Local Orientation (S-HELO) uses local orientation computation for improving the performance of sketch-based image retrieval [7]. The proposed histogram of line relationship (HLR) solves the appearance problem between sketches and images. It removes the noisy edges by choosing suitable edge shape that best corresponds to object boundaries [8]. Manual annotations of text-based and user-based sketch techniques cause incompatible image retrieval. Xie et al. proposed analogy relevance feedback method based on the user intentions that is helpful in maximizing retrieval results [9]. Case Based Long Term Learning (CB-LTL) method of relevance feedback [10] incorporates user intentions and increases retrieval performance. Mandal et al. proposed signature-based bag of visual words (S-BoVW) that employs jumble of words with image’s texture and color to retrieve similar images [11]. BoVW based methods improve the retrieval performance but their computation is expensive.In this paper, we propose Noise Resilient Local Gradient Orientation (NRLGO) that overcomes the noise factor within the visual information and strengthens the CBIR to retrieve accurate and relevant results. In our proposed NRLGO, the color, texture, and local contrast information were used for feature representation of visual data. While semantic attributes are estimated through calculating the correlation between visual features and regular structure data. NRLGO has noise resilience characteristics and saves the local structures of visual information from noise by finding an uncertain state. The major contribution of NRLGO is summarized as follows:(1)
The NRLGO holds a noise resilient attribute by finding the uncertain bit within the visual information which is significant to increase the CBIR system’s performance.(2)
NRLGO computes pixel value and difference and then multiplies them by the same constant value. In this way, the bright changes will be canceled and the discriminative feature representation is improved.(3)
Multiresolution gradient orientations of NRLGO improve the gradient magnitude by removing noise factor and detecting the edges.(4)
NRLGO estimates the gradient orientation through dividing theθ value into T dominant orientations. Thus, gradient orientation strengthens the visual feature representation of image.(1)arctan2us11,us10=θ,us11>0andus10>0,π−θ,us11>0andus10<0,θ−π,us11<0andus10<0,−θ,us11<0andus10>0.The remainder of the paper is structured as follows: The related work is given in Section2. The suggested NRLGO is presented in Section 3 which discusses the attributes of NRLGO against noise, illumination variation, and rotation variation. Section 4 summarizes the experimental findings and our contribution against state-of-the-art techniques. In Section 5, discussion is given, and Section 6 concludes our contribution.


## 2. Related Work

The CBIR system plays a vital role in retrieving images that are related from large dataset that are semantically and contextually similar to search image. The CBIR systems extract visual feature representation and estimate the similarity distance for scoring the images that are comparable from the visual database. In the CBIR systems, visual feature descriptors were used in two ways: local and global descriptors. The local descriptors extract image’s local interest points while global descriptors take the complete image for feature extraction [12]. Different CBIR methods are used to achieve efficient retrieval performance. The scale invariant feature transform (SIFT) [13] was introduced as a local descriptor, and it is invariant to scale and rotation. To deal with the difficulty of image retrieval using a high-dimensional feature vector, the PCA-SIFT [14] was used. The Hessian matrix and distance ratio were used to solve the computational issue of speeded up robust features (SURF) [15]. The local binary pattern (LBP) based descriptor was proposed by [16]; however, LBP is applicable for gray-scale images only. Moreover, LBP does not perform accurate prediction in presence of noise. This limitation of LBP was overcome by multichannel decoded LBP [17]. The multichannel decoded LBP [17] performs computation on the color and gray-scale images. To further improve the performance of LBP, the fusion of color histogram with LBP was applied to achieve better retrieval performance for color images [18]. To minimize noise in regular patterns, the local ternary pattern (LTP) [19] was developed. The LTP extracts the spatial features in three directions while LBP extract textual features in two directions. The extended local ternary pattern (ELTP) [20] is robust and overcomes the noise within the images. The scale invariant local binary pattern (SILTP) [21] was used to extract local features for gray-scale images and estimate the pixel difference for complex background scenes. Edge detection Sobel filter calculates orientation of edges but has low signal to noise ratio [22]. Edges obtained from Sobel filter have large thickness which gives mismatched images. Canny filter [23] solved the issue of thickness of edges. It has good signal to noise ratio.Color and image edges are sensitive to the human visual system. In HSV color space, CDH is utilized to improve human visual perception. Entropy is used for feature selection, and correlation is developed between the features after feature selection [24]. The use of visual feature discrimination to measure the similarity between the image passed by the user and images in the database is proposed in an approach based on weight learner [25]. Spatial pyramid matching (SPM) is proposed for spatial distribution of images that increased the retrieval accuracy, but when image alignment is not done properly, it results in rotation and translation variance. Geometric relation is established on the basis of center of image among a group of similar words [26]. Color feature is fused with texture feature to achieve highest retrieval performance as a single attribute is not more resilient for obtaining efficient retrieval performance. As a result, for color extraction, color moment is used, and Gabor descriptor is used to extract the texture feature. After that, features are represented using the Color and Edge Directivity Descriptor (CEDD) [27]. The images are first transformed to RGB color space. Texture information is obtained with the help of evaluated local binary pattern and by predefined prepattern unit. This texture information is then fused with color channels to increase the retrieval performance [28]. Feature extraction [29] is described by fusing top-hat transforms and local binary patterns for other image processing cases to analyze the color images. Top-hat transform is used to extract the shape, and color local binary pattern is used for texture classification.In [30], extraction of color feature is done by color difference histogram, and Gabor descriptor is used for the texture classification. Joint property of color with the texture increases the retrieval accuracy. Rotation invariant and gray-scale invariant property is achieved by using the texture feature with color feature. Another technique that uses the color and texture features is developed for image retrieval. HSV and Lab color space are used to extract the color attribute, and texture classification is done by using Haralick features in RGB and gray-scale images [31]. Plant disease [32] is identified by using color, and texture classification is done by using gray-level cooccurrence matrix (GLCM).CBIR systems are used to represent the visual information in global context based on different deep learning techniques. Bilinear convolutional neural network was proposed for efficient retrieval performance that consists of CNN architecture with bilinear root pooling [33]. Similarly, another deep learning method for CBIR which extracts global features through two parallel CNN models was presented. In [34] SPoC global descriptor was developed through aggregation of deep learning local features for CBIR. The cross-dimensional CNN weights were aggregated to represent the global information for CBIR [35]. The triplet network [36] was used for optimizing the ReLU max aggregation convolutions (R-MAC). The R-MAC involves the pooling of regions of image to cover the point of interest. Moreover, the deep CNN model [37] was used to compress the image descriptors and activate the layers of CNN. The multilayered CNN was used for features extraction, and features were encoded with VLAD encoding scheme [38]. The CNN based CBIR systems require large amount of data for model learning and robust machines for computation of loss and hyperparameters, while local features and distance based CBIR approaches require no clustering and are computationally less expensive than deep learning methods.The CBIR systems have been studied for decades. Still, occlusion, cluttered background, viewpoint variation, and noise make the retrieval process a challenging task. The literature reveals that all the existing methods for CBIR systems were established using clear visual information, and little effort was made to remove the noise within the visual information. However, presence of noise in visual information is responsible for degraded performance of local or global descriptors (deep learning methods). This is due to the fact that the visual information might be occluded with noise, or maliciously designed data was introduced within the visual information to deceive the CBIR models. Therefore, noise resilient local descriptor is required to overcome the noise or artifacts within the images for CBIR.


## 3. Noise Resilient Local Gradient Orientation

The CBIR systems compute the discriminative low-level attribute of image including color, edges, and patterns. The distribution of pixels within the visual information is uniformed and represents the discriminative attributes of visual information like color, texture, and patterns. The uniform distribution of pixels is sensitive to small uncertain changes and these small uncertain changes disturb the uniform pixel distribution and cause nonuniform patterns within the image. This uncertain noise confuses the CBIR models to retrieve relevant information. Therefore, we proposed Noise Resilient Local Gradient Orientation (NRLGO) to overcome the noise patterns and sustain the uniform pattern of the pixels. The proposed NRLGO consists of three steps: estimation of noise and protection of the local visual structure; then low-level feature extraction (color, texture, and contrast); and generation of microstructures, as illustrated in Figure1. The nonuniform noise signal was estimated, and resilient model was established through error correction method. Then, discriminative color and texture features were estimated through quantizing HSV color space and local gradient orientation (LGO). Moreover, NRLBP exacts contrast information through quantizing V values of HSV. The Manhattan distance between the search image and the database image is used to determine their similarity. The proposed NRLGO method is completely described in Figure 2.Figure 1
NRLBP encoding scheme in addition to comparison of LBP and LTP. NRLBP uniform code is formed by finding the uncertain bitY at m = 1, and s is chosen as 2 for LTB and NRLBP.Figure 2
Visual representation of proposed NRLGO descriptor for CBIR system. The proposed NRLGO consists of three steps: estimation and removal of noise to protect the local visual structure; then the color and texture feature extraction; and, at the end, microstructure generation.


### 3.1. Noise Resilient Local Binary Pattern

The proposed NRLBP establishes a noise resilient attribute through computing the small pixel difference and estimating the uncertain bitY and uncertain state. The uncertain bit Y and uncertain state were estimated on the basis of corrected noise-free bits within the image. The intensity difference xp is calculated between center pixel and its corresponding neighboring pixel through the three-state fuzzy logic’s Bp−1N representation, and s is threshold which is chosen as 2 for NRLBP [39].(2)Bp−1N=1,ifxp≥s,U,ifxp<s,0,ifxp≤−s.The noise factor is considered as uncertain bit and occurs either from0 to 1 or from 1 to 0. The uncertain bit Y=y1,y2,…,yn represents the feature vector where yi,yi∈0,1 is formed by variablen. The uncertain state UY can be mathematically represented by the following equation:(3)Bp−1NBp−2N,…………,B1NB0N=UY.The regular visual patterns involve edges, edge-ends, corners, etc. The regular patterns exist more frequently than irregular patterns within the images. As regular expression for normal visual patterns exists, it is possible to estimate the regular expression of uncertain stateU from the regular visual patterns present within the image.NRLBP corrects the irregular patterns to regular patterns by removing noise on the basis of error correction mechanism. For example, the image shown in Figure1 holds an uncertain code 11Y100Y0. In order to remove the noise, there is a need to predict the uncertain bit and then modify the uncertain bit to form uniform patterns, for example, 11110000 and11111000. Suppose that φr represents the set of all regular LBP patterns. For instance, the image consists of 58 regular patterns. On the basis of uncertain state UY, a list of NRLBP codes is provided as follows:(4)RNRLBP=UY|y∈0,1t,UY∈φr.The uniform code of11110000 is obtained after error correction, and noise was removed. After that, the histogram of NRLBP for a local region of image was formed, and the number of elements in RNRLBP is represented by n. The bin of histogram was representing each element of RNRLBP which was added by 1/n when t>0. When t=0, the irregular bin was added by 1. The same process was repeated for each pixel in the region. Furthermore, NRLBP has reduced histogram bins from 118to59 bins. The feature vector of the local image region can be generated by summing the feature vectors collected from each pixel in the image region.


### 3.2. Feature Extraction

Feature extraction includes the extraction of color, texture, and local contrast information. Converting RGB to HSV color space extracts the color characteristic. HSV (high-saturation value) color space is quantized into72 colors with, 8, 3, and 3 bins. Texture classification is done by using multiresolution gradient orientation. It improves the gradient magnitude by removing noise factor and detecting the edges. The V value of the HSV color space is used to acquire local contrast information.


#### 3.2.1. Color Feature

The chrominance signals of visual information hold significant discriminative information to represent the images in CBIR models. To reflect the chrominance attribute of visual information, different color spaces are used like YCbCr, HSV, HIS, Lab. The RGB color model is made up of three components: red, green, and blue. In proposed method, HSV color space was selected to segregate the hue, saturation, and intensity of image chrominance.In addition, the HSV color model represents the chrominance attribute of image, and it has cylindrical representation with hue (H), saturation (S), and value (V). The hue component of the HSV color space describes the wavelength of colors ranging from 0 to 360, beginning with red at 0, yellow at 60, green at 120, cyan at 180, blue at 240, and magenta at 300. The saturation component describes the chrominance saturation level between 0 and 1, where 0 denotes gray and 1 denotes primary color. The value V component describes the intensity values between 0 and 1, where 1 reflects black and 1 reflects white. The difference between HSV and RGB is that the HSV separates the intensity values from the color element. The RGB system is based on three color elements: red, green, and blue.The HSV color space is color invariant because the V component of HSV represents the image’s brightness and is independent of color [24]. With 8, 3, and 3 bins, the HSV color space is quantized into 72 colors. The quantization is essential in persevering images from light and intensity and also reduces the time complexity.(5)qh=h×bhmaxh,qs=s×bsmaxs,qv=v×bvmaxv,where bh, bs, and bv represent the quantized bins of hue, saturation, and value component of HSV [12].(6)cm=qh×bs×bv+qs×bv+qv.The quantized values for hue (h), saturation (s), and value (v) of HSV in color map cm are qh, qs, and qv. nc is the number of quantized colors. s and t are spatial coordinates in quantized color map qcm.(7)qcmj=s,t|s,t∈cm=j,0≤j≤nc−1.


#### 3.2.2. Local Gradient Orientation (LGO)

For gradient computation, the orientation component of Weber local descriptor (WLD) was applied to HSV color model. This is due to the fact that by using the gray-scale images majority of chrominance information is lost. Therefore, the local gradient orientations were extracted, which holds rotation invariant attribute from chrominance part and is beneficial to obtain discriminative features. The gradient orientation [40] was calculated from the angle between the reference axis and the vector in horizontal and vertical location x.(8)θyc=αs1=arctanut11ut10,f10=−1+1 and f11=−1+1. When filters f10 and f11 are applied to an input image, then outputs ut11 and ut10 are obtained. ut10=y5−y1 and us11=y7−y3.θ is quantized into T gradient orientations. Firstly, mapping is done before quantization f:θ⟶θ′:θ′=arctan2us11,us10+π and(9)arctan2us11,us10=θ,us11>0andus10>0,π−θ,us11>0andus10<0,θ−π,us11<0andus10<0,−θ,us11<0andus10>0,θ∈−π/2,π/2, and θ′∈0,2π. Mapping takes value of θ under consideration.Quantization function is given as follows.φs=frθ′=2q/Qπ, and q=modθ′/2π/T+1/2. If T = 8 then φs=sπ/4,s=0,1,T−1.The multiscale gradient orientation is useful for extracting gradient orientation to describe different granular aspects of visual representation. Multiscale gradient orientation is able to improve the discriminative ability of a single resolution. It is obtained by using square neighbors ofP pixels, considering the length of 2R+1. As illustrated in Figure 3, P denotes the set of neighbors, and R denotes the spatial resolution.Figure 3
Squared symmetric neighborhood for different (P, R). P represents neighbors and R represents the spatial resolution. Multiresolution analysis is done by choosing P as 4, 16, and 24 and R as 1.0, 2.0, and 3.0.
(a)(b)(c)The multiscale gradient orientation was obtained through combining the histograms of different operators at varyingP andR. By varying the P and R, whole image pixel values are considered. A multiscale analysis of WLD orientation can be done using the data generated by several operators of varying scales (P, R). Despite the fact that the operator is based on a squared symmetric neighbor set of P members on a square with side length (2R + 1), it can also be used for a circular situation. In general, it can improve the discrimination of a single resolution (P, R).The WLD orientation is used for texture estimation, and it also eliminates the effect of noise. The WLD orientation reduces the noise and illumination variation by computing the differences between the pixel and its neighbors. WLD orientation is tolerant of the presence of noise in an image. The influence of noise is reduced by using a WLD orientation, which is analogous to smoothing in image processing. Furthermore, the sum of its p-neighbor differences is divided by the current pixel’s intensity, reducing the influence of noise in an image.WLD orientation has also been established to mitigate the impact of changing brightness. It calculates the differences between the center pixel and its neighbors. As a result, a brightness modification that adds a constant to each image pixel has no effect on the disparities in values. WLD orientation, on the other hand, is responsible for dividing the differences. Thus, when each pixel value is multiplied by a constant, the differences are also multiplied by the same constant, canceling out the contrast change. As a result, the description is not affected by changes in brightness.Finally, quantized orientation map incorporating multiscale granularity is represented as follows:(10)qos=12omg×No180,qosf=s,t|s,t∈qos=j,0≤No=1,where s and t are spatial coordinates of orientation structure and j is the resultant value obtained across quantized value No=6.


#### 3.2.3. Local Contrast Information

The pixel intensity is used to describe the local contrast information [41], as the visual information depends on intensity, and specific contrast range improves the visibility of visual information and increases the performance of CBIR systems. The V element of HSV color model was used to extract the intensity feature. This is due to the fact that V component of HSV color model called luma separates the chrominance element of image. The local contrast information map for input image fs,t is calculated as [42](11)lcms,t=∑s=1U∑t=1vmaxps,t,maxHs,t,bs,t.From the mathematical expression represented in (11), the constants s and t are the coordinates of input image fs,t. The dimensions of input image are represented by m and n. Then, the local contrast information was quantized at 10 levels to retain the salient information. The final intensity map was obtained and mathematically represented by the following expression:(12)qlcms,t=IMs,t∗ni,qlcms,tl=s,t|s,t∈qlcms,t=l,0≤l≤ni−1.


### 3.3. Interlinked Microstructure Identification

Natural images contain large color, edge, and shape attribute, which are considered as low-level image attributes. More relevant images are retrieved from large datasets on the basis of color, edge, and shape attribute of image. Human eye shows sensitivity toward color and orientation features. Orientation reflects an informative content in image. Strong orientation gives a uniform pattern. As most natural standard images do not contain strong orientation, there is no uniform pattern. Natural images contain uniform and nonuniform patterns contributing spatial information, forming the microstructures. Orientation is not enough to discuss the image’s spatial characteristics, so color, edge, orientation, and local contrast information are fused to describe the richer image contents. In this paper, microstructures are proposed on the basis of color, orientation, and local contrast information attributes. Color feature and microstructure which is obtained by associating texture and local contrast information are joined to obtain micro-color structure. Orientation attributes and microstructure which is derived by linking color and intensity features are joined to obtain micro-orientation structure. Micro-intensity map is derived by intensity features and micro-map which is derived by linking orientation and intensity features. As microstructures are combination of color, orientation, and intensity features, they will increase the retrieval performance. Quantized orientation and quantized intensity ranges are used to derive micro-color map. The whole orientationOorientation is divided into 3 × 3 small grids [43]. Orientation is scale and rotation invariant, so it is used to derive color map. As orientation is quantized at level 6, it can vary from 0 to 5. For horizontal and vertical location, pixel of 3lengths is defined for each grid. Suppose that G∧ii=Gii+Gii+1+Gii+2+⋯GP are the grids which involve Oorientation where ii=1,2,……P where P represents length of grid. Similarly, suppose that G∧jj=Gjj+Gjj+1+Gjj+2+⋯GQ are the grids which involve Ointensity where jj=1,2,……Q where Q represents length of grid. Suppose that Gii is the grid of length 3×3 moved on quantized orientation and is the grid of length 3×3 moved on quantized local contrast information. Relationship is established between the pixels in the center and the pixels in the surrounding areas as a result of resemblance measures, which are helpful in finding the regular patterns. Suppose that cpo, cpc are the central pixels and ki and kj are neighboring pixels of quantized orientation and intensity microstructures. While moving grid of 3×3, neighbors of quantized orientation and intensity have the same value as central pixels of orientation and intensity; then, uniform or regular patterns and microstructure basic block are obtained. If no values of neighboring pixels of ki and kj are the same as those of the central pixels cpo, cpc, then irregular patterns are formed and microstructure basic block is not obtained. Five steps are followed to achieve a finally single image’s microstructure.(1)
Starting from0,0, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 0,0 and is labeled as M1s,t on the basis of regular patterns in both quantized orientation and intensity where 0≤s≤m−1, 0≤t≤n−1.(2)
At0,1, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 0,1 and is labeled as M2s,t on the basis of regular patterns in both quantized orientation and intensity where 0≤s≤m−1, 1≤t≤n−1.(3)
At1,0, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 1,0 and is labeled as M3s,t on the basis of regular patterns in both quantized orientation and intensity where 1≤s≤m−1, 0≤t≤n−1.(4)
At1,1, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 1,1 and is labeled as M4s,t on the basis of regular patterns in both quantized orientation and intensity where 1≤s≤m−1, 1≤t≤n−1.(5)
Final micro-map of image is obtained and demonstrated asx by combining all four maps.


## 3.1. Noise Resilient Local Binary Pattern

The proposed NRLBP establishes a noise resilient attribute through computing the small pixel difference and estimating the uncertain bitY and uncertain state. The uncertain bit Y and uncertain state were estimated on the basis of corrected noise-free bits within the image. The intensity difference xp is calculated between center pixel and its corresponding neighboring pixel through the three-state fuzzy logic’s Bp−1N representation, and s is threshold which is chosen as 2 for NRLBP [39].(2)Bp−1N=1,ifxp≥s,U,ifxp<s,0,ifxp≤−s.The noise factor is considered as uncertain bit and occurs either from0 to 1 or from 1 to 0. The uncertain bit Y=y1,y2,…,yn represents the feature vector where yi,yi∈0,1 is formed by variablen. The uncertain state UY can be mathematically represented by the following equation:(3)Bp−1NBp−2N,…………,B1NB0N=UY.The regular visual patterns involve edges, edge-ends, corners, etc. The regular patterns exist more frequently than irregular patterns within the images. As regular expression for normal visual patterns exists, it is possible to estimate the regular expression of uncertain stateU from the regular visual patterns present within the image.NRLBP corrects the irregular patterns to regular patterns by removing noise on the basis of error correction mechanism. For example, the image shown in Figure1 holds an uncertain code 11Y100Y0. In order to remove the noise, there is a need to predict the uncertain bit and then modify the uncertain bit to form uniform patterns, for example, 11110000 and11111000. Suppose that φr represents the set of all regular LBP patterns. For instance, the image consists of 58 regular patterns. On the basis of uncertain state UY, a list of NRLBP codes is provided as follows:(4)RNRLBP=UY|y∈0,1t,UY∈φr.The uniform code of11110000 is obtained after error correction, and noise was removed. After that, the histogram of NRLBP for a local region of image was formed, and the number of elements in RNRLBP is represented by n. The bin of histogram was representing each element of RNRLBP which was added by 1/n when t>0. When t=0, the irregular bin was added by 1. The same process was repeated for each pixel in the region. Furthermore, NRLBP has reduced histogram bins from 118to59 bins. The feature vector of the local image region can be generated by summing the feature vectors collected from each pixel in the image region.


## 3.2. Feature Extraction

Feature extraction includes the extraction of color, texture, and local contrast information. Converting RGB to HSV color space extracts the color characteristic. HSV (high-saturation value) color space is quantized into72 colors with, 8, 3, and 3 bins. Texture classification is done by using multiresolution gradient orientation. It improves the gradient magnitude by removing noise factor and detecting the edges. The V value of the HSV color space is used to acquire local contrast information.


### 3.2.1. Color Feature

The chrominance signals of visual information hold significant discriminative information to represent the images in CBIR models. To reflect the chrominance attribute of visual information, different color spaces are used like YCbCr, HSV, HIS, Lab. The RGB color model is made up of three components: red, green, and blue. In proposed method, HSV color space was selected to segregate the hue, saturation, and intensity of image chrominance.In addition, the HSV color model represents the chrominance attribute of image, and it has cylindrical representation with hue (H), saturation (S), and value (V). The hue component of the HSV color space describes the wavelength of colors ranging from 0 to 360, beginning with red at 0, yellow at 60, green at 120, cyan at 180, blue at 240, and magenta at 300. The saturation component describes the chrominance saturation level between 0 and 1, where 0 denotes gray and 1 denotes primary color. The value V component describes the intensity values between 0 and 1, where 1 reflects black and 1 reflects white. The difference between HSV and RGB is that the HSV separates the intensity values from the color element. The RGB system is based on three color elements: red, green, and blue.The HSV color space is color invariant because the V component of HSV represents the image’s brightness and is independent of color [24]. With 8, 3, and 3 bins, the HSV color space is quantized into 72 colors. The quantization is essential in persevering images from light and intensity and also reduces the time complexity.(5)qh=h×bhmaxh,qs=s×bsmaxs,qv=v×bvmaxv,where bh, bs, and bv represent the quantized bins of hue, saturation, and value component of HSV [12].(6)cm=qh×bs×bv+qs×bv+qv.The quantized values for hue (h), saturation (s), and value (v) of HSV in color map cm are qh, qs, and qv. nc is the number of quantized colors. s and t are spatial coordinates in quantized color map qcm.(7)qcmj=s,t|s,t∈cm=j,0≤j≤nc−1.


### 3.2.2. Local Gradient Orientation (LGO)

For gradient computation, the orientation component of Weber local descriptor (WLD) was applied to HSV color model. This is due to the fact that by using the gray-scale images majority of chrominance information is lost. Therefore, the local gradient orientations were extracted, which holds rotation invariant attribute from chrominance part and is beneficial to obtain discriminative features. The gradient orientation [40] was calculated from the angle between the reference axis and the vector in horizontal and vertical location x.(8)θyc=αs1=arctanut11ut10,f10=−1+1 and f11=−1+1. When filters f10 and f11 are applied to an input image, then outputs ut11 and ut10 are obtained. ut10=y5−y1 and us11=y7−y3.θ is quantized into T gradient orientations. Firstly, mapping is done before quantization f:θ⟶θ′:θ′=arctan2us11,us10+π and(9)arctan2us11,us10=θ,us11>0andus10>0,π−θ,us11>0andus10<0,θ−π,us11<0andus10<0,−θ,us11<0andus10>0,θ∈−π/2,π/2, and θ′∈0,2π. Mapping takes value of θ under consideration.Quantization function is given as follows.φs=frθ′=2q/Qπ, and q=modθ′/2π/T+1/2. If T = 8 then φs=sπ/4,s=0,1,T−1.The multiscale gradient orientation is useful for extracting gradient orientation to describe different granular aspects of visual representation. Multiscale gradient orientation is able to improve the discriminative ability of a single resolution. It is obtained by using square neighbors ofP pixels, considering the length of 2R+1. As illustrated in Figure 3, P denotes the set of neighbors, and R denotes the spatial resolution.Figure 3
Squared symmetric neighborhood for different (P, R). P represents neighbors and R represents the spatial resolution. Multiresolution analysis is done by choosing P as 4, 16, and 24 and R as 1.0, 2.0, and 3.0.
(a)(b)(c)The multiscale gradient orientation was obtained through combining the histograms of different operators at varyingP andR. By varying the P and R, whole image pixel values are considered. A multiscale analysis of WLD orientation can be done using the data generated by several operators of varying scales (P, R). Despite the fact that the operator is based on a squared symmetric neighbor set of P members on a square with side length (2R + 1), it can also be used for a circular situation. In general, it can improve the discrimination of a single resolution (P, R).The WLD orientation is used for texture estimation, and it also eliminates the effect of noise. The WLD orientation reduces the noise and illumination variation by computing the differences between the pixel and its neighbors. WLD orientation is tolerant of the presence of noise in an image. The influence of noise is reduced by using a WLD orientation, which is analogous to smoothing in image processing. Furthermore, the sum of its p-neighbor differences is divided by the current pixel’s intensity, reducing the influence of noise in an image.WLD orientation has also been established to mitigate the impact of changing brightness. It calculates the differences between the center pixel and its neighbors. As a result, a brightness modification that adds a constant to each image pixel has no effect on the disparities in values. WLD orientation, on the other hand, is responsible for dividing the differences. Thus, when each pixel value is multiplied by a constant, the differences are also multiplied by the same constant, canceling out the contrast change. As a result, the description is not affected by changes in brightness.Finally, quantized orientation map incorporating multiscale granularity is represented as follows:(10)qos=12omg×No180,qosf=s,t|s,t∈qos=j,0≤No=1,where s and t are spatial coordinates of orientation structure and j is the resultant value obtained across quantized value No=6.


### 3.2.3. Local Contrast Information

The pixel intensity is used to describe the local contrast information [41], as the visual information depends on intensity, and specific contrast range improves the visibility of visual information and increases the performance of CBIR systems. The V element of HSV color model was used to extract the intensity feature. This is due to the fact that V component of HSV color model called luma separates the chrominance element of image. The local contrast information map for input image fs,t is calculated as [42](11)lcms,t=∑s=1U∑t=1vmaxps,t,maxHs,t,bs,t.From the mathematical expression represented in (11), the constants s and t are the coordinates of input image fs,t. The dimensions of input image are represented by m and n. Then, the local contrast information was quantized at 10 levels to retain the salient information. The final intensity map was obtained and mathematically represented by the following expression:(12)qlcms,t=IMs,t∗ni,qlcms,tl=s,t|s,t∈qlcms,t=l,0≤l≤ni−1.


## 3.2.1. Color Feature

The chrominance signals of visual information hold significant discriminative information to represent the images in CBIR models. To reflect the chrominance attribute of visual information, different color spaces are used like YCbCr, HSV, HIS, Lab. The RGB color model is made up of three components: red, green, and blue. In proposed method, HSV color space was selected to segregate the hue, saturation, and intensity of image chrominance.In addition, the HSV color model represents the chrominance attribute of image, and it has cylindrical representation with hue (H), saturation (S), and value (V). The hue component of the HSV color space describes the wavelength of colors ranging from 0 to 360, beginning with red at 0, yellow at 60, green at 120, cyan at 180, blue at 240, and magenta at 300. The saturation component describes the chrominance saturation level between 0 and 1, where 0 denotes gray and 1 denotes primary color. The value V component describes the intensity values between 0 and 1, where 1 reflects black and 1 reflects white. The difference between HSV and RGB is that the HSV separates the intensity values from the color element. The RGB system is based on three color elements: red, green, and blue.The HSV color space is color invariant because the V component of HSV represents the image’s brightness and is independent of color [24]. With 8, 3, and 3 bins, the HSV color space is quantized into 72 colors. The quantization is essential in persevering images from light and intensity and also reduces the time complexity.(5)qh=h×bhmaxh,qs=s×bsmaxs,qv=v×bvmaxv,where bh, bs, and bv represent the quantized bins of hue, saturation, and value component of HSV [12].(6)cm=qh×bs×bv+qs×bv+qv.The quantized values for hue (h), saturation (s), and value (v) of HSV in color map cm are qh, qs, and qv. nc is the number of quantized colors. s and t are spatial coordinates in quantized color map qcm.(7)qcmj=s,t|s,t∈cm=j,0≤j≤nc−1.


## 3.2.2. Local Gradient Orientation (LGO)

For gradient computation, the orientation component of Weber local descriptor (WLD) was applied to HSV color model. This is due to the fact that by using the gray-scale images majority of chrominance information is lost. Therefore, the local gradient orientations were extracted, which holds rotation invariant attribute from chrominance part and is beneficial to obtain discriminative features. The gradient orientation [40] was calculated from the angle between the reference axis and the vector in horizontal and vertical location x.(8)θyc=αs1=arctanut11ut10,f10=−1+1 and f11=−1+1. When filters f10 and f11 are applied to an input image, then outputs ut11 and ut10 are obtained. ut10=y5−y1 and us11=y7−y3.θ is quantized into T gradient orientations. Firstly, mapping is done before quantization f:θ⟶θ′:θ′=arctan2us11,us10+π and(9)arctan2us11,us10=θ,us11>0andus10>0,π−θ,us11>0andus10<0,θ−π,us11<0andus10<0,−θ,us11<0andus10>0,θ∈−π/2,π/2, and θ′∈0,2π. Mapping takes value of θ under consideration.Quantization function is given as follows.φs=frθ′=2q/Qπ, and q=modθ′/2π/T+1/2. If T = 8 then φs=sπ/4,s=0,1,T−1.The multiscale gradient orientation is useful for extracting gradient orientation to describe different granular aspects of visual representation. Multiscale gradient orientation is able to improve the discriminative ability of a single resolution. It is obtained by using square neighbors ofP pixels, considering the length of 2R+1. As illustrated in Figure 3, P denotes the set of neighbors, and R denotes the spatial resolution.Figure 3
Squared symmetric neighborhood for different (P, R). P represents neighbors and R represents the spatial resolution. Multiresolution analysis is done by choosing P as 4, 16, and 24 and R as 1.0, 2.0, and 3.0.
(a)(b)(c)The multiscale gradient orientation was obtained through combining the histograms of different operators at varyingP andR. By varying the P and R, whole image pixel values are considered. A multiscale analysis of WLD orientation can be done using the data generated by several operators of varying scales (P, R). Despite the fact that the operator is based on a squared symmetric neighbor set of P members on a square with side length (2R + 1), it can also be used for a circular situation. In general, it can improve the discrimination of a single resolution (P, R).The WLD orientation is used for texture estimation, and it also eliminates the effect of noise. The WLD orientation reduces the noise and illumination variation by computing the differences between the pixel and its neighbors. WLD orientation is tolerant of the presence of noise in an image. The influence of noise is reduced by using a WLD orientation, which is analogous to smoothing in image processing. Furthermore, the sum of its p-neighbor differences is divided by the current pixel’s intensity, reducing the influence of noise in an image.WLD orientation has also been established to mitigate the impact of changing brightness. It calculates the differences between the center pixel and its neighbors. As a result, a brightness modification that adds a constant to each image pixel has no effect on the disparities in values. WLD orientation, on the other hand, is responsible for dividing the differences. Thus, when each pixel value is multiplied by a constant, the differences are also multiplied by the same constant, canceling out the contrast change. As a result, the description is not affected by changes in brightness.Finally, quantized orientation map incorporating multiscale granularity is represented as follows:(10)qos=12omg×No180,qosf=s,t|s,t∈qos=j,0≤No=1,where s and t are spatial coordinates of orientation structure and j is the resultant value obtained across quantized value No=6.


## 3.2.3. Local Contrast Information

The pixel intensity is used to describe the local contrast information [41], as the visual information depends on intensity, and specific contrast range improves the visibility of visual information and increases the performance of CBIR systems. The V element of HSV color model was used to extract the intensity feature. This is due to the fact that V component of HSV color model called luma separates the chrominance element of image. The local contrast information map for input image fs,t is calculated as [42](11)lcms,t=∑s=1U∑t=1vmaxps,t,maxHs,t,bs,t.From the mathematical expression represented in (11), the constants s and t are the coordinates of input image fs,t. The dimensions of input image are represented by m and n. Then, the local contrast information was quantized at 10 levels to retain the salient information. The final intensity map was obtained and mathematically represented by the following expression:(12)qlcms,t=IMs,t∗ni,qlcms,tl=s,t|s,t∈qlcms,t=l,0≤l≤ni−1.


## 3.3. Interlinked Microstructure Identification

Natural images contain large color, edge, and shape attribute, which are considered as low-level image attributes. More relevant images are retrieved from large datasets on the basis of color, edge, and shape attribute of image. Human eye shows sensitivity toward color and orientation features. Orientation reflects an informative content in image. Strong orientation gives a uniform pattern. As most natural standard images do not contain strong orientation, there is no uniform pattern. Natural images contain uniform and nonuniform patterns contributing spatial information, forming the microstructures. Orientation is not enough to discuss the image’s spatial characteristics, so color, edge, orientation, and local contrast information are fused to describe the richer image contents. In this paper, microstructures are proposed on the basis of color, orientation, and local contrast information attributes. Color feature and microstructure which is obtained by associating texture and local contrast information are joined to obtain micro-color structure. Orientation attributes and microstructure which is derived by linking color and intensity features are joined to obtain micro-orientation structure. Micro-intensity map is derived by intensity features and micro-map which is derived by linking orientation and intensity features. As microstructures are combination of color, orientation, and intensity features, they will increase the retrieval performance. Quantized orientation and quantized intensity ranges are used to derive micro-color map. The whole orientationOorientation is divided into 3 × 3 small grids [43]. Orientation is scale and rotation invariant, so it is used to derive color map. As orientation is quantized at level 6, it can vary from 0 to 5. For horizontal and vertical location, pixel of 3lengths is defined for each grid. Suppose that G∧ii=Gii+Gii+1+Gii+2+⋯GP are the grids which involve Oorientation where ii=1,2,……P where P represents length of grid. Similarly, suppose that G∧jj=Gjj+Gjj+1+Gjj+2+⋯GQ are the grids which involve Ointensity where jj=1,2,……Q where Q represents length of grid. Suppose that Gii is the grid of length 3×3 moved on quantized orientation and is the grid of length 3×3 moved on quantized local contrast information. Relationship is established between the pixels in the center and the pixels in the surrounding areas as a result of resemblance measures, which are helpful in finding the regular patterns. Suppose that cpo, cpc are the central pixels and ki and kj are neighboring pixels of quantized orientation and intensity microstructures. While moving grid of 3×3, neighbors of quantized orientation and intensity have the same value as central pixels of orientation and intensity; then, uniform or regular patterns and microstructure basic block are obtained. If no values of neighboring pixels of ki and kj are the same as those of the central pixels cpo, cpc, then irregular patterns are formed and microstructure basic block is not obtained. Five steps are followed to achieve a finally single image’s microstructure.(1)
Starting from0,0, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 0,0 and is labeled as M1s,t on the basis of regular patterns in both quantized orientation and intensity where 0≤s≤m−1, 0≤t≤n−1.(2)
At0,1, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 0,1 and is labeled as M2s,t on the basis of regular patterns in both quantized orientation and intensity where 0≤s≤m−1, 1≤t≤n−1.(3)
At1,0, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 1,0 and is labeled as M3s,t on the basis of regular patterns in both quantized orientation and intensity where 1≤s≤m−1, 0≤t≤n−1.(4)
At1,1, a grid of size 3×3 is moved on both quantized orientation and intensity from the left to the right and from the top to the bottom. The micro-map is established at 1,1 and is labeled as M4s,t on the basis of regular patterns in both quantized orientation and intensity where 1≤s≤m−1, 1≤t≤n−1.(5)
Final micro-map of image is obtained and demonstrated asx by combining all four maps.


## 4. Feature Quantization

Extraction of discriminative set of attributes for retrieving images is a difficult task. After obtaining discriminative set of features, the second task is the image’s attribute representation, that is, how to represent the extracted features. In this work, microstructures are suggested for feature representation. The main steps involved in forming correlated microstructures are shown in Figure4.Figure 4
Correlated microstructure. It is formed by correlating the quantized color, orientation, and local contrast information quantized values.


### 4.1. Color Feature

Microstructure’s value for the input imagefs,t is demonstrated as f1s,t = a, where a∈0,1,2,……,m−1 and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(13)Ga=L1fPa0=a0,,fPai=aiPai−Pa0=18L1¯fPa0=a0,a0=ai,i=1,2,3,…8.Every block of input image is3×3, supposing that Pao reflects the central pixel with position Pao=so,to and Pai reflects the neighboring pixels of central pixel with position Pai=si,ti. The value for the central pixel is Pao=e neighboring pixels and denotes the number of cooccurring values for ao and ai. L¯1 is used to represent the number of occurrences of ao. The micro-color structure is derived by establishing the relationship between microstructure image f1s,t and quantized color structureqcs. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f1s,t. The features of micro-color structure have 72 dimensions.


### 4.2. Orientation Feature

Microstructure’s value for the input imagefs,t is demonstrated as f2s,t=b, where b∈0,1,2,…,m−1, and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(14)Gb=L2fPb0=a0,,fPbi=aiPbi−Pb0=18L2¯fPb0=b0,b0=bi,i=1,2,3,…,8.Every block of input image is3×3, supposing that Pbo reflects the central pixel with position Po=so,to and Pbi reflects the neighboring pixels of central pixel with position Pi=si,ti. The value for the central pixel is Pbo=e neighboring pixels and denotes the number of cooccurring values for bo and bi. L¯2 is used to represent the number of occurrences of bo. The micro-color structure is derived by establishing the relationship between microstructure image f2s,t and quantized orientation structure qos. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f2s,t. The features of micro-orientation structure have 6 dimensions.


### 4.3. Local Contrast Information

Microstructure’s value for the input imagefs,t is demonstrated as f3s,t = c, where c∈0,1,2,…,m−1, and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(15)Gc=L3fPc0=c0,,fPci=ciPci−Pc0=18L3¯fPc0=c0,c0=ci,i=1,2,3,…,8.Every block of input image is3×3, supposing that Pco reflects the central pixel with position Po=so,to and Pci reflects the neighboring pixels of central pixel with position Pi=si,ti. The value for the central pixel is Pco=e neighboring pixels and denotes the number of cooccurring values for co and ci. L¯3 is used to represent the number of occurrences of co. The micro-color structure is derived by establishing the relationship between microstructure image f3s,t and quantized local contrast structure qlcs. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f3s,t. The features of micro local contrast information structure have 10 dimensions.


## 4.1. Color Feature

Microstructure’s value for the input imagefs,t is demonstrated as f1s,t = a, where a∈0,1,2,……,m−1 and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(13)Ga=L1fPa0=a0,,fPai=aiPai−Pa0=18L1¯fPa0=a0,a0=ai,i=1,2,3,…8.Every block of input image is3×3, supposing that Pao reflects the central pixel with position Pao=so,to and Pai reflects the neighboring pixels of central pixel with position Pai=si,ti. The value for the central pixel is Pao=e neighboring pixels and denotes the number of cooccurring values for ao and ai. L¯1 is used to represent the number of occurrences of ao. The micro-color structure is derived by establishing the relationship between microstructure image f1s,t and quantized color structureqcs. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f1s,t. The features of micro-color structure have 72 dimensions.


## 4.2. Orientation Feature

Microstructure’s value for the input imagefs,t is demonstrated as f2s,t=b, where b∈0,1,2,…,m−1, and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(14)Gb=L2fPb0=a0,,fPbi=aiPbi−Pb0=18L2¯fPb0=b0,b0=bi,i=1,2,3,…,8.Every block of input image is3×3, supposing that Pbo reflects the central pixel with position Po=so,to and Pbi reflects the neighboring pixels of central pixel with position Pi=si,ti. The value for the central pixel is Pbo=e neighboring pixels and denotes the number of cooccurring values for bo and bi. L¯2 is used to represent the number of occurrences of bo. The micro-color structure is derived by establishing the relationship between microstructure image f2s,t and quantized orientation structure qos. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f2s,t. The features of micro-orientation structure have 6 dimensions.


## 4.3. Local Contrast Information

Microstructure’s value for the input imagefs,t is demonstrated as f3s,t = c, where c∈0,1,2,…,m−1, and m reflects the dimensionality of micro-color structure which is obtained on the basis of the relationship between microstructure of quantized orientation and intensity. The equation used to derive the microstructure features is as follows:(15)Gc=L3fPc0=c0,,fPci=ciPci−Pc0=18L3¯fPc0=c0,c0=ci,i=1,2,3,…,8.Every block of input image is3×3, supposing that Pco reflects the central pixel with position Po=so,to and Pci reflects the neighboring pixels of central pixel with position Pi=si,ti. The value for the central pixel is Pco=e neighboring pixels and denotes the number of cooccurring values for co and ci. L¯3 is used to represent the number of occurrences of co. The micro-color structure is derived by establishing the relationship between microstructure image f3s,t and quantized local contrast structure qlcs. Final micro-color structure is obtained from quantized color map’s pixel values, which are in regular region present in f3s,t. The features of micro local contrast information structure have 10 dimensions.


## 5. Experimental Results and Findings

In this section, experiment is performed on Corel dataset to evaluate the retrieval performance of NRLGO method. The performance of NRLGO is given as follows.


### 5.1. Dataset

Different databases are used for evaluating the retrieval accuracy of images. In our proposed work, Corel database is used for measuring efficiency of NRLGO. Corel database consists of Corel 1k, Corel 5k, and Corel 10k. Corel 1k contains 1000 images of different contents such as Africans, mountains, elephants, buildings, beaches, and horses. There are 10 categories and 100 images per category with dimension of 256 × 384. Corel 5k is derived from Corel 10k and contains 5000 images of different contents such as waves, trees, lions, ducks, and food. There are 50 categories and 100 images per category. Corel 10k is a large dataset and contains 10000 images of different contents such as 250 cars, flowers, trains, furniture, butterflies, and tractors. There are 100 categories and 100 images per category with dimension of 192 × 128.


### 5.2. Evaluation Parameters

Precision and recall are the most important parameters for evaluating retrieval performance. Precision of input image is described as ratio of number of images retrieved that are similar to total number of images retrieved. Recall of input image is defined as ratio of similar images retrieved to total number of similar images.(16)PI=NsM,RI=NsMs,where M reflects the total number of images that have been retrieved and Ms reflects the total number of similar images that have been retrieved. Average precision and recall are also calculated to measure the image retrieval performance being 100; Ncdescribes the total number of images of Corel dataset in each category; and values 10, 50, and 100 describe the number of categories of Corel dataset. Average retrieval precision and recall are calculated by (18) and (20).(17)RPI=1Mc∑j=1McPI,(18)ARP=1Qc∑k=1QcRPI,(19)RRI=1Mc∑j=1McRI,(20)ARR=1Qc∑k=1QcRRI.F-score has characteristics of both precision and recall, and it combines precision and recall into single similarity measure as shown in the following:(21)Fs=2×PI×RIPI+RI.


### 5.3. Distance Metric

Manhattan distance (L1) is used to measure the similarity between the search image and the images in dataset by using the following equation:(22)Du,v=∑i=1NiuI−vi,ui=u1,u2,……,uN is the collection of attributes extracted from database’s images and vi=v1,v2,……,vN is the collection of attributes extracted from search image, where Wi and Vj=1,2,3,…,Ni, where Ni=8372+6+5 is the dimension of feature vector. Manhattan distance is good to measure similarity for a large image database by reducing the computational costs.


### 5.4. Performance of NRLGO

Color, texture, and local contrast information are used to retrieve relevant images from large dataset. In proposed NRLGO, color, edges, and local contrast information are extracted from the color image. The effectiveness of NRLGO is obtained by using 192, 128, 108, and 72 color quantization levels; 6, 12, 18, 24, 30, and 36 orientation quantization levels; and 10, 15, and 20 local contrast information quantization levels.Tables1–3 demonstrate the efficient performance on Corel dataset with 5000 images by changing color and texture levels by keeping local contrast information values fixed at 5, 10, and 15. In Table 1 highest precision of 65.82% is achieved when Nc = 72 and No = 6, and in Table 2 highest precision of 65.32% is achieved when Nc = 128 and No = 6. Moreover, in Table 3 highest precision of 65.58% is achieved when Nc = 128 and No = 24. So, in the proposed NRLGO, we obtained the descriptor of dimensions (72 + 6 + 5 = 83) by setting values for color Nc = 72, texture Nt = 6 and local contrast information Nci = 5. It is observed that, in Tables 1–3, ARP decreases when Nc = 128; Nci is set to 5, 10, and 15; and No values are selected from 6 to 36. This decrement is basically because of sensitivity to visual system toward continuously changing texture orientations. In some cases, nonuniform pattern of average retrieval precision is noticed; for example, in Table 3, when Nc = 72 and Nci = 15, average retrieval precision firstly increases for No from 6 to 18, then decreases at No = 24, and again increases for No from 30 to 36. This irregular pattern occurred because of noise due to quantization that increases variability within the class and decreases the performance of NRLGO.Table 1
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 5 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 5)Precision (%)Recall (%)612182430366121824303619262.163.1163.1062.6062.6862.797.467.577.577.517.527.5312863.4964.1263.8064.2363.1663.547.617.697.657.707.587.6210862.7362.5164.0562.6662.7062.407.527.507.687.527.527.487265.863.363.663.463.664.57.897.597.647.617.647.74Table 2
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 10 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 10)Precision (%)Recall (%)612182430366121824303619264.0564.2663.9464.0163.4763.357.687.717.677.687.617.6012865.3264.3865.0465.0564.6264.567.897.727.807.807.757.7410864.7164.1563.6865.0763.6863.637.767.697.647.807.647.637264.8765.3064.6765.4064.3565.377.787.837.767.857.727.84Table 3
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 15 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 15)Precision (%)Recall (%)612182430366121824303619264.5963.7663.9664.2164.0663.857.757.657.677.707.687.6612865.4765.5164.9465.5864.5664.757.857.867.797.877.747.7710864.8064.3064.9463.9064.6965.097.777.717.797.667.767.817265.3065.3265.4664.7565.4565.167.837.837.857.777.857.82Table4 shows the ARP obtained on Corel 5k by varying No from 6 to 36 and Nc from 16 to 32, 64, and 128 in RGB color space. Highest precision is achieved when color quantization is 128 and texture level is 6 at intensity level 5. It is shown from the table that HSV color space gave higher retrieval performance. As RGB color channels ignore the color characteristics, irregular pattern is obtained. Thus, by using HSV color space, average precision and recall have been increased. HSV color space is used in NRLGO. Not only is it effective toward user intentions, it also takes standard color characteristics into considerations. Hence, HSV color space increases average retrieval performances. Table 5 demonstrates retrieval accuracy obtained on Corel 1k, Corel 5k, and Corel 10k by using different similarity measures. In the proposed method, NRLGO, bin-by- bin similarity measures are applied. Similarity is measured between the search image and database image by using Manhattan distance. L1 increases the retrieval performance: performance of 83.5% on Corel dataset with 1000 images, 65.82% on Corel dataset with 5000 images, and 53.07% on Corel dataset with 10000 images. L1 does not perform any square or square root calculation, so it has high performance for large datasets, while L2 has poor performance at higher distance values and has high computation. It is clear from Table 5 that average retrieval accuracy is smaller on Corel dataset with 5000 images and Corel dataset with 10000 images by using Euclidean distance.Table 4
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 5 on Corel 5k in RGB.
Color quantization levelTexture classification (Corel 5k, contrast information = 5)Precision (%)Recall (%)612182430366121824303612860.160.559.959.559.859.77.077.107.117.067.137.096458.557.457.855.256.958.56.806.696.836.816.816.853253.550.255.055.654.749.46.206.236.316.306.276.251644.448.746.647.447.648.55.305.395.475.505.555.57Table 5
Average retrieval precision and average retrieval recall of NRLGO by varying combinations of gradient orientation and color quantization levels by keeping contrast information fixed at 5 on Corel 5k in RGB.
DatasetPerformanceDistance or similarity metricsChi-squareSquare chordMTHEuclidean distanceShepherdManhattanCorel 1kPrecision (%)73.576.7083.4597.9183.2083.50Recall (%)8.649.2010.0111.759.9810.02Corel 5kPrecision (%)55.4252.8959.6419.9164.0065.8Recall (%)6.836.347.152.397.687.89Corel 10kPrecision (%)47.9140.9649.9310.0052.7253.07Recall (%)5.534.915.9912.006.326.36Euclidean distance includes the square operation, so it does not perform well on larger distance values and is also computationally expensive. Furthermore, on Corel 1k, retrieval accuracy is less at square chord. Color, texture, and local contrast information attributes are selected for proposed method NRLGO. However, NRLGO performance is observed at different color, texture, and local contrast information’s combinations. ARP and ARR at 7 different combinations are shown in Table6 for performance comparisons.Table 6
Performance of NRLGO at Corel dataset with 1000, 5000, and 1000 images at different combinations of color, gradient orientation, and local contrast information vector.
DatasetPerformanceColorTextureContrastTexture + contrastColor + textureColor + contrastProposedCorel 1kPrecision (%)76.854.652.859.479.578.2583.5Recall (%)9.026.386.106.909.229.309.96Corel 5kPrecision (%)60.227.531.539.561.661.865.8Recall (%)7.153.063.894.507.307.257.89Corel 10kPrecision (%)48.520.525.928.650.749.353.07Recall (%)5.512.303.053.455.805.956.36It is evident that when the combination of color, texture, and local contrast information is used, better average retrieval precision (ARP) and average retrieval recall (ARR) are achieved. Besides, poor performance is achieved when using only color, texture, or local contrast information. Local contrast information is an essential feature as it distinguishes images on the basis of difference in illumination; however, it is less emphasized for efficient image retrieval methods. Standard images contain more color and intensity features. Therefore, contrast information is useful to distinguish between the images with the same contrast and the images with different contrast information. Proposed method NRLGO gave better retrieval performance in all combinations of color, texture, and local contrast information as compared to other descriptors on Corel dataset with 1000, 5000, and 10000 images. The retrieval accuracy of NRLGO descriptor is observed on Corel 1k in comparison with MTH [44], MSD [43], CMSD [2], CDH [45], SED [46], and ENN [47] descriptors. Table 7 shows the efficient retrieval accuracy of NRLGO in comparison to the state-of-the-art techniques at different categories of Corel dataset with 1000 images. Texture is penetrating feature in beach and mountain images. It is observed that beach category has variation in textural classifications. Therefore, human perception is more sensitive toward textural variations.Table 7
Category-wise performance comparison of NRLGO at Corel 1k with state-of-the-art descriptors.
CategoryPerformanceMTH [44]MSD [43]SED [46]ENN [47]CDH [45]CMSD [12]LeNet-F6 [48]ProposedAfricanPrecision69.1783.3382.508577.5086.668088.25Recall8.3010.009.909.009.3010.408.0010.30BeachPrecision61.6743.3328.337556.6742.086050.75Recall7.405.203.407.006.805.056.55.15BuildingPrecision45.8363.3347.507047.5081.667585.00Recall5.507.605.706.005.709.808.009.95BusPrecision68.3376.6773.337571.6781.668083.70Recall8.209.208.807.008.609.809.009.95DinosaurPrecision100.00100.0090.00100100100100100Recall12.0012.0010.801212.0012.0012.0012.00ElephantPrecision70.8365.0055.007562.5072.0885100Recall8.507.806.607.007.508.6510.0012.00FlowerPrecision75.0086.6772.508060.8384.167586.20Recall9.0010.408.708.007.3010.108.0010.20HorsePrecision100.0097.5062.508591.6794.169095.20Recall12.0011.707.509.0011.0011.3011.0011.40MountainPrecision39.1729.1740.006544.1750.006060.05Recall4.703.504.805.005.306.006.56.10FoodPrecision52.5076.6764.175545.0092.917594.9Recall6.309.207.703.005.4011.158.0011.38AveragePrecision68.2572.1661.5876.565.7578.547883.5Recall8.198.667.397.37.899.428.79.96Table7 shows that dinosaur’s category has 100% retrieval performance. Category-wise result evaluation shows that average retrieval precision is low at beach category, i.e., 50.75%, in comparison with MTH, MSD, SED, ENN, CDH, LeNET-F6, and CMSD. In beach category, most of mountain category images are retrieved, so this results in decrement of ARP in both mountain and beach category. In Table 8, 10, 20, 30, 40, 50, 60, 70, 80, 90, and 100 images are retrieved; a total of 100 images are retrieved per category; and R is the relevant images retrieved. Highest precision is achieved at top 10 images retrieved, and lowest precision is achieved at 100 images retrieved. To achieve multiresolution property that increases the retrieval accuracy, we performed experiment by varying P and R as shown in Table 9. Retrieval accuracy is increased when we select the value of P equal to 24 and R equal to 3.0. Similarly, we used different threshold values and achieved highest precision and recall when we selected the threshold value equal to 2. NRLGO extracts noise-robust features that remain invariant to variation in scale, illumination, and orientation. Experiment is performed on Corel dataset by adding Gaussian noise for texture recognition. Gaussian Noise of 5, 10, and 15 (%) is added to dinosaur color imageTable 8
Top 10 to 100 images are retrieved using NRLGO method for precision parameter.
P class102030405060708090100African0.650.900.730.750.700.660.640.510.610.60Beach0.400.750.660.600.600.540.520.500.470.46Building10.950.860.700.720.710.710.580.670.58Bus11110.980.910.910.860.820.81Dinosaur1111111111Elephant10.500.760.600.600.510.480.450.430.33Flower10.950.930.870.780.710.640.670.550.52Horse10.60110.880.960.930.980.840.82Mountain0.600.700.600.570.440.510.480.570.460.44Food0.700.600.600.570.620.600.580.570.560.53Average (R\P)0.8350.7950.7410.760.7320.710.680.6690.640.609Table 9
Retrieval accuracy observed at 9 different angles at Corel 1k dataset.
(P, R)(8, 1)(16, 2)(24, 3)AverageAccuracy82.483.584.683.5In Table9, by using different values of P and R, the multiresolution property is analyzed. Highest accuracy is observed when we choose the value of P equal to 24 and R equal to 3.In Table10, different values of s are used. Highest precision and recall are observed when we set threshold value equal to s = 2.NRLGO firstly converts the noisy color images to gray-scale images and then extracts the edges perfectly even in the presence of noise. It extracts the edges under different noise values as shown in Figure5. Accordingly, the accuracy of NRLGO is good in comparison to state-of-the-art descriptors on Corel dataset with 1000 images.Table 10
Retrieval accuracy observed at different threshold values using Corel 1k dataset. Highest retrieval accuracy is achieved ats = 2.
Noise resistantThreshold values (s)1234610Precision (%)83.483.582.882.382.783.2Recall (%)9.919.969.949.889.759.94Figure 5
Extraction of edges in presence of Gaussian Noise. Firstly, Gaussian noise of 5, 10, and 15 (%) is inserted. Secondly, conversion of RGB to gray scale is done; then, NRLGO is applied to detect the edges.


## 5.1. Dataset

Different databases are used for evaluating the retrieval accuracy of images. In our proposed work, Corel database is used for measuring efficiency of NRLGO. Corel database consists of Corel 1k, Corel 5k, and Corel 10k. Corel 1k contains 1000 images of different contents such as Africans, mountains, elephants, buildings, beaches, and horses. There are 10 categories and 100 images per category with dimension of 256 × 384. Corel 5k is derived from Corel 10k and contains 5000 images of different contents such as waves, trees, lions, ducks, and food. There are 50 categories and 100 images per category. Corel 10k is a large dataset and contains 10000 images of different contents such as 250 cars, flowers, trains, furniture, butterflies, and tractors. There are 100 categories and 100 images per category with dimension of 192 × 128.


## 5.2. Evaluation Parameters

Precision and recall are the most important parameters for evaluating retrieval performance. Precision of input image is described as ratio of number of images retrieved that are similar to total number of images retrieved. Recall of input image is defined as ratio of similar images retrieved to total number of similar images.(16)PI=NsM,RI=NsMs,where M reflects the total number of images that have been retrieved and Ms reflects the total number of similar images that have been retrieved. Average precision and recall are also calculated to measure the image retrieval performance being 100; Ncdescribes the total number of images of Corel dataset in each category; and values 10, 50, and 100 describe the number of categories of Corel dataset. Average retrieval precision and recall are calculated by (18) and (20).(17)RPI=1Mc∑j=1McPI,(18)ARP=1Qc∑k=1QcRPI,(19)RRI=1Mc∑j=1McRI,(20)ARR=1Qc∑k=1QcRRI.F-score has characteristics of both precision and recall, and it combines precision and recall into single similarity measure as shown in the following:(21)Fs=2×PI×RIPI+RI.


## 5.3. Distance Metric

Manhattan distance (L1) is used to measure the similarity between the search image and the images in dataset by using the following equation:(22)Du,v=∑i=1NiuI−vi,ui=u1,u2,……,uN is the collection of attributes extracted from database’s images and vi=v1,v2,……,vN is the collection of attributes extracted from search image, where Wi and Vj=1,2,3,…,Ni, where Ni=8372+6+5 is the dimension of feature vector. Manhattan distance is good to measure similarity for a large image database by reducing the computational costs.


## 5.4. Performance of NRLGO

Color, texture, and local contrast information are used to retrieve relevant images from large dataset. In proposed NRLGO, color, edges, and local contrast information are extracted from the color image. The effectiveness of NRLGO is obtained by using 192, 128, 108, and 72 color quantization levels; 6, 12, 18, 24, 30, and 36 orientation quantization levels; and 10, 15, and 20 local contrast information quantization levels.Tables1–3 demonstrate the efficient performance on Corel dataset with 5000 images by changing color and texture levels by keeping local contrast information values fixed at 5, 10, and 15. In Table 1 highest precision of 65.82% is achieved when Nc = 72 and No = 6, and in Table 2 highest precision of 65.32% is achieved when Nc = 128 and No = 6. Moreover, in Table 3 highest precision of 65.58% is achieved when Nc = 128 and No = 24. So, in the proposed NRLGO, we obtained the descriptor of dimensions (72 + 6 + 5 = 83) by setting values for color Nc = 72, texture Nt = 6 and local contrast information Nci = 5. It is observed that, in Tables 1–3, ARP decreases when Nc = 128; Nci is set to 5, 10, and 15; and No values are selected from 6 to 36. This decrement is basically because of sensitivity to visual system toward continuously changing texture orientations. In some cases, nonuniform pattern of average retrieval precision is noticed; for example, in Table 3, when Nc = 72 and Nci = 15, average retrieval precision firstly increases for No from 6 to 18, then decreases at No = 24, and again increases for No from 30 to 36. This irregular pattern occurred because of noise due to quantization that increases variability within the class and decreases the performance of NRLGO.Table 1
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 5 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 5)Precision (%)Recall (%)612182430366121824303619262.163.1163.1062.6062.6862.797.467.577.577.517.527.5312863.4964.1263.8064.2363.1663.547.617.697.657.707.587.6210862.7362.5164.0562.6662.7062.407.527.507.687.527.527.487265.863.363.663.463.664.57.897.597.647.617.647.74Table 2
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 10 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 10)Precision (%)Recall (%)612182430366121824303619264.0564.2663.9464.0163.4763.357.687.717.677.687.617.6012865.3264.3865.0465.0564.6264.567.897.727.807.807.757.7410864.7164.1563.6865.0763.6863.637.767.697.647.807.647.637264.8765.3064.6765.4064.3565.377.787.837.767.857.727.84Table 3
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 15 on Corel 5k in HSV.
Color quantizationTexture classification (Corel 5k, contrast information = 15)Precision (%)Recall (%)612182430366121824303619264.5963.7663.9664.2164.0663.857.757.657.677.707.687.6612865.4765.5164.9465.5864.5664.757.857.867.797.877.747.7710864.8064.3064.9463.9064.6965.097.777.717.797.667.767.817265.3065.3265.4664.7565.4565.167.837.837.857.777.857.82Table4 shows the ARP obtained on Corel 5k by varying No from 6 to 36 and Nc from 16 to 32, 64, and 128 in RGB color space. Highest precision is achieved when color quantization is 128 and texture level is 6 at intensity level 5. It is shown from the table that HSV color space gave higher retrieval performance. As RGB color channels ignore the color characteristics, irregular pattern is obtained. Thus, by using HSV color space, average precision and recall have been increased. HSV color space is used in NRLGO. Not only is it effective toward user intentions, it also takes standard color characteristics into considerations. Hence, HSV color space increases average retrieval performances. Table 5 demonstrates retrieval accuracy obtained on Corel 1k, Corel 5k, and Corel 10k by using different similarity measures. In the proposed method, NRLGO, bin-by- bin similarity measures are applied. Similarity is measured between the search image and database image by using Manhattan distance. L1 increases the retrieval performance: performance of 83.5% on Corel dataset with 1000 images, 65.82% on Corel dataset with 5000 images, and 53.07% on Corel dataset with 10000 images. L1 does not perform any square or square root calculation, so it has high performance for large datasets, while L2 has poor performance at higher distance values and has high computation. It is clear from Table 5 that average retrieval accuracy is smaller on Corel dataset with 5000 images and Corel dataset with 10000 images by using Euclidean distance.Table 4
Average retrieval precision and average retrieval recall of NRLGO at varying values of gradient orientation and color quantization levels by keeping local contrast information fixed at 5 on Corel 5k in RGB.
Color quantization levelTexture classification (Corel 5k, contrast information = 5)Precision (%)Recall (%)612182430366121824303612860.160.559.959.559.859.77.077.107.117.067.137.096458.557.457.855.256.958.56.806.696.836.816.816.853253.550.255.055.654.749.46.206.236.316.306.276.251644.448.746.647.447.648.55.305.395.475.505.555.57Table 5
Average retrieval precision and average retrieval recall of NRLGO by varying combinations of gradient orientation and color quantization levels by keeping contrast information fixed at 5 on Corel 5k in RGB.
DatasetPerformanceDistance or similarity metricsChi-squareSquare chordMTHEuclidean distanceShepherdManhattanCorel 1kPrecision (%)73.576.7083.4597.9183.2083.50Recall (%)8.649.2010.0111.759.9810.02Corel 5kPrecision (%)55.4252.8959.6419.9164.0065.8Recall (%)6.836.347.152.397.687.89Corel 10kPrecision (%)47.9140.9649.9310.0052.7253.07Recall (%)5.534.915.9912.006.326.36Euclidean distance includes the square operation, so it does not perform well on larger distance values and is also computationally expensive. Furthermore, on Corel 1k, retrieval accuracy is less at square chord. Color, texture, and local contrast information attributes are selected for proposed method NRLGO. However, NRLGO performance is observed at different color, texture, and local contrast information’s combinations. ARP and ARR at 7 different combinations are shown in Table6 for performance comparisons.Table 6
Performance of NRLGO at Corel dataset with 1000, 5000, and 1000 images at different combinations of color, gradient orientation, and local contrast information vector.
DatasetPerformanceColorTextureContrastTexture + contrastColor + textureColor + contrastProposedCorel 1kPrecision (%)76.854.652.859.479.578.2583.5Recall (%)9.026.386.106.909.229.309.96Corel 5kPrecision (%)60.227.531.539.561.661.865.8Recall (%)7.153.063.894.507.307.257.89Corel 10kPrecision (%)48.520.525.928.650.749.353.07Recall (%)5.512.303.053.455.805.956.36It is evident that when the combination of color, texture, and local contrast information is used, better average retrieval precision (ARP) and average retrieval recall (ARR) are achieved. Besides, poor performance is achieved when using only color, texture, or local contrast information. Local contrast information is an essential feature as it distinguishes images on the basis of difference in illumination; however, it is less emphasized for efficient image retrieval methods. Standard images contain more color and intensity features. Therefore, contrast information is useful to distinguish between the images with the same contrast and the images with different contrast information. Proposed method NRLGO gave better retrieval performance in all combinations of color, texture, and local contrast information as compared to other descriptors on Corel dataset with 1000, 5000, and 10000 images. The retrieval accuracy of NRLGO descriptor is observed on Corel 1k in comparison with MTH [44], MSD [43], CMSD [2], CDH [45], SED [46], and ENN [47] descriptors. Table 7 shows the efficient retrieval accuracy of NRLGO in comparison to the state-of-the-art techniques at different categories of Corel dataset with 1000 images. Texture is penetrating feature in beach and mountain images. It is observed that beach category has variation in textural classifications. Therefore, human perception is more sensitive toward textural variations.Table 7
Category-wise performance comparison of NRLGO at Corel 1k with state-of-the-art descriptors.
CategoryPerformanceMTH [44]MSD [43]SED [46]ENN [47]CDH [45]CMSD [12]LeNet-F6 [48]ProposedAfricanPrecision69.1783.3382.508577.5086.668088.25Recall8.3010.009.909.009.3010.408.0010.30BeachPrecision61.6743.3328.337556.6742.086050.75Recall7.405.203.407.006.805.056.55.15BuildingPrecision45.8363.3347.507047.5081.667585.00Recall5.507.605.706.005.709.808.009.95BusPrecision68.3376.6773.337571.6781.668083.70Recall8.209.208.807.008.609.809.009.95DinosaurPrecision100.00100.0090.00100100100100100Recall12.0012.0010.801212.0012.0012.0012.00ElephantPrecision70.8365.0055.007562.5072.0885100Recall8.507.806.607.007.508.6510.0012.00FlowerPrecision75.0086.6772.508060.8384.167586.20Recall9.0010.408.708.007.3010.108.0010.20HorsePrecision100.0097.5062.508591.6794.169095.20Recall12.0011.707.509.0011.0011.3011.0011.40MountainPrecision39.1729.1740.006544.1750.006060.05Recall4.703.504.805.005.306.006.56.10FoodPrecision52.5076.6764.175545.0092.917594.9Recall6.309.207.703.005.4011.158.0011.38AveragePrecision68.2572.1661.5876.565.7578.547883.5Recall8.198.667.397.37.899.428.79.96Table7 shows that dinosaur’s category has 100% retrieval performance. Category-wise result evaluation shows that average retrieval precision is low at beach category, i.e., 50.75%, in comparison with MTH, MSD, SED, ENN, CDH, LeNET-F6, and CMSD. In beach category, most of mountain category images are retrieved, so this results in decrement of ARP in both mountain and beach category. In Table 8, 10, 20, 30, 40, 50, 60, 70, 80, 90, and 100 images are retrieved; a total of 100 images are retrieved per category; and R is the relevant images retrieved. Highest precision is achieved at top 10 images retrieved, and lowest precision is achieved at 100 images retrieved. To achieve multiresolution property that increases the retrieval accuracy, we performed experiment by varying P and R as shown in Table 9. Retrieval accuracy is increased when we select the value of P equal to 24 and R equal to 3.0. Similarly, we used different threshold values and achieved highest precision and recall when we selected the threshold value equal to 2. NRLGO extracts noise-robust features that remain invariant to variation in scale, illumination, and orientation. Experiment is performed on Corel dataset by adding Gaussian noise for texture recognition. Gaussian Noise of 5, 10, and 15 (%) is added to dinosaur color imageTable 8
Top 10 to 100 images are retrieved using NRLGO method for precision parameter.
P class102030405060708090100African0.650.900.730.750.700.660.640.510.610.60Beach0.400.750.660.600.600.540.520.500.470.46Building10.950.860.700.720.710.710.580.670.58Bus11110.980.910.910.860.820.81Dinosaur1111111111Elephant10.500.760.600.600.510.480.450.430.33Flower10.950.930.870.780.710.640.670.550.52Horse10.60110.880.960.930.980.840.82Mountain0.600.700.600.570.440.510.480.570.460.44Food0.700.600.600.570.620.600.580.570.560.53Average (R\P)0.8350.7950.7410.760.7320.710.680.6690.640.609Table 9
Retrieval accuracy observed at 9 different angles at Corel 1k dataset.
(P, R)(8, 1)(16, 2)(24, 3)AverageAccuracy82.483.584.683.5In Table9, by using different values of P and R, the multiresolution property is analyzed. Highest accuracy is observed when we choose the value of P equal to 24 and R equal to 3.In Table10, different values of s are used. Highest precision and recall are observed when we set threshold value equal to s = 2.NRLGO firstly converts the noisy color images to gray-scale images and then extracts the edges perfectly even in the presence of noise. It extracts the edges under different noise values as shown in Figure5. Accordingly, the accuracy of NRLGO is good in comparison to state-of-the-art descriptors on Corel dataset with 1000 images.Table 10
Retrieval accuracy observed at different threshold values using Corel 1k dataset. Highest retrieval accuracy is achieved ats = 2.
Noise resistantThreshold values (s)1234610Precision (%)83.483.582.882.382.783.2Recall (%)9.919.969.949.889.759.94Figure 5
Extraction of edges in presence of Gaussian Noise. Firstly, Gaussian noise of 5, 10, and 15 (%) is inserted. Secondly, conversion of RGB to gray scale is done; then, NRLGO is applied to detect the edges.


## 6. Discussion

An image retrieval approach, based on Noise Resilient Local Gradient Orientation, that reduces the noise is proposed. Experiment is performed using Corel dataset, i.e., Corel dataset with 1000, 5000, and 10000 images, and the NRLGO achieves higher retrieval accuracy in comparison to state-of-the-art image techniques based on feature extraction techniques.


### 6.1. Comparison with Other Techniques

Natural images consist of regular and irregular textural patterns. Different textural descriptors like GLCM [25], Gabor features [28], SQ-Spatiogram [49], CMSD [2], and GMM-mSpatiogram [49] are proposed to check the performance on these regular and irregular patterns. It is observed that these textural descriptors perform well on regular patterns; their performance is not good at irregular patterns.Table11 demonstrates the retrieval accuracy of proposed NRLGO descriptor with uniform codes as compared to state-of-the-art methods, namely, GLCM [50], SQ-Spatiogram [49], GMM-mSpatiogram [49], SED [25], CMSD [2], and CPV-THF [42]. Experimental result shows that increase in retrieval accuracy of 27.8%, 24.13%, 14%, 9.5%, 2.66%, 1.9%, and 5.5% is achieved as compared to GLCM, SQ-Spatiogram [49], GMM-mSpatiogram [49], SED, CMSD, and CPV-THF on Corel 5k. On Corel 10k, for GLCM, SED, CMSD, CPV-THF, and LeNET-F6 [48], retrieval accuracy increase of 21.17%, 15.43%, 5.82%, 6.67%, 2.82%, and 79% is observed. Table 12 shows retrieval performance of proposed descriptor with respect to texture, color, and shape in comparison with state-of-the-art descriptor at Corel dataset with 5000 images and Corel dataset with 10000 images. Retrieval accuracy of NRLGO descriptor as compared to other descriptors is 29.6%, 26.4%, 29.98%, 17.36%, 27.32%, 5.52%, 2.82%, and 12.06% greater on Gabor [51], EHD [8], color moment [18], LBP [52], STH [53], and MTSD [54] at Corel dataset with 5000 images. At Corel dataset with 10000 images, the accuracy is 23.97%, 20.77%, 26.69%, 18.06%, 5.04%, and 1.12% on Gabor, EHD, color moment, LBP, STH, and MTSD.Table 11
Performance of NRLGO is compared with uniform pattern-based state-of-the-art methods on Corel dataset with 5000 images and Corel dataset with 10000 images.
DatasetPerformanceCPV-THF [42]GLCM [50]<SQs [49]GMMs [49]SED [46]CMSD [12]ProposedCorel 5kPrecision (%)63.9038.041.6751.8056.363.1465.8Recall (%)7.661.085.136.2221.37.577.89Corel 10kPrecision (%)52.2831.937.6447.2546.450.2553.07Recall (%)6.27.634.515.6718.36.036.36Table 12
Performance comparison of NRLGO with color, texture, and shape based state-of-the-art techniques on Corel 5k and Corel 10k.
DatabaseParametersGabor [51]EHD [8]CM [18]LBP [52]STH [53]MTSD [54]ProposedCorel 5kPrecision (%)36.239.435.8248.4460.2862.9865.8Recall (%)4.354.744.2928.877.239.457.89Corel 10kPrecision (%)29.132.326.3835.0148.0351.9553.07Recall (%)3.503.883.1618.595.767.796.36Increase in retrieval precision and recall shows that the proposed method outperforms the shape, orientation, and color techniques. Figures6–8 demonstrate performance of NRLGO with state-of-the-art techniques in form of curve using precision and recall parameters. x-axis is labeled as recall (%) and y-axis as precision (%). Precision-recall curves are drawn opposite to each other. For example, if average retrieval precision of descriptor is high, then obtained curve is drawn away from the point of origin. If ARP is low, then precision-recall will be short. We have also measured performance of NRLGO on the basis of F-score (%) with state-of-the-art methods like GA [55], GMM-mSpatiogram [49], ODBTC [56], Tetrolet Transform [57] and BiCBIR [58] as shown in Table 13 with the increase of F-score of 0.087%, 0.062%, 0.053%, 0.03%, and 0.015%.Figure 6
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 1k using precision and recall parameter.Figure 7
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 5k using precision and recall parameters.Figure 8
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 10k using precision and recall parameters.Table 13
F-score is compared with state-of-the-art descriptors at Corel 1k.
CategoryGA [55]ODBTC [56]GMM + spatiogram [49]Tetrolet transform [57]BiCBIR [58]ProposedAfrican0.1870.2820.2420.3170.3170.321Beach0.1790.1550.2170.2000.2500.265Building0.2030.2270.2350.1830.2830.311Bus0.2980.2950.2970.3330.3330.352Dinosaur0.3280.3310.3330.3330.3330.30Elephant0.1930.2440.2350.3000.3000.342Flower0.3000.3210.3160.3330.3330.321Horse0.2600.3130.3060.3330.3330.342Mountain0.1710.1580.2410.2500.2670.286Food0.2310.2690.2630.3330.3170.380Average0.2350.2600.2690.2920.3070.322When there is uncertainty in the performance of a descriptor, curve with different turning point is obtained. When the performance of two descriptors is the same in image category, then the curves overlapped at some point. NRLGO outperforms on Corel 1000, Corel 5000, and Corel 10000 the other state-of-the-art descriptors as shown in Figure6–8. These figures indicate that the ARP of proposed NRLGO is high, so precision-recall curve lies away from the point of origin. Proposed method NRLGO has a curve with few turning points and outperforms other descriptors for image categories of each dataset. MTH, SED, and MTH are textural descriptors used for efficient image retrieval including the correlation between texture and color, so local contrast information is missing. As correlation achieved by using intensity feature, the poor performance of MTH, SED, and STH is achieved. GLCM, EHD, and Gabor features are textural descriptors; they gave poor performance on natural images because using texture descriptor demonstrates limited texture classification of an image. Gabor filter is also used in textural classification due to the high relation between its cooccurrence and texture. EOAC does not consider textural characteristics of an image. MSD established correlation between color and orientation only and does not take local contrast information into account. The color moment (CM) approach considers only the spatial data of pixels that are near boundary of an image. LBP takes only textural characteristics of an image. A GUI application is established; it takes query image from African, dinosaur, bus, and flower category by the user and then retrieves 12 images from each category. Performance of NRLGO is observed on the basis of image’s features like color, texture, and shape (uniform-nonuniform).In Figure9, F-score at Corel 1k is calculated and then its average is obtained. It shows that highest F-score of 0.322 is achieved by using NRLGO method in comparison with state-of-the-art descriptors. Figure 10 shows that highest precision of 0.835 is achieved at retrieval of top 10 images and lowest precision of 0.60 is achieved at retrieval of 100 images. It shows that retrieval performance decreased when the retrieval number of images increased from 10 to 100.Figure 9
F-score is calculated for each category in Corel 1k and compared with the state-of-the-art techniques.Figure 10
Precision is calculated for the number of images retrieved at Corel 1k.Retrieval of top 12 images done against the search images selected from the African, dinosaur, bus, and flower category is shown in Figures11–14.Figure 11
Retrieval of top 12 images as a result of search image selected from African category.Figure 12
Retrieval of top 12 images as a result of search image selected from dinosaur category.Figure 13
Retrieval of top 12 images as a result of search image selected from bus category.Figure 14
Retrieval of top 12 images as a result of search image selected from flower category.In the proposed NRLGO method, NRLBP solves the issue of small pixel difference. It is used to save the local structures from noise by finding an uncertain state. It finds the value of uncertain state on the basis of corrected bits of LBP code. Uniform codes describe the image local structure and nonuniform codes describe the noise patterns. Thus, for finding uncertain state, error correction method that recovers the nonuniform patterns is used. Color feature is obtained in HSV color space by quantizing input image into 72 quantization levels. Texture classification is done by using local gradient orientation. Multiscale gradient orientation is used to extract multiple granularity features. It will reduce noise and illumination variations. V component of HSV is used to extract the intensity attribute. Similarity measure and efficient indexing are done by using Manhattan distance. Color, texture, and local contrast information micro-maps are used to demonstrate extracted features. They describe the important content that is within the uniform region and hide the irrelevant content.


## 6.1. Comparison with Other Techniques

Natural images consist of regular and irregular textural patterns. Different textural descriptors like GLCM [25], Gabor features [28], SQ-Spatiogram [49], CMSD [2], and GMM-mSpatiogram [49] are proposed to check the performance on these regular and irregular patterns. It is observed that these textural descriptors perform well on regular patterns; their performance is not good at irregular patterns.Table11 demonstrates the retrieval accuracy of proposed NRLGO descriptor with uniform codes as compared to state-of-the-art methods, namely, GLCM [50], SQ-Spatiogram [49], GMM-mSpatiogram [49], SED [25], CMSD [2], and CPV-THF [42]. Experimental result shows that increase in retrieval accuracy of 27.8%, 24.13%, 14%, 9.5%, 2.66%, 1.9%, and 5.5% is achieved as compared to GLCM, SQ-Spatiogram [49], GMM-mSpatiogram [49], SED, CMSD, and CPV-THF on Corel 5k. On Corel 10k, for GLCM, SED, CMSD, CPV-THF, and LeNET-F6 [48], retrieval accuracy increase of 21.17%, 15.43%, 5.82%, 6.67%, 2.82%, and 79% is observed. Table 12 shows retrieval performance of proposed descriptor with respect to texture, color, and shape in comparison with state-of-the-art descriptor at Corel dataset with 5000 images and Corel dataset with 10000 images. Retrieval accuracy of NRLGO descriptor as compared to other descriptors is 29.6%, 26.4%, 29.98%, 17.36%, 27.32%, 5.52%, 2.82%, and 12.06% greater on Gabor [51], EHD [8], color moment [18], LBP [52], STH [53], and MTSD [54] at Corel dataset with 5000 images. At Corel dataset with 10000 images, the accuracy is 23.97%, 20.77%, 26.69%, 18.06%, 5.04%, and 1.12% on Gabor, EHD, color moment, LBP, STH, and MTSD.Table 11
Performance of NRLGO is compared with uniform pattern-based state-of-the-art methods on Corel dataset with 5000 images and Corel dataset with 10000 images.
DatasetPerformanceCPV-THF [42]GLCM [50]<SQs [49]GMMs [49]SED [46]CMSD [12]ProposedCorel 5kPrecision (%)63.9038.041.6751.8056.363.1465.8Recall (%)7.661.085.136.2221.37.577.89Corel 10kPrecision (%)52.2831.937.6447.2546.450.2553.07Recall (%)6.27.634.515.6718.36.036.36Table 12
Performance comparison of NRLGO with color, texture, and shape based state-of-the-art techniques on Corel 5k and Corel 10k.
DatabaseParametersGabor [51]EHD [8]CM [18]LBP [52]STH [53]MTSD [54]ProposedCorel 5kPrecision (%)36.239.435.8248.4460.2862.9865.8Recall (%)4.354.744.2928.877.239.457.89Corel 10kPrecision (%)29.132.326.3835.0148.0351.9553.07Recall (%)3.503.883.1618.595.767.796.36Increase in retrieval precision and recall shows that the proposed method outperforms the shape, orientation, and color techniques. Figures6–8 demonstrate performance of NRLGO with state-of-the-art techniques in form of curve using precision and recall parameters. x-axis is labeled as recall (%) and y-axis as precision (%). Precision-recall curves are drawn opposite to each other. For example, if average retrieval precision of descriptor is high, then obtained curve is drawn away from the point of origin. If ARP is low, then precision-recall will be short. We have also measured performance of NRLGO on the basis of F-score (%) with state-of-the-art methods like GA [55], GMM-mSpatiogram [49], ODBTC [56], Tetrolet Transform [57] and BiCBIR [58] as shown in Table 13 with the increase of F-score of 0.087%, 0.062%, 0.053%, 0.03%, and 0.015%.Figure 6
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 1k using precision and recall parameter.Figure 7
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 5k using precision and recall parameters.Figure 8
Performance of NRLGO is compared with state-of-the-art descriptors on Corel 10k using precision and recall parameters.Table 13
F-score is compared with state-of-the-art descriptors at Corel 1k.
CategoryGA [55]ODBTC [56]GMM + spatiogram [49]Tetrolet transform [57]BiCBIR [58]ProposedAfrican0.1870.2820.2420.3170.3170.321Beach0.1790.1550.2170.2000.2500.265Building0.2030.2270.2350.1830.2830.311Bus0.2980.2950.2970.3330.3330.352Dinosaur0.3280.3310.3330.3330.3330.30Elephant0.1930.2440.2350.3000.3000.342Flower0.3000.3210.3160.3330.3330.321Horse0.2600.3130.3060.3330.3330.342Mountain0.1710.1580.2410.2500.2670.286Food0.2310.2690.2630.3330.3170.380Average0.2350.2600.2690.2920.3070.322When there is uncertainty in the performance of a descriptor, curve with different turning point is obtained. When the performance of two descriptors is the same in image category, then the curves overlapped at some point. NRLGO outperforms on Corel 1000, Corel 5000, and Corel 10000 the other state-of-the-art descriptors as shown in Figure6–8. These figures indicate that the ARP of proposed NRLGO is high, so precision-recall curve lies away from the point of origin. Proposed method NRLGO has a curve with few turning points and outperforms other descriptors for image categories of each dataset. MTH, SED, and MTH are textural descriptors used for efficient image retrieval including the correlation between texture and color, so local contrast information is missing. As correlation achieved by using intensity feature, the poor performance of MTH, SED, and STH is achieved. GLCM, EHD, and Gabor features are textural descriptors; they gave poor performance on natural images because using texture descriptor demonstrates limited texture classification of an image. Gabor filter is also used in textural classification due to the high relation between its cooccurrence and texture. EOAC does not consider textural characteristics of an image. MSD established correlation between color and orientation only and does not take local contrast information into account. The color moment (CM) approach considers only the spatial data of pixels that are near boundary of an image. LBP takes only textural characteristics of an image. A GUI application is established; it takes query image from African, dinosaur, bus, and flower category by the user and then retrieves 12 images from each category. Performance of NRLGO is observed on the basis of image’s features like color, texture, and shape (uniform-nonuniform).In Figure9, F-score at Corel 1k is calculated and then its average is obtained. It shows that highest F-score of 0.322 is achieved by using NRLGO method in comparison with state-of-the-art descriptors. Figure 10 shows that highest precision of 0.835 is achieved at retrieval of top 10 images and lowest precision of 0.60 is achieved at retrieval of 100 images. It shows that retrieval performance decreased when the retrieval number of images increased from 10 to 100.Figure 9
F-score is calculated for each category in Corel 1k and compared with the state-of-the-art techniques.Figure 10
Precision is calculated for the number of images retrieved at Corel 1k.Retrieval of top 12 images done against the search images selected from the African, dinosaur, bus, and flower category is shown in Figures11–14.Figure 11
Retrieval of top 12 images as a result of search image selected from African category.Figure 12
Retrieval of top 12 images as a result of search image selected from dinosaur category.Figure 13
Retrieval of top 12 images as a result of search image selected from bus category.Figure 14
Retrieval of top 12 images as a result of search image selected from flower category.In the proposed NRLGO method, NRLBP solves the issue of small pixel difference. It is used to save the local structures from noise by finding an uncertain state. It finds the value of uncertain state on the basis of corrected bits of LBP code. Uniform codes describe the image local structure and nonuniform codes describe the noise patterns. Thus, for finding uncertain state, error correction method that recovers the nonuniform patterns is used. Color feature is obtained in HSV color space by quantizing input image into 72 quantization levels. Texture classification is done by using local gradient orientation. Multiscale gradient orientation is used to extract multiple granularity features. It will reduce noise and illumination variations. V component of HSV is used to extract the intensity attribute. Similarity measure and efficient indexing are done by using Manhattan distance. Color, texture, and local contrast information micro-maps are used to demonstrate extracted features. They describe the important content that is within the uniform region and hide the irrelevant content.


## 7. Conclusion

A feature descriptor named Noise Resilient Local Gradient Orientation is proposed in this paper for improving the retrieval performance. NRLGO relies on noise removal, color, texture classifications, and local contrast information. Small pixel difference causes noise in color image which leads to changing code abruptly resulting in poor feature extraction. NRLBP protects the local structure from noise by finding the uncertain state first. HSV color space is quantized into 72 levels to obtain the color attribute. Texture classification is done by using LGO approach at 6 quantization levels, which quantize orientation further into supreme orientation. V component of HSV is quantized into 10 levels to obtain the local contrast information. Incorporation of multiresolution gradient orientation gave better texture detection, which increases the relationship between color, texture, and local contrast information. To achieve the resemblance between search image and image in the database, Manhattan distance is used. Correlation is established between color, texture, and local contrast information. For image characteristic representation, correlation is developed between the color, orientation, and local contrast data. Microstructures are developed on the basis of correlation information to develop more fine details in subject field. Multiresolution orientation increased the retrieval performance of proposed NRLGO. NRLGO extracts noise-robust features that remain invariant to variation in scale, illumination, and orientation. Experimental results show that NRLGO has improved retrieval performance of texture, local contrast information, and color features in comparison with state-of-the-art descriptors on Corel with 1000 images, 5000 images, and 10000 images.



---

*Source: 4151482-2021-07-14.xml*