Graph wavelet transform for image texture classification

IET Image ProcessingVolume 15, Issue 10 p. 2372-2383 ORIGINAL RESEARCH PAPEROpen Access Graph wavelet transform for image texture classification Yu-Long Qiao, College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this authorYue Zhao, Corresponding Author zhaoyue1994@hrbeu.edu.cn College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR China Correspondence Yue Zhao, College of Information and Communications Engineering, Harbin Engineering University, 145 Nantong Street, Nangang District, Harbin, Heilongjiang, PR China. Email: zhaoyue1994@hrbeu.edu.cnSearch for more papers by this authorChun-Yan Song, College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this authorKai-Ge Zhang, Department of Computer Science, Utah State University, Logan, Utah, USASearch for more papers by this authorXue-Zhi Xiang, College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this author Yu-Long Qiao, College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this authorYue Zhao, Corresponding Author zhaoyue1994@hrbeu.edu.cn College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR China Correspondence Yue Zhao, College of Information and Communications Engineering, Harbin Engineering University, 145 Nantong Street, Nangang District, Harbin, Heilongjiang, PR China. Email: zhaoyue1994@hrbeu.edu.cnSearch for more papers by this authorChun-Yan Song, College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this authorKai-Ge Zhang, Department of Computer Science, Utah State University, Logan, Utah, USASearch for more papers by this authorXue-Zhi Xiang, College of Information and Communications Engineering, Harbin Engineering University, Harbin, Heilongjiang, PR ChinaSearch for more papers by this author First published: 16 April 2021 https://doi.org/10.1049/ipr2.12220AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Graph is a data structure that can represent complex relationships among data. Graph signal processing, unlike traditional signal processing, explicitly considers the structure and relationship among the signal samples. Graph wavelet transform can provide a multiscale analysis for the graph signal. It is well known that texture is a region property in an image, which is characterized with the intensity and relationship among pixels. In this context of the graph signal processing framework, an image texture can be considered as the signal on the graph. Therefore, a texture classification method based on graph wavelet transform is proposed. Specifically, image textures are decomposed into multiscale components by using two-channel graph wavelet filter banks. Then the local singular value decomposition is applied to each subband. In order to improve the noise-resistant ability, the maximum, mean and median values of the local singular values of graph-wavelet transformation coefficients are extracted. Finally, the Weibull distributions are used to model those extracted values to describe the image textures. The experiments on the benchmark texture datasets are conducted to demonstrate the effectiveness of the proposed method. 1 INTRODUCTION Texture is an important property of the visual scene and provides a power cue for identification and categorization of different objects and natural scenes in the human visual system [1]. In the field of image processing and computer vision, texture classification refers to as assigning an unknown texture sample to one of a set of predefined texture categories, which has attracted a lot of attention [2]. It has been applied in many fields, such as medical diagnosis [3, 4], forest species recognition [5] and face recognition [6]. The core issue of texture classification is to describe the image texture with effective features. Tuceryan and Jain [1] classified the various texture feature extraction methods before 1993 into five categories: statistical methods, geometric methods, structural methods, mathematical model methods and signal processing methods [7]. The wavelet transform [8, 9], as a classical signal processing technique, possesses better time-frequency localization and multiresolution analysis properties. Thus, various wavelet transform based methods have been proved for the image texture description. Mallat [8] firstly applied the discrete wavelet transform (DWT) in the texture feature extraction. Because DWT is a translation-variant transform, Unser [10] proposed the wavelet frame transform based texture classification. Qiao et al. [11] made use of the complex wavelet frame transform, and combined the magnitude and phase of the complex coefficients into a real measure to characterize the texture. Due to the approximately shift invariance and good directional selectivity of the dual-tree complex wavelet transform (DT-CWT), Celik and Tjahjadi [12] suggested that the statistical measures of magnitude and phase of the DT-CWT coefficients are better than the DWT based method. Ji et al. [13] applied the multi-fractal spectrum (MFS) analysis on wavelet coefficients to extract robust texture descriptors for texture classification. Wang et al. [14] extracted the texture feature based on the fractal dimensions and barycentric coordinates of the bit planes of the wavelet coefficients in both the detail wavelet subbands and approximate wavelet subbands. Yang et al. [15] obtained the image pyramid by applying dual-tree complex wavelet transform (DTCWT) on the original image, and then generated the local binary patterns (LBP) in DTCWT domain to extract features. In recent years, Rieze wavelets [16] and steerable circular harmonic wavelets [17] have shown great potential in texture classification. In addition, researchers have exploited the traditional texture features, Gray Level Co-occurrence Matrix(GLCM) [18, 19], Madras Rubber Factory model [20] and Local Binary Patterns (LBP) method [21] in the wavelet domain. Graph signal processing is one of the emerging fields in signal processing. In past years, Crovella and Kolaczyk [22] designed wavelet-like functions on graphs which are localized in space and time. In 2011, Hammond et al. [23] constructed a class of wavelet operators in the graph spectral domain, which leads to the research on applying the concepts of filter banks from classical signal processing to graph signals. Narang and Ortega [24] constructed critically sampled wavelet filter banks on graph for analyzing graph-signals defined on any arbitrary finite weighted graph. Then the biorthogonal wavelet filter banks on graph [25] have been introduced. Tanaka and Sakiyama [26] proposed M-channel oversampled filter banks for graph signals. Jiang et al. [27] designed biorthogonal graph filter banks with the lifting-based method. Teke et al. [28] extended the classical multirate signal processing theory to graph signals. Nicolas and Pierre [29] leveraged the graph wavelet to complete the multiscale detection of communities in networks. In recent years, graph wavelets are combined with the convolutional neural network (CNN), and the graph wavelet neural network is proposed in the works [30, 31]. Texture is a region property in an image, which is characterized with the intensities and relationships among pixels. Therefore, the texture features, that describe the relationships among the pixel intensities have been demonstrated power discriminative performance in texture classification, such as GLCM, Markov random field, LBP, some variants of the LBP [32], contourlet transform [33, 34] and wavelet transforms. Although the traditional wavelet transforms have been successfully applied in texture feature extraction, it only implicitly considers the local structure of a texture by using the filter banks. Unlike traditional signal processing, graph signal processing explicitly considers the structure and relationship among the signal samples. In this context, an image texture can be considered to be the graph signal with very special structures. The graph wavelet transform (wavelet transform on graph) combines the advantages of both the wavelet transform and graph representation of the signal. It can be reasonably applied to describe the image texture. Therefore, in this paper, we extract the image texture feature in the graph wavelet transform domain. It is well known that the singular value decomposition (SVD that is related with Principal Component Analysis) is a matrix decomposition method, in which larger singular values point to the major information of the matrix. The truncation of lower singular values also results in very good approximation of the original matrix, which can be considered as extracting the major information or denoising. Thus, the SVD (or PCA) based texture features are designed to be tolerance to image blur and noise corruption [35]. Ramakrishnan et al. [36] proposed a new approach based on multiwavelets transformation and singular value decomposition, in which the energy, entropy, local homogeneity and max–min ratio are extracted from the selected singular values as image texture features. Selvan et al. [37] modeled the singular value distribution of wavelet transform coefficients with an exponential function. The estimated six model parameters are taken as image texture features. However, Kim and So [38] demonstrated that it was hardly expected that a six-dimensional feature vector would yield a state-of-the-art performance, and then they proposed a modified texture feature based Haar wavelet transform and SVD. For a texture image, the singular value decomposition of the whole coefficient matrix of a wavelet subband only captures the major information of that subband. It is difficult to characterize the local property of the texture. Meanwhile, the exponential distribution model parameters of those singular values hardly describe a texture. Therefore, in this paper, we divide the graph wavelet subband into overlapping regions and apply SVD in each region. Then the maximum values, mean values and median values of each subband are modeled with Weibull distributions, respectively. The model parameters are accumulated into the feature vector for texture classification. The remaining structure of this paper is as follows. Section 2 mainly introduces the graph signal processing and graph wavelet filter banks. The scheme of this paper including graph wavelet transform for image textures, local singular value decomposition and Weibull distribution model of wavelet coefficients are proposed in Section 3. In Section 4, we conduct the texture classification experiments and demonstrate the performance of the proposed method. Finally, our work is concluded in Section 5. 2 GRAPH WAVELET TRANSFORM A graph represents a set of elements and a set of pairwise relationships between those elements, in which the element is called vertex, and the relationship is called edge. Formally, a graph G is defined as G = ( V , E ) , where V represents the set of vertices, and the E represents the set of edges. In this paper, we only consider the undirected graph without self-loops. The connectivity of a graph can be represented by an adjacency matrix A, which is defined as A i , j = w i , j , vertices i and j are connected 0 , otherwise , (1) w i , j is the weight that describes the relationship between vertices i and j. A graph signal f : V → R is defined on the graph, which is indexed by the vertices and may be represented by a vector. When the image is represented as a graph signal, the grey values of pixels and the relationships between different pixels can be considered. The degree matrix D is a diagonal matrix, whose the ith diagonal element is d i that represents the degree of vertex i, whose value is the sum of the weight of all edges connected to node i. The unnormalized graph Laplacian matrix is defined as L = D − A . The symmetric normalized graph Laplacian matrix is L = I − D − 1 / 2 A D − 1 / 2 , where I is the identity matrix. It is a symmetric positive semi-definite matrix and has a set of real eigenvalues 0 ≤ λ 1 ≤ λ 2 ≤ ⋯ ⋯ ≤ λ N ≤ 2 . Recently, researchers build a graph signal processing framework from the graph spectral theory, such as graph Fourier transform, graph filtering and graph wavelet filter banks, which provides potential transforms for image texture analysis. 2.1 Graph filtering The spectral representation of the graph signal is related to the graph Laplacian matrix L or the normalized version L, which is a symmetric positive semi-definite matrix. We follow the work [25, 26], the matrix L (or L) can be decomposed with the eigenvalue decomposition as follows, L = U Λ U T = ∑ i = 1 N λ i u i u i T , (2)where u 1 , u 2 , ⋯ u N − 1 , and u N are the eigenvectors corresponding to the eigenvalues λ 1 , λ 2 , ⋯ λ N − 1 , and λ N , respectively. Similar to the traditional Fourier transform, the eigenvalues are considered as the graph frequencies and eigenvectors as the graph Fourier transform basis. Therefore, for a graph signal f ∈ R N , the graph Fourier transform is defined as f ¯ k = u k , f = ∑ i = 1 N f i u k i . (3)By using the eigenspace projection matrix P λ i = ∑ λ i u i u i T . (4)The graph filter for graph signal in the spectral domain is defined as follows, H = ∑ λ i ∈ σ L h λ i P λ i , (5)where σ ( L ) is the set of eigenvalues of the graph Laplacian matrix L (or L). The filter H acts on a given graph signal f by modulating each Fourier mode as Hf = ∑ k = 1 N h λ k f ¯ k u k . (6)Due to the high complexity of the eigenvalue decomposition, the Chebyshev polynomial approximation is used to implement the graph signal filtering [23]. 2.2 Graph wavelet filter banks The critically sampled two-channel wavelet filter banks for the bipartite graph signal is shown in Figure 1. It is assumed that the signal f is defined on the bipartite graph G = ( L , H , E ) . The graph filters H 0 and H 1 are analysis filters, and G 0 and G 1 are synthesis filters, which can be defined as Equation (3) with different graph filter kernels h 0 ( λ ) , h 1 ( λ ) , g 0 ( λ ) and g 1 ( λ ) . The operators ↓ J L and ↑ J L are downsampling and upsampling functions in the first channel, respectively. In order to design critically sampled wavelet filter banks, Narang [24] defines the diagonal downsampling matrix J L = d i a g { J L ( n ) } , in which J L n = + 1 , i f n ∈ L − 1 , i f n ∈ H . (7) FIGURE 1Open in figure viewerPowerPoint Two-channel graph wavelet filter bank The downsampling and then upsampling operator can be represented as L D U = 1 2 I + J L , (8)where I is the identity matrix. In the second channel, the diagonal downsampling matrix is J H = − J L . Based on the perfect reconstruction condition, 1 2 G 0 I + J L H 0 + 1 2 G 1 I + J H H 1 = I . (9)Narang et al. [24] gives the necessary and sufficient condition for the perfect reconstruction condition in bipartite graphs filter banks as follows, g 0 ( λ ) h 0 ( λ ) + g 1 ( λ ) h 1 ( λ ) = c 2 g 0 ( λ ) h 0 ( 2 − λ ) − g 1 ( λ ) h 1 ( 2 − λ ) = 0 . (10) They also extend the well-known quadrature mirror filter (QMF) solution to the case of bipartite graphs, such as the ideal kernel and Meyer kernel functions. 2.3 2D graph wavelet transform A digital image is a regular 2D signal, which can be considered as a graph signal by connecting each pixel in the image to its adjacent pixels (vertices) and defining the pixel value as the value of the graph signal. Formally, we define an image texture as a signal on the undirected graph G = ( V , E ) , in which V is the set of pixels (vertices), and E is the set of links (edges) of pixels. Three typical graphs [24] for an image are shown in Figure 2. The graph in Figure 2a is 8-connected image graph that is 4-partite, in which each pixel is a vertex on the graph and each vertex is connected to its eight adjacent vertices. FIGURE 2Open in figure viewerPowerPoint Graph-formulation of 2D images. (a) 8-connected image-graph G, (b) 4-connected rectangular image-graph, (c) 4-connected diagonal image graph The graph wavelet filter banks are implemented on bipartite graphs. Therefore 2D graph wavelet transform on an image can be performed by decomposing an arbitrary graph into bipartite subgraphs. For example, the 8-connected graph can be decomposed into two 4-connected bipartite graphs B 1 = ( L 1 , H 1 , E 1 ) and B 2 = ( L 2 , H 2 , E 2 ) as shown in Figure 2b,c [24]. Similar to the traditional 2D separable wavelet transform, two-channel graph wavelet filter banks are applied in a cascaded manner, by filtering along a series of bipartite subgraphs of the original graph. Specifically, the graph analysis filters are firstly applied on the first bipartite graph B1(shown in Figure 2b). After downsampling the filtering signals, the graph analysis filtering and downsampling are operated on the signals on two disjoint graphs B 2 ( L 1 ) and B 2 ( H 1 ) , separately, which is shown in Figure 3a. Narang [24] has also proven that the graph wavelet decomposition on bipartite graphs in Figure 3a is the same as the implementation in Figure 3b. In Figure 3, the graph filter kernels h 0 and h 1 are used to construct the graph filters with Equation (3) on the corresponding bipartite subgraphs. The downsampling operator ↓ J i j ( i = L 1 , H 1 and j = L 2 , H 2 ) is to preserve those vertices corresponding to J i ( · ) = 1 and J j ( · ) = 1 . FIGURE 3Open in figure viewerPowerPoint 2D separable two-channel filter bank. (a) Graph wavelet decomposition on an image. (b) Equivalent implementation of graph wavelet decomposition 3 FEATURE EXTRACTION AND CLASSIFICATION It is well known that the traditional wavelet transform possesses better time-frequency localization and multiscale analysis properties. The wavelet transform based texture feature extraction method explores those advantages and provides excellent texture descriptors. However, the wavelet transform does not explicitly consider the local relationship of a texture. Therefore, in this paper, an image texture is represented as a graph signal. Graph wavelet transform combines the advantages of wavelet transform and graph signal processing. It provides a multiscale analysis way for the graph signal. This new transform is used to obtain the multiscale representation of an image texture. The second key component of our method is the local singular value decomposition, because the truncation of lower singular values can be considered as extracting the major information or denoising. Unlike the general truncation method, we take the median, maximum and mean values of the local singular values in each local region. Finally, we use the Weibull distributions to characterize their statistical properties. The model parameters serve as the texture feature. The image texture feature extraction method proposed in this paper is summarized in Figure 5, which includes the following aspects: 1) graph wavelet transform of texture images; 2) local singular value decomposition; 3) Weibull distribution modeling and parameter estimation. 3.1 Graph wavelet transform One of the advantages of the critically sampled two-channel graph-based wavelet transform is that the number of transform coefficients is the same as the number of image pixels. Thus this critically sampled transform has been used in image compression [39]. However, as mentioned in the work [10], the overcomplete wavelet decomposition is much better than the complete wavelet transformation for texture analysis. Therefore, we combine the overcomplete transform with critically sampled transform as shown in Figure 4. FIGURE 4Open in figure viewerPowerPoint Overcomplete two-channel graph filter bank decomposition of an image texture. Only the LL channel is subjected to the downsampling operation and used for the next level decomposition Compared with the critically sampled graph wavelet transform shown in Figure 3b, our graph wavelet transform omits the downsampling operation in the LH, HL and HH channels and only preserves the downsampling operator in the LL channel. That is to say, after one level decomposition, the sizes of graph wavelet coefficients of the LH, HL and HH channels equal to the size of the original image texture, and the size of the LL channel is one fourth of the size of the original image. Then the next level decomposition is performed on the LL channel. In the traditional wavelet frame transform [10], the LL channel is not downsampled. But the analysis filters must be upsampled by inserting a zero between two filter taps. For graph filters, they are defined in the graph spectral domain. We cannot simply unsampled the graph filters for the next level decomposition in the vertex domain. Therefore, in order to obtain the multiscale overcomplete graph wavelet decomposition, we apply the downsampling operator to the LL channel. 3.2 Local singular value decomposition Singular value decomposition is a very powerful tool, which is successfully applied in the dimensionality reduction, solving linear equations and noise reduction [36]. Let Y be a matrix of size m × n , its SVD is defined as Y = U Σ V T , (11)where Σ is a diagonal m × n matrix with non-negative real diagonal elements. The diagonal entries of Σ are singular values. The left singular vectors and right singular vectors form the orthogonal matrices U and V, respectively. The superscript T denotes the matrix transpose. Selvan et al. [37] showed that the singular value decomposition of wavelet transform coefficients can be used for texture classification. Kim and So [38] modified the SVD based texture feature in the Haar wavelet domain, and demonstrated the improved classification performance. Nevertheless, it is well known that texture is a region property. The singular value decomposition of the whole coefficient matrix of a wavelet subband only capture the major information of that subband. It is difficult to characterize the local property of the texture. Meanwhile, the number of non-zero singular values equals to the rank of a coefficient matrix. When the size of the texture sample is small, it is hard to obtain a good probability distribution model for fitting the singular values, especially in the critically sampled wavelet domain, in which the size of each subband at next level decomposition is one quarter of the size of the subband at the current level. Therefore, in this paper, we make use of local SVD of graph wavelet subbands to extract the texture feature. As for a graph wavelet subband, the local SVD is shown in the Figure 6. We perform the singular value decomposition on the region within the window. Then the window is moved to the next coefficient to perform local singular value decomposition on the windowed region. That is to say, we use the SVD to extract the major information in the local region. Selecting the size of the window plays an important role in texture feature extraction. It can directly affect the performance of texture classification. At present, there is no general way to determine the optimal size of the window without supervision. Considering the computational complexity and ensuring a good texture classification, we empirically choose a window with the size of 4 × 4. It is well known that low singular value truncation is considered as a noise reduction method. This manipulation can reduce the noise influence in texture classification when the image texture is corrupted by noise. There are many methods for low singular value truncation in the literature. For example, it can define a measure called index of separability [36] as α = σ 1 2 ∑ i = 1 m σ i 2 , (12)to select the significant singular values. This truncation method works well when number of singular values is large. However, in the local SVD with 4 × 4 window, there are 4 singular values. It seems that the general truncation does not suitable for texture description. In order to extract the texture features and make the proposed method to be noise-resistant to some extent, we take the median, maximum and mean values of the local singular values in each window, and form three sets respectively, which will be modeled with Weibull distributions later. It is assumed that there are N local regions in a subband, and the singular values of the ith region form a vector S i . We can obtain three sets F 1 , F 2 and F 3 , F 1 = { max ( S 1 ) , max ( S 2 ) , … , max ( S i ) , … max ( S N ) } F 2 = { mean ( S 1 ) , mean ( S 2 ) , … , mean ( S i ) , … mean ( S N ) } F 3 = { median ( S 1 ) , median ( S 2 ) , … , median ( S i ) , … median ( S N ) } , (13)which are composed of the median, maximum and mean values of every vector S i , respectively. 3.3 Weibull distribution model The Weibull distribution has been widely used in various applications, such as reliability analysis and weather forecasting. In general, there are one-parameter Weibull distribution, two-parameter Weibull distribution, three-parameter Weibull distribution and hybrid Weibull distribution. From the perspective of probability and statistics, the probability density function of the two-parameter Weibull distribution [40] is defined as follows, f ( x ; λ , k ) = k λ x λ k − 1 e − ( x / λ ) k x ≥ 0 0 x < 0 , (14)where x is a random variable, λ > 0 is the proportional parameter, k > 0 is the shape parameter. Obviously, the Weibull distribution function becomes an exponential distribution, if the shape parameter k equals to 1. When k = 2, it is a Rayleigh distribution. If k = 3.458, it is most similar to a normal distribution. Thus the Weibull distribution possesses great potential in the data modeling. When we model the extracted sets F1, F2 and F3 in a subband, the key issue is to estimate the distribution parameters. Without loss of generality, let the elements of the set F1 to be 0 ≤ x 1 ≤ x 2 ≤ … ≤ x N , it can be determined the likelihood function from the Weibull probability density function as, L ( x ; λ , k ) = ∏ i = 1 N k λ x i λ k − 1 exp − x i λ k . (15) Taking the partial derivative of the logarithm of the likelihood function to be zero, we obtain the following equations, 1 k = ∑ i = 1 N x i k ln x i ∑ i = 1 N x i k ln x i ∑ i = 1 N x i k ∑ i = 1 N x i k − 1 N ∑ i = 1 N ln x i λ k = 1 N ∑ i = 1 N x i k , (16) from which the parameters can be estimated. In order to verify the fitting performance of Weibull distribution, we choose a Brodatz image texture, and model the maximum, mean and median values of local singular values in graph wavelet subbands. The fitting results on LL, LH channels of the first level graph wavelet decomposition are shown in Figure 7. It can be seen that the Weibull distribution works well in the local singular value fitting experiments. 3.4 Feature extraction and classification For an image texture, the feature extraction procedures are summarized as follows, 1) Decompose the image texture with L-level graph wavelet filter banks as shown in Figure 5, and obtain graph wavelet subbands W S l , c ( l = 1 , … , L and c = LL , LH , HL , HH ). 2) In each subband W S l , c , implement SVD on the local region and form the sets F l , c , j , in which j = max , mean , median . 3) Model F l , c , j with Weibull distribution, and estimate the parameters k l , c , j and λ l , c , j . 4) Accumulate all parameters k l , c , j and λ l , c , j into a feature vector to describe the texture. FIGURE 5Open in figure viewerPowerPoint The proposed image texture feature extraction method FIGURE 6Open in figure viewerPowerPoint Local singular value decomposition. We choose 4 × 4 window to calculate local singular values FIGURE 7Open in figure viewerPowerPoint The histograms of the max, mean, median and values of local singular values and the corresponding fitting curves for the Brodatz image texture “d004”. (a) LL-max, (b) LL-mean, (c) LL-median, (d) LH-max, (e) LH-m