# Feature Selection of the Rich Model Based on the Correlation of Feature Components

Authors: Shunhao Jin; Fenlin Liu; Chunfang Yang; Yuanyuan Ma; Yuan Liu
Journal: Security and Communication Networks
 (2021)

---

## Abstract

Currently, the popular Rich Model steganalysis features usually contain a large number of redundant feature components which may bring “curse of dimensionality” and large computation cost, but the existing feature selection methods are difficult to effectively reduce the dimensionality when there are many strongly correlated effective feature components. This paper proposes a novel selection method for Rich Model steganalysis features. First, the separability of each feature component in the submodels of Rich Model is measured based on the Fisher criterion, and the feature components are sorted in the descending order based on the separability. Second, the correlation coefficient between any two feature components in each submodel is calculated, and feature selection is performed according to the Fisher value of each component and the correlation coefficients. Finally, the selected submodels are combined as the final steganalysis feature. The results show that the proposed feature selection method can effectively reduce the dimensionalities of JPEG domain and spatial domain Rich Model steganalysis features without affecting the detection accuracies.


---

## Body

## 1. Introduction

Digital steganography is a technology that embeds information in the redundancy of digital images, audio, video, text, and so on to achieve the purpose of covert communication [1–6]. With the presentation of the HUGO steganography algorithm [7] in 2010, adaptive steganography based on the framework of “distortion function + STC coding” has become the mainstream of image steganography. Based on this framework, researchers have successively proposed a series of adaptive steganography algorithms with high antidetection performance, which make the traditional steganalysis algorithms mostly invalid [8–12]. In 2012, Fridrich and Kodovský proposed the Rich Model steganalysis feature [13], which effectively improved the detection performance for HUGO steganography. Since then, many features with thousands of dimensions have been successively proposed for steganalysis, such as PSRM (Project Spatial Rich Model) features [14], PHARM (Phase-Aware Projection Rich Model) features [15], GFR (Gabor Filter Rich Model) features [16], and CC-JRM (Cartesian-Calibrated JPEG Rich Model) features [17]. These features may bring large computation and storage costs and even the problem of “curse of dimensionality.” In order to reduce the dimensionality of the steganalysis feature, researchers have carried out a series of works in two different ways containing feature transformation and feature selection.Feature dimensionality reduction based on feature transformation generally transforms the feature vector into another feature space so that the effective information would mainly concentrate around some components of the transformed feature, and the most effective feature components are selected from the transformed feature to achieve the purpose of feature dimensionality reduction. For example, Qin et al. [18] used PCA (Principal Component Analysis) to obtain the main components in the feature to achieve the purpose of reducing the feature dimension. However, the PCA method is not suitable for the feature which is not linearly distinguishable [19]. Wang et al. [20] performed one-dimensional discrete Fourier feature transformation on SRM steganalysis feature and then selected the spectral coefficients of the positive half axis as the feature vector, which effectively reduced the feature dimension. Boroumand and Fridrich [21] obtained a class of nonlinear transformations by approximating a symmetric positive semidefinite kernel function and then used them to reduce the dimensionality of feature and improve the detection accuracy at the same time. However, the two methods [20] proposed above are only applicable to the spatial domain steganography.Dimensionality reduction based on feature selection is mainly to select a part of the feature components that can most effectively distinguish the cover images and the stegoimages from the original feature vector. For example, Xuan et al. [22] and Davidson and Jalan [23] used Bhattacharyya distance and Mahalanobis distance to measure the distinguishability of feature components, respectively, and then selected part of feature components with the maximum distances between them of the cover images and the stegoimages. Lu et al. [24] utilized an improved Fisher criterion to measure the importance of feature vectors and then selected the feature subvector with the highest Fisher value for detection. Zhang et al. [25] applied the method in [24] to the reduction of each submodel in SRM. Ma et al. [26] proposed a feature dimensionality reduction method based on decision rough set α-positive region reduction and selected partial feature components that conform to the principle of nonreduction in the positive domain and the principle of independence. However, this method brings large computation cost in the process of positive domain reduction.In a word, the above methods can significantly reduce the dimensionalities of Rich Model steganalysis features. However, the current existing feature selection methods do not take the correlation between feature components into consideration, so they cannot effectively reduce the highly separable and correlated feature components. Aiming at this problem, a feature selection method is proposed based on the correlation of feature components. The experimental results show that the proposed selection method can reduce the dimensionalities of CC-JRM [17] and SRM [13] steganalysis features while maintaining the detection accuracies.The rest of the paper is organized as follows. Section2 introduces the related knowledge involved in the method of this paper, while Section 3 details the feature selection method based on the correlation of feature components in the Rich Model steganalysis features. Accordingly, Section 4 gives the experimental results of this method and finally summarizes the full text.


## 2. Related Knowledge

### 2.1. Pearson Correlation and Fisher Criterion

In statistics, the Pearson correlation coefficient is applied to measure the degree of linear correlation between two variablesX1 and X2, and its value is between −1 and 1. The Pearson correlation coefficient rX1,X2 between two feature components is defined as(1)rX1,X2=covX1,X2σX1σX2=EX1−μ1X2−μ2σX1σX2,where covX1,X2 represents the covariance between two variables X1 and X2. σX1 and σX2 represent the standard deviation of the two variables. μ1 and μ2 represent the mean value of the two variables. When the absolute value of the correlation coefficient rX1,X2 is larger, it means that the correlation between the two variables is stronger.Activated by [7], Lu et al. [24] used a Fisher criterion function to measure the separability of a single feature component as follows:(2)Fscored=μdC−μdS2σdC2+σdS2,where μdC and σdC denote the mean value and standard deviation of the d-th feature components of the features of the cover samples and μdS and σdS denote the mean and standard deviation of the d-th feature components of the features of the corresponding stegosamples, respectively. μdC−μdS2 reflects the degree of dispersion between the d-th feature components computed from the cover and stegosamples. The larger the value is, the greater the difference between cover and stegosamples will be. σdC2+σdS2 reflects the degree of intraclass aggregation of the d-th feature components computed from the cover and stegosamples. Accordingly, the smaller the value of σdC2+σdS2 is, the smaller the intraclass difference will be. In steganalysis, when the Fisher value is larger, it means that the feature component contributes more to the detection of stegoimages.


### 2.2. Rich Model

Currently, the Rich Model steganalysis feature is still one of the most effective methods to detect adaptive steganography. This feature is proposed by Fridrich et al. in 2012 [13], and its extraction procedure is shown in Figure 1. Firstly, a variety of linear and nonlinear high-pass filters are used to filter the image from diverse directions and angles, resulting in various types of residual images. Then, the fourth-order cooccurrence matrices are computing from each residual image along with different directions. Due to the symmetry of the cooccurrence matrix, some cooccurrence matrices are merged to form a new residual cooccurrence matrix. Finally, each new-formed cooccurrence matrix is regarded as a submodel feature, and all submodel features are combined to the final Rich Model steganalysis feature.Figure 1
Extraction procedure of the Rich Model steganalysis feature.


## 2.1. Pearson Correlation and Fisher Criterion

In statistics, the Pearson correlation coefficient is applied to measure the degree of linear correlation between two variablesX1 and X2, and its value is between −1 and 1. The Pearson correlation coefficient rX1,X2 between two feature components is defined as(1)rX1,X2=covX1,X2σX1σX2=EX1−μ1X2−μ2σX1σX2,where covX1,X2 represents the covariance between two variables X1 and X2. σX1 and σX2 represent the standard deviation of the two variables. μ1 and μ2 represent the mean value of the two variables. When the absolute value of the correlation coefficient rX1,X2 is larger, it means that the correlation between the two variables is stronger.Activated by [7], Lu et al. [24] used a Fisher criterion function to measure the separability of a single feature component as follows:(2)Fscored=μdC−μdS2σdC2+σdS2,where μdC and σdC denote the mean value and standard deviation of the d-th feature components of the features of the cover samples and μdS and σdS denote the mean and standard deviation of the d-th feature components of the features of the corresponding stegosamples, respectively. μdC−μdS2 reflects the degree of dispersion between the d-th feature components computed from the cover and stegosamples. The larger the value is, the greater the difference between cover and stegosamples will be. σdC2+σdS2 reflects the degree of intraclass aggregation of the d-th feature components computed from the cover and stegosamples. Accordingly, the smaller the value of σdC2+σdS2 is, the smaller the intraclass difference will be. In steganalysis, when the Fisher value is larger, it means that the feature component contributes more to the detection of stegoimages.


## 2.2. Rich Model

Currently, the Rich Model steganalysis feature is still one of the most effective methods to detect adaptive steganography. This feature is proposed by Fridrich et al. in 2012 [13], and its extraction procedure is shown in Figure 1. Firstly, a variety of linear and nonlinear high-pass filters are used to filter the image from diverse directions and angles, resulting in various types of residual images. Then, the fourth-order cooccurrence matrices are computing from each residual image along with different directions. Due to the symmetry of the cooccurrence matrix, some cooccurrence matrices are merged to form a new residual cooccurrence matrix. Finally, each new-formed cooccurrence matrix is regarded as a submodel feature, and all submodel features are combined to the final Rich Model steganalysis feature.Figure 1
Extraction procedure of the Rich Model steganalysis feature.


## 3. Feature Selection of Rich Model Based on Correlation

In this section, the principle and procedure of the feature selection method of the Rich Model steganalysis features is proposed. Then, an algorithm is proposed to search the proper correlation coefficient threshold.


### 3.1. The Method of Feature Selection

Among the feature components of the high-dimensional Rich Model steganalysis feature, there are some feature components whose variances are 0 in both cover and stegoimages. Particularly, this situation is common in the feature components computed from the quantized DCT coefficients in JPEG image. These feature components have no positive effect on distinguishing the cover and stegoimages. Therefore, the feature components whose variances are 0 in cover samples and stegosamples will be eliminated at first. The obtained features areF¯C and F¯S whose dimensions are D.Next, equation (2) is used to measure the Fisher value of each feature component in F¯Cand F¯S and obtain the corresponding D-dimensional vector of Fisher value Fscore¯. Then the feature components in F¯C and F¯S are sorted in descending order by Fscore¯ to generate the new features F⟶C and F⟶S.Moreover, equation (1) is used to calculate the correlation coefficient matrices of the components in the cover image feature and the components in the stegoimage feature, respectively. The correlation coefficient matrices are denoted as follows:(3)RC=r11c…r1Dc⋮⋱⋮rD1c⋯rDDc,RS=r11s…r1Ds⋮⋱⋮rD1s⋯rDDs,where RC and RS are both symmetric matrices with size of D × D and the element rijc and rijs represent the strength of the linear correlation between the i-th feature component and the j-th feature component of cover image feature and stegoimage feature, respectively, −1≤rijC,rijS≤1. When rijc=−1 and rijs=−1, the two feature components are fully negative linear correlation. Correspondingly, when rijc=1 and rijs=1, the two feature components are fully positive linear correlation.Finally, sinceRC and RS are symmetric matrices, it is only necessary to traverse the elements below (above) the main diagonal in order, such as r11c⟶⋯⟶rD1c⟶r22c⟶⋯⟶rijc⟶⋯⟶rDD−1c⟶rDDc(i≥j). Then, when the element satisfies with equation (4), the feature component group fi,fj is reduced to one element.(4)rijc>T,rijs>T,rijc×rijs>0,where T is a preset correlation coefficient threshold. According to equation (4), the selection rules of feature components are provided.(1)
Whenrijc>T and rijs>T, there is a strong positive linear correlation between fi and fj, feature component fj is retained, and feature component fi is removed(2)
Whenrijc<−T and rijs<−T, there is a strong negative linear correlation between fi and fj, feature component fj is retained, and feature component fi is removed(3)
Whenrijc>T and rijs<−T (or rijc<−T and rijs>T), both feature components need to be retained(4)
When−T<rijc<T or −T<rijs<T, both feature components need to be retainedThe selected features based on rules are denoted asFTC⟶ and FTS⟶.Based on the above rules, the main procedure of the proposed method is shown in Figure2. Taking the SRM [13] as an example, firstly, the high-dimensional SRM steganalysis feature is divided into 106 submodel features. Then, each submodel removes the feature components whose variance is 0 in cover and stegoimage. Next, feature components of each submodel are sorted according to the Fisher value obtained by equation (2). After that, the effective feature components are selected based on the proposed feature selection rules. Finally, the feature components selected from each submodel are combined to a final feature for steganalysis.Figure 2
Feature selection framework based on the correlation of feature components.


### 3.2. Selection Algorithm of Correlation Coefficient Threshold T

In the feature selection process, the correlation coefficient thresholdT is of critical significance. When the value of T is larger, the dimensionality of the selected feature is smaller, and vice versa. The selection of the threshold T should follow the idea that the selected feature components can retain the diversity of the original feature as much as possible, and the redundant feature components can be removed as much as possible. In addition, in order to ensure the finiteness of the process, the significant digits of T are set as 5. Algorithm 1 describes the steps for selecting the proper T.Algorithm 1:Selection algorithm of correlation coefficient threshold T.
(i)
Input: steganalysis feature of cover imagesFC, steganalysis feature of stegoimages FS.(ii)
Output: the proper correlation coefficient thresholdTpro.(iii)
Step 1: calculate the initial detection accuracy. According toT∈0.9,1, obtain 9 values T10.91, T20.92, ……, T90.99 with an interval of 0.01, and calculate their corresponding detection accuracy A1, A2, ……, A9and the detection accuracy of original feature A0.(iv)
Step 2: compare detection performance. Compare withA0 in the order of A1⟶A2⟶⋯⟶A9. When Ai−A0>−0.005, stop subsequent comparisons and assign i−1 and i to a and b, respectively.(v)
Step 3: reduce the selection range ofT. Let Tc=Ta+Tb/2, calculate its corresponding detection accuracy Ac, and compare Ac with Ab. If Ac>Ab, then assign Tc to Tb; otherwise, assign Tc to Ta.(vi)
Step 4: repeat Step 3 until the significant digits ofTc reach 5, and stop the operation and regard Tc as the optimal truncation threshold Tpro.(vii)Step 5: ReturnTpro.


## 3.1. The Method of Feature Selection

Among the feature components of the high-dimensional Rich Model steganalysis feature, there are some feature components whose variances are 0 in both cover and stegoimages. Particularly, this situation is common in the feature components computed from the quantized DCT coefficients in JPEG image. These feature components have no positive effect on distinguishing the cover and stegoimages. Therefore, the feature components whose variances are 0 in cover samples and stegosamples will be eliminated at first. The obtained features areF¯C and F¯S whose dimensions are D.Next, equation (2) is used to measure the Fisher value of each feature component in F¯Cand F¯S and obtain the corresponding D-dimensional vector of Fisher value Fscore¯. Then the feature components in F¯C and F¯S are sorted in descending order by Fscore¯ to generate the new features F⟶C and F⟶S.Moreover, equation (1) is used to calculate the correlation coefficient matrices of the components in the cover image feature and the components in the stegoimage feature, respectively. The correlation coefficient matrices are denoted as follows:(3)RC=r11c…r1Dc⋮⋱⋮rD1c⋯rDDc,RS=r11s…r1Ds⋮⋱⋮rD1s⋯rDDs,where RC and RS are both symmetric matrices with size of D × D and the element rijc and rijs represent the strength of the linear correlation between the i-th feature component and the j-th feature component of cover image feature and stegoimage feature, respectively, −1≤rijC,rijS≤1. When rijc=−1 and rijs=−1, the two feature components are fully negative linear correlation. Correspondingly, when rijc=1 and rijs=1, the two feature components are fully positive linear correlation.Finally, sinceRC and RS are symmetric matrices, it is only necessary to traverse the elements below (above) the main diagonal in order, such as r11c⟶⋯⟶rD1c⟶r22c⟶⋯⟶rijc⟶⋯⟶rDD−1c⟶rDDc(i≥j). Then, when the element satisfies with equation (4), the feature component group fi,fj is reduced to one element.(4)rijc>T,rijs>T,rijc×rijs>0,where T is a preset correlation coefficient threshold. According to equation (4), the selection rules of feature components are provided.(1)
Whenrijc>T and rijs>T, there is a strong positive linear correlation between fi and fj, feature component fj is retained, and feature component fi is removed(2)
Whenrijc<−T and rijs<−T, there is a strong negative linear correlation between fi and fj, feature component fj is retained, and feature component fi is removed(3)
Whenrijc>T and rijs<−T (or rijc<−T and rijs>T), both feature components need to be retained(4)
When−T<rijc<T or −T<rijs<T, both feature components need to be retainedThe selected features based on rules are denoted asFTC⟶ and FTS⟶.Based on the above rules, the main procedure of the proposed method is shown in Figure2. Taking the SRM [13] as an example, firstly, the high-dimensional SRM steganalysis feature is divided into 106 submodel features. Then, each submodel removes the feature components whose variance is 0 in cover and stegoimage. Next, feature components of each submodel are sorted according to the Fisher value obtained by equation (2). After that, the effective feature components are selected based on the proposed feature selection rules. Finally, the feature components selected from each submodel are combined to a final feature for steganalysis.Figure 2
Feature selection framework based on the correlation of feature components.


## 3.2. Selection Algorithm of Correlation Coefficient Threshold T

In the feature selection process, the correlation coefficient thresholdT is of critical significance. When the value of T is larger, the dimensionality of the selected feature is smaller, and vice versa. The selection of the threshold T should follow the idea that the selected feature components can retain the diversity of the original feature as much as possible, and the redundant feature components can be removed as much as possible. In addition, in order to ensure the finiteness of the process, the significant digits of T are set as 5. Algorithm 1 describes the steps for selecting the proper T.Algorithm 1:Selection algorithm of correlation coefficient threshold T.
(i)
Input: steganalysis feature of cover imagesFC, steganalysis feature of stegoimages FS.(ii)
Output: the proper correlation coefficient thresholdTpro.(iii)
Step 1: calculate the initial detection accuracy. According toT∈0.9,1, obtain 9 values T10.91, T20.92, ……, T90.99 with an interval of 0.01, and calculate their corresponding detection accuracy A1, A2, ……, A9and the detection accuracy of original feature A0.(iv)
Step 2: compare detection performance. Compare withA0 in the order of A1⟶A2⟶⋯⟶A9. When Ai−A0>−0.005, stop subsequent comparisons and assign i−1 and i to a and b, respectively.(v)
Step 3: reduce the selection range ofT. Let Tc=Ta+Tb/2, calculate its corresponding detection accuracy Ac, and compare Ac with Ab. If Ac>Ab, then assign Tc to Tb; otherwise, assign Tc to Ta.(vi)
Step 4: repeat Step 3 until the significant digits ofTc reach 5, and stop the operation and regard Tc as the optimal truncation threshold Tpro.(vii)Step 5: ReturnTpro.


## 4. Experimental Results and Analysis

In this section, the proposed feature selection method was tested on typical JPEG image steganalysis feature CC-JRM and spatial image steganalysis feature SRM for the steganography algorithms J-UNIWARD and S-UNIWARD.


### 4.1. Experimental Setting

All experiments were conducted on the BOSSbase 1.01 database. The database contains 10,000 grayscale images with a size of 512 × 512, and the format of them is PGM. First, 10,000 grayscale images were used to generate stegoimages by S-UNIWARD with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5. Then, SRM steganalysis features were extracted from the cover images and stegoimages. Finally, six groups of SRM steganalysis features (34671 dimensions) [13] were obtained.On the other way, 10,000 grayscale images were compressed by JPEG standard with a quality factor of 75 to generate the cover JPEG images. The 10,000 cover JPEG images were used to generate stegoimages by J-UNIWARD with payloads of 0.1,0.2,0.3,0.4, and 0.5. Then, six groups of CC-JRM steganalysis features (22510 dimensions) [17] were extracted from the cover images and stegoimages.In experiments, an ensemble classifier [27] was trained. For each payload, cover images and stegoimages were divided into two groups; one group was used for training which contained 5000 randomly selected cover images and 5000 corresponding stegoimages. The remaining images composed the testing group. The performance of steganalysis is evaluated by PE¯ which is the mean value of PE over 10 tests. PE is computed by the false alarm and missed detection as follows:(5)PE=minPFAPFA+PMDPFA2,where PFA represents the false alarm, namely, the probability of judging a cover image as a stegoimage, and PMD represents the missed detection, namely, the probability of judging a stegoimage as a cover image. Besides, the average detection accuracy PA¯=1−PE¯ was used to reflect the detection performance more intuitively. The larger PA¯ is, the better the performance of steganalysis is.


### 4.2. Effectiveness Analysis of Feature Selection

This section will explore the relationship between dimensions of the selected feature and the detection performance in more aspects. The MMD (Maximum Mean Discrepancy) mentioned in [28] will be used to measure the similarity between the feature distributions of cover images and stegoimages, so as to describe the classification performance of the selected feature. Taking X and Y samples as an example, the formula of MMD is as follows:(6)MMDX,Y=1m2∑i,j=1mkxi,xj−2mn∑i,j=1m,nkxi,yi+1n2∑i,j=1nkyi,yj1/2,where xi represents the feature extracted from the i-th cover image, correspondingly, yi represents the feature extracted from the i-th stegoimage, m and n, respectively, represent the number of cover images and stegoimages, and k⋅,⋅ is the radial basis function (RBF). When the MMD value between X and Y is smaller, the feature distributions of cover images and stegoimages are more similar, and the performance of classification is worse, and vice versa.Figure3 shows the scatter plot of the MMD and detection accuracy for different numbers of feature components selected from the Ah_T3 submodel feature in the CC-JRM steganalysis feature [17] by the proposed selection method. The results were computed on the cover JPEG images and their stegoversions of J-UNIWARD with a payload of 0.5. The red “▲” represents the MMD values of the selected feature components, and the blue “●” represents the corresponding detection accuracy. It can be found that as the number of the selected feature components increases, the MMD value and detection accuracy also increase. When the number of selected feature components reaches a certain range, the MMD value and detection accuracy tend to be flat, which means that the subsequent addition of strongly correlated feature components does not bring much improvement to the MMD value and detection accuracy.Figure 3
The classification performance of the Ah_T3 submodel.Figure4 shows the scatter plot of the MMD value and detection accuracy for different numbers of feature components selected from the s1_minmax22v_q1 submodel feature in the SRM steganalysis feature [13]. The used stegoimages were generated by the S-UNIWARD algorithm with a payload of 0.5. The corresponding result can be found in Figure 4.Figure 4
The classification performance of the selected s1_minmax22v_q1 submodel.Additionally, we also selected the feature components from the Dix2_T2 submodel of CC-JRM steganalysis feature [17] and the s1-minmax22h_q1 submodel of SRM steganalysis feature [13] and then compared the performances of the selected features for different payloads with randomly selected features. In order to obtain ideal selected features, proper T was searched by Algorithm 1. The experimental results are shown in Tables 1 and 2. It can be seen that the proposed feature selection method nearly maintains the original detection accuracy, and the detection accuracy is higher than that of the randomly selected feature for each case. This should be attributed to the following two reasons. On the one hand, strongly correlated feature components are redundant, and they do not affect the detection performance. On the other hand, the Fisher criterion ensures that the retained feature components are effective instead of randomly selecting.Table 1
The performances of the original Dix2_T2 submodel of CC-JRM and the subfeatures selected by different methods.
PayloadPA¯(260-D)RandomProposedNumber of feature componentsPA¯Number of feature componentsPA¯0.10.5033880.5020880.50460.20.5073870.5065870.50820.30.5167720.5098720.51510.40.5287700.5189700.52790.50.5469860.5324860.5491Table 2
The performances of the original s1-minmax22h submodel of SRM and the subfeatures selected by different methods.
PayloadPA¯(325-D)RandomProposedNumber of feature componentsPA¯Number of feature componentsPA¯0.10.5397820.5162820.53910.20.5873810.5567810.58280.30.6278800.5432800.62660.40.6709840.5994840.66860.50.7053810.6274810.7046


### 4.3. Performance Test of the Feature Selection Method for CC-JRM Steganalysis Features

The CC-JRM steganalysis feature proposed by Fridrich et al. is a typical Rich Model steganalysis feature for JPEG image steganography, which can effectively detect the common JPEG image steganography algorithms, such as J-UNIWARD [29], UED [30], and nsF5 [31]. Under different threshold T, the proposed method was used to select feature components from CC-JRM features of cover and stegoimages with a payload of 0.1. Table 3 shows the number and detection accuracy of the selected feature components.Table 3
The number and detection accuracy of the feature components selected from CC-JRM steganalysis features under different thresholdT.
Number of feature components22510 (original feature)1556710635798563885265443638893050Detection accuracy (%)53.0653.2353.2953.3553.3953.3253.4853.3252.52T10.990.980.970.960.950.938930.930.92It can be seen from Table3 that the detection accuracy of the original 22510-D CC-JRM in steganalysis is 53.06%. The optimal detection accuracy of selected features can reach 53.48%, which is slightly higher than that of the original features. Meanwhile, the number of these selected feature components is 4436, which is only 19.7% of the number of original feature components. It demonstrates that the proposed method can effectively reduce the feature dimension while maintaining the detection accuracy of steganalysis.Figure5 shows the detection accuracy of CC-JRM before and after feature selection with a payload of 0.1. The horizontal axis represents the number of feature components, and the vertical axis represents the detection accuracy. The blue “▲” represents the detection accuracy of the selected most proper feature. The green “▼” represents the detection accuracy of the original feature. It can be inducted from the figure that when the feature components are insufficient, the detection accuracy performance is not desirable because some valid features are not selected. Meanwhile, when the feature components are retained too much, some redundant and even harmful feature components may be brought. Moreover, retaining too many redundant feature components will increase the time and space complexity of training and classification. And as the number of the feature components gradually approaches the original feature dimension, the detection accuracy of steganalysis will not improve significantly.Figure 5
Detection accuracy of CC-JRM before and after feature selection with a payload of 0.1.In order to test the performance of the method under other payloads, the paper also conducted experiments with the payloads of 0.2, 0.3, 0.4, and 0.5, and the results are shown in Figure6.Figure 6
Detection accuracies of CC-JRM before and after feature selection with different payloads: (a) payload = 0.2, (b) payload = 0.3, (c) payload = 0.4, and (d) payload = 0.5.
(a)(b)(c)(d)Figures6(a)∼6(d) are the detection accuracy of CC-JRM before and after feature selection with payloads of 0.2, 0.3, 0.4, and 0.5. The same as Figure 5, the horizontal axis represents the number of feature components, and the vertical axis represents detection accuracy. From Figures 6(b) to 6(d), it can be seen that the dimensionality of steganalysis feature can be effectively reduced while the detection accuracy is not significantly affected, and the difference between the detection accuracies before and after selection is within 0.4%. In Figure 6(a), it can be found that the detection accuracy is even improved in the case of low payload.The feature selection method proposed in this paper is also compared with those of Zhang et al. [25] and Ma et al. [26]. The specific results are shown in Figures 7 and 8. The four different colors in the figures represent the experimental results of different feature selection methods. The detection accuracies of three selection methods are not much different from the original features. However, the dimensionality of the selected feature by the proposed method is significantly lower than those of the other methods. The reason is that the proposed method eliminates redundant feature components strongly correlated to the remained feature components, but the other two methods just select effective feature components and do not consider the redundant relation between feature components. On the other hand, Ma’s method selects the optimal feature components from overall features and maximizes the retention of beneficial feature components in the original features. Therefore, although the dimensionality reduction of Ma’s method is not as good as the proposed method and Zhang’s method, the detection performance of selected features by Ma’s method is slightly better than theirs.Figure 7
Comparison of dimensionalities of selected CC-JRM steganalysis features under different selection methods with different payloads.Figure 8
Comparison of detection accuracies of selected CC-JRM steganalysis features under different selection methods with different payloads.


### 4.4. Performance Test of the Feature Selection Method for SRM Steganalysis Features

The previous section has tested the effectiveness of the proposed method in JPEG domain. In order to explore the generalization ability of the method, this section tests the effectiveness of this method in the spatial domain. The same as in JPEG domain, this section sets the experimental groups with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5 (bbp), respectively, and the experimental results are shown in Figure9.Figure 9
Detection accuracies of SRM before and after feature selection with different payloads: (a) payload = 0.1, (b) payload = 0.2, (c) payload = 0.3, (d) payload = 0.4, and (e) payload = 0.5.
(a)(b)(c)(d)(e)As in the previous section, Figures9(a)∼9(e) provide the detection accuracy of SRM before and after feature selection with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5, respectively. The horizontal axis represents the feature number, and the vertical axis represents detection accuracy. The blue “▲” represents detection accuracy of the most proper feature after selection, and the green “▼” represents detection accuracy of the original feature. It can be seen from Figure 9 that, with the gradual increase of the feature number, the detection accuracy of steganalysis will also increase. When the feature number increases to a certain value, the detection accuracy of steganalysis will stabilize. Experiments show that the difference of the detection accuracy between the selected feature by the proposed method and the original feature is within 0.3%. Moreover, the dimensionality of the selected feature is far lower than that of the original feature (less than 50%). Based on the above analysis, the proposed method is still effective in the spatial domain.In addition, the comparison results between the proposed method and the method proposed by Zhang et al. [25] and Ma et al. [26] are approximately the same as in JPEG domain. The experimental results are shown in Figure 10.Figure 10
Comparison of dimensionalities of selected SRM steganalysis features under different selection methods with different payloads.It can be seen that the dimensionalities of selected features by the proposed method are obviously lower than those of selected features by the other two methods. In the case of low payloads, the comparison is particularly conspicuous.In summary, the feature selection method proposed in this paper can effectively reduce the JPEG domain feature dimension and spatial domain feature dimension. Moreover, for low payloads of JPEG image steganography, the selected JPEG domain feature has a certain improvement in detection accuracy compared with the original feature. In addition, the dimensionality reduction of JPEG domain feature is greater than that of the spatial steganalysis feature. The main reason is that the JPEG domain Rich Model steganalysis feature has a stronger linear correlation.


## 4.1. Experimental Setting

All experiments were conducted on the BOSSbase 1.01 database. The database contains 10,000 grayscale images with a size of 512 × 512, and the format of them is PGM. First, 10,000 grayscale images were used to generate stegoimages by S-UNIWARD with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5. Then, SRM steganalysis features were extracted from the cover images and stegoimages. Finally, six groups of SRM steganalysis features (34671 dimensions) [13] were obtained.On the other way, 10,000 grayscale images were compressed by JPEG standard with a quality factor of 75 to generate the cover JPEG images. The 10,000 cover JPEG images were used to generate stegoimages by J-UNIWARD with payloads of 0.1,0.2,0.3,0.4, and 0.5. Then, six groups of CC-JRM steganalysis features (22510 dimensions) [17] were extracted from the cover images and stegoimages.In experiments, an ensemble classifier [27] was trained. For each payload, cover images and stegoimages were divided into two groups; one group was used for training which contained 5000 randomly selected cover images and 5000 corresponding stegoimages. The remaining images composed the testing group. The performance of steganalysis is evaluated by PE¯ which is the mean value of PE over 10 tests. PE is computed by the false alarm and missed detection as follows:(5)PE=minPFAPFA+PMDPFA2,where PFA represents the false alarm, namely, the probability of judging a cover image as a stegoimage, and PMD represents the missed detection, namely, the probability of judging a stegoimage as a cover image. Besides, the average detection accuracy PA¯=1−PE¯ was used to reflect the detection performance more intuitively. The larger PA¯ is, the better the performance of steganalysis is.


## 4.2. Effectiveness Analysis of Feature Selection

This section will explore the relationship between dimensions of the selected feature and the detection performance in more aspects. The MMD (Maximum Mean Discrepancy) mentioned in [28] will be used to measure the similarity between the feature distributions of cover images and stegoimages, so as to describe the classification performance of the selected feature. Taking X and Y samples as an example, the formula of MMD is as follows:(6)MMDX,Y=1m2∑i,j=1mkxi,xj−2mn∑i,j=1m,nkxi,yi+1n2∑i,j=1nkyi,yj1/2,where xi represents the feature extracted from the i-th cover image, correspondingly, yi represents the feature extracted from the i-th stegoimage, m and n, respectively, represent the number of cover images and stegoimages, and k⋅,⋅ is the radial basis function (RBF). When the MMD value between X and Y is smaller, the feature distributions of cover images and stegoimages are more similar, and the performance of classification is worse, and vice versa.Figure3 shows the scatter plot of the MMD and detection accuracy for different numbers of feature components selected from the Ah_T3 submodel feature in the CC-JRM steganalysis feature [17] by the proposed selection method. The results were computed on the cover JPEG images and their stegoversions of J-UNIWARD with a payload of 0.5. The red “▲” represents the MMD values of the selected feature components, and the blue “●” represents the corresponding detection accuracy. It can be found that as the number of the selected feature components increases, the MMD value and detection accuracy also increase. When the number of selected feature components reaches a certain range, the MMD value and detection accuracy tend to be flat, which means that the subsequent addition of strongly correlated feature components does not bring much improvement to the MMD value and detection accuracy.Figure 3
The classification performance of the Ah_T3 submodel.Figure4 shows the scatter plot of the MMD value and detection accuracy for different numbers of feature components selected from the s1_minmax22v_q1 submodel feature in the SRM steganalysis feature [13]. The used stegoimages were generated by the S-UNIWARD algorithm with a payload of 0.5. The corresponding result can be found in Figure 4.Figure 4
The classification performance of the selected s1_minmax22v_q1 submodel.Additionally, we also selected the feature components from the Dix2_T2 submodel of CC-JRM steganalysis feature [17] and the s1-minmax22h_q1 submodel of SRM steganalysis feature [13] and then compared the performances of the selected features for different payloads with randomly selected features. In order to obtain ideal selected features, proper T was searched by Algorithm 1. The experimental results are shown in Tables 1 and 2. It can be seen that the proposed feature selection method nearly maintains the original detection accuracy, and the detection accuracy is higher than that of the randomly selected feature for each case. This should be attributed to the following two reasons. On the one hand, strongly correlated feature components are redundant, and they do not affect the detection performance. On the other hand, the Fisher criterion ensures that the retained feature components are effective instead of randomly selecting.Table 1
The performances of the original Dix2_T2 submodel of CC-JRM and the subfeatures selected by different methods.
PayloadPA¯(260-D)RandomProposedNumber of feature componentsPA¯Number of feature componentsPA¯0.10.5033880.5020880.50460.20.5073870.5065870.50820.30.5167720.5098720.51510.40.5287700.5189700.52790.50.5469860.5324860.5491Table 2
The performances of the original s1-minmax22h submodel of SRM and the subfeatures selected by different methods.
PayloadPA¯(325-D)RandomProposedNumber of feature componentsPA¯Number of feature componentsPA¯0.10.5397820.5162820.53910.20.5873810.5567810.58280.30.6278800.5432800.62660.40.6709840.5994840.66860.50.7053810.6274810.7046


## 4.3. Performance Test of the Feature Selection Method for CC-JRM Steganalysis Features

The CC-JRM steganalysis feature proposed by Fridrich et al. is a typical Rich Model steganalysis feature for JPEG image steganography, which can effectively detect the common JPEG image steganography algorithms, such as J-UNIWARD [29], UED [30], and nsF5 [31]. Under different threshold T, the proposed method was used to select feature components from CC-JRM features of cover and stegoimages with a payload of 0.1. Table 3 shows the number and detection accuracy of the selected feature components.Table 3
The number and detection accuracy of the feature components selected from CC-JRM steganalysis features under different thresholdT.
Number of feature components22510 (original feature)1556710635798563885265443638893050Detection accuracy (%)53.0653.2353.2953.3553.3953.3253.4853.3252.52T10.990.980.970.960.950.938930.930.92It can be seen from Table3 that the detection accuracy of the original 22510-D CC-JRM in steganalysis is 53.06%. The optimal detection accuracy of selected features can reach 53.48%, which is slightly higher than that of the original features. Meanwhile, the number of these selected feature components is 4436, which is only 19.7% of the number of original feature components. It demonstrates that the proposed method can effectively reduce the feature dimension while maintaining the detection accuracy of steganalysis.Figure5 shows the detection accuracy of CC-JRM before and after feature selection with a payload of 0.1. The horizontal axis represents the number of feature components, and the vertical axis represents the detection accuracy. The blue “▲” represents the detection accuracy of the selected most proper feature. The green “▼” represents the detection accuracy of the original feature. It can be inducted from the figure that when the feature components are insufficient, the detection accuracy performance is not desirable because some valid features are not selected. Meanwhile, when the feature components are retained too much, some redundant and even harmful feature components may be brought. Moreover, retaining too many redundant feature components will increase the time and space complexity of training and classification. And as the number of the feature components gradually approaches the original feature dimension, the detection accuracy of steganalysis will not improve significantly.Figure 5
Detection accuracy of CC-JRM before and after feature selection with a payload of 0.1.In order to test the performance of the method under other payloads, the paper also conducted experiments with the payloads of 0.2, 0.3, 0.4, and 0.5, and the results are shown in Figure6.Figure 6
Detection accuracies of CC-JRM before and after feature selection with different payloads: (a) payload = 0.2, (b) payload = 0.3, (c) payload = 0.4, and (d) payload = 0.5.
(a)(b)(c)(d)Figures6(a)∼6(d) are the detection accuracy of CC-JRM before and after feature selection with payloads of 0.2, 0.3, 0.4, and 0.5. The same as Figure 5, the horizontal axis represents the number of feature components, and the vertical axis represents detection accuracy. From Figures 6(b) to 6(d), it can be seen that the dimensionality of steganalysis feature can be effectively reduced while the detection accuracy is not significantly affected, and the difference between the detection accuracies before and after selection is within 0.4%. In Figure 6(a), it can be found that the detection accuracy is even improved in the case of low payload.The feature selection method proposed in this paper is also compared with those of Zhang et al. [25] and Ma et al. [26]. The specific results are shown in Figures 7 and 8. The four different colors in the figures represent the experimental results of different feature selection methods. The detection accuracies of three selection methods are not much different from the original features. However, the dimensionality of the selected feature by the proposed method is significantly lower than those of the other methods. The reason is that the proposed method eliminates redundant feature components strongly correlated to the remained feature components, but the other two methods just select effective feature components and do not consider the redundant relation between feature components. On the other hand, Ma’s method selects the optimal feature components from overall features and maximizes the retention of beneficial feature components in the original features. Therefore, although the dimensionality reduction of Ma’s method is not as good as the proposed method and Zhang’s method, the detection performance of selected features by Ma’s method is slightly better than theirs.Figure 7
Comparison of dimensionalities of selected CC-JRM steganalysis features under different selection methods with different payloads.Figure 8
Comparison of detection accuracies of selected CC-JRM steganalysis features under different selection methods with different payloads.


## 4.4. Performance Test of the Feature Selection Method for SRM Steganalysis Features

The previous section has tested the effectiveness of the proposed method in JPEG domain. In order to explore the generalization ability of the method, this section tests the effectiveness of this method in the spatial domain. The same as in JPEG domain, this section sets the experimental groups with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5 (bbp), respectively, and the experimental results are shown in Figure9.Figure 9
Detection accuracies of SRM before and after feature selection with different payloads: (a) payload = 0.1, (b) payload = 0.2, (c) payload = 0.3, (d) payload = 0.4, and (e) payload = 0.5.
(a)(b)(c)(d)(e)As in the previous section, Figures9(a)∼9(e) provide the detection accuracy of SRM before and after feature selection with payloads of 0.1, 0.2, 0.3, 0.4, and 0.5, respectively. The horizontal axis represents the feature number, and the vertical axis represents detection accuracy. The blue “▲” represents detection accuracy of the most proper feature after selection, and the green “▼” represents detection accuracy of the original feature. It can be seen from Figure 9 that, with the gradual increase of the feature number, the detection accuracy of steganalysis will also increase. When the feature number increases to a certain value, the detection accuracy of steganalysis will stabilize. Experiments show that the difference of the detection accuracy between the selected feature by the proposed method and the original feature is within 0.3%. Moreover, the dimensionality of the selected feature is far lower than that of the original feature (less than 50%). Based on the above analysis, the proposed method is still effective in the spatial domain.In addition, the comparison results between the proposed method and the method proposed by Zhang et al. [25] and Ma et al. [26] are approximately the same as in JPEG domain. The experimental results are shown in Figure 10.Figure 10
Comparison of dimensionalities of selected SRM steganalysis features under different selection methods with different payloads.It can be seen that the dimensionalities of selected features by the proposed method are obviously lower than those of selected features by the other two methods. In the case of low payloads, the comparison is particularly conspicuous.In summary, the feature selection method proposed in this paper can effectively reduce the JPEG domain feature dimension and spatial domain feature dimension. Moreover, for low payloads of JPEG image steganography, the selected JPEG domain feature has a certain improvement in detection accuracy compared with the original feature. In addition, the dimensionality reduction of JPEG domain feature is greater than that of the spatial steganalysis feature. The main reason is that the JPEG domain Rich Model steganalysis feature has a stronger linear correlation.


## 5. Conclusion

At present, the Rich Model steganalysis features have favorable detection results for adaptive steganography; however, the Rich Model steganalysis features have disadvantages of high dimensionality and slow training. In order to reduce the computation cost of training brought by the high-dimensional Rich Model steganalysis features, a new Rich Model steganalysis feature selection method is proposed based on the correlation of feature components. The experimental results show that the proposed method can effectively reduce the dimensionalities of features while maintaining the detection accuracy of steganalysis.Although the method in this paper has taken the correlation between feature components into consideration, the positive role of the complementarity between feature components in steganalysis is still worthy of further study. In the next step, we will also try to combine the proposed method with deep learning in steganalysis [32] and even extend it to other fields, such as visual search [33].



---

*Source: 6680528-2021-04-29.xml*