# Cultural Tourism Industry Feature Extraction Based on Multiscale Feature Fusion Algorithm and Artificial Intelligence

Authors: Wang Fang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

The cultural tourism industry combines the similarities of the cultural industry and tourism industry, which can be the most effective way to meet people's spiritual and cultural needs as well as their leisure needs simultaneously, and has a vast development potential. However, there are numerous and dispersed areas where the market value of cultural tourism resources is clustered, and frequently, each city has the clustering area with the highest market value concentration of cultural tourism resources. This feature of the spatial distribution of the market value of cultural tourism resources is significant for promoting the development of cultural tourism as a whole and constructing the industry's overall structure. It has broad application potential for extracting and differentiating cultural tourism industry characteristics. Texture feature extraction is typically performed using dual-tree complex wavelet transform (DT-CWT) and Gabor wavelet. In this paper, we propose a multiscale DT-CWT and Gabor-based method for identifying the cultural tourism industry. The method first decomposes the images of cultural tourism into multiscale space using a Gaussian pyramid, then extracts the multiscale features of the images using DT-CWT and Gabor, and lastly achieves feature fusion. Using a support vector machine (SVM) classifier to achieve classification, the effectiveness of the feature extraction method is determined. The experimental findings demonstrate that the method proposed in this paper can achieve a high rate of recognition.


---

## Body

## 1. Introduction

Cultural tourism resources are the core content of modern tourism development. Precisely selected cultural tourism resources, creative boutique products suitable for market demand, have become the focus of attention from all walks of life. The scientific classification and value assessment of cultural tourism resources are the basis for supporting the development of cultural tourism boutique and therefore should be given high attention [1].First, the uniqueness of the resource is not strong enough. Many local cultural tourism resources have this problem.Second, the destruction of resources is serious. In urban construction projects, there is a widespread overdevelopment, destructive development, so much so that the construction-oriented ecological environment and historical and cultural resources suffered damage and other problems are very serious [2].Third, the audience of the resources is not broad enough. Cultural tourism resources, the core or culture, in addition to a small number of very strong resources can be art, history, music, and other human common cultural languages to achieve a wide range of identity; many resources are regional and limited. The limitation of language, expression, and regional culture has led to the fact that the audience that can accept and identify with these cultures is not broad enough to form attractive tourism resources. Examples include various local dramas and the costume patterns of various ethnic minorities.Fourth, the wisdom tourism system does not pay enough attention to cultural tourism resources. The focus of intelligent tourism is still on specific matters such as tourism e-government, tourism e-commerce, and digital scenic spots, while the vision of redistributing, combining, processing, disseminating, and selling the components of the tourism industry chain to promote the transformation of traditional tourism into modern tourism and accelerate the development of tourism has not been realized. Specific to cultural tourism resources is the lack of systematic and scientific planning for this aspect. Except for a few excellent tourism enterprises that are aware of this and have begun to create a holistic approach to cultural resources, the vast majority of tourism enterprises are still at the stage of posting a few articles or pictures online to introduce the local cultural characteristics and do not have a long-term, strategic plan to create a strategic brand of cultural tourism resources.Current image texture feature extraction techniques include symbiotic matrix, wavelet transformation, and others. However, the majority of the literature focuses on single-scale images, and multiscale image features have received less attention. Multiscale representation of images can enhance texture analysis, as can the extraction of multiscale texture features. Researchers have proposed some feature extraction models employing the Gabor transform and combining it with other algorithms in response to the popularity of the Gabor algorithm. Experiments demonstrated the method's effective classification capabilities. Literature [3] decomposes the image into multiscale space by Gaussian pyramid and then obtains LBP images of each scale space and on this basis extracts grayscale cogeneration matrix texture features and then uses support vector machine (SVM) classifier for classification, and the experimental results indicate that the method has a good discriminative effect. Dual-tree complex wavelet transform (DT-CWT) and Gabor transform are the two most fundamental extraction methods for texture features [4]. DT-CWT overcomes the limitations of traditional discrete wavelet transform in terms of translation sensitivity and frequency aliasing, possesses translation invariance and antialiasing, and can describe the directional information of image texture more comprehensively, whereas Gabor transform is extremely sensitive to local subtle changes in image. The flowchart of the multiscale DT-CWT and Gabor features fusion and classification algorithm is decomposed into multiscale space, and then DT-CWT and Gabor filters are applied to each scale image to characterize the texture features using the image's mean and variance. To achieve feature fusion, the extracted features from the two methods are concatenated into a vector set. The SVM classifier is finally used for classification. Figure 1 depicts the algorithm flowchart for the proposed method. First, input the decomposition vector into DT-CWT and Gabor, fuse after feature extraction, and finally input it into SVM to get the classification result.Figure 1
Algorithm flow base on multiscale feature fusion and classification.In this paper, we use a multiscale fusion approach for feature extraction of image video aspects of cultural resources and subsequent fusion and classification for features, which is important for our study of feature distribution of cultural resources, and here our contribution is as follows:(1)
Combining some properties of DT-CWT and Gabor wavelet for texture feature extraction, a multiscale fusion technique for cultural resource feature extraction is proposed.(2)
Images are decomposed into multiscale spaces by Gaussian pyramids, the features of each layer are fused, and an SVM classifier with an RBF kernel function is used for classification.(3)
From the experimental results, the proposed method in this paper has better recognition effect than the features extracted by other methods.


## 2. Related Work

### 2.1. Concept of Cultural Tourism Resources

The initial definition of resources is the original endowment of nature, which was subsequently expanded to include natural and cultural resources, and the utility characteristics of resources were clarified. Natural resources are the combination of natural environmental factors and conditions that can generate economic value under certain conditions of time and place to enhance the current and future welfare of humans; cultural resources are the material and spiritual products that condense the undifferentiated labor of humans, including the material and immaterial cultural wealth accumulated over the course of history [5]. Tourism resources are the concept of resources based on functional attributes, which refers to the sum of various things and factors that exist in a certain area, can be attractive to tourists, can be used by the tourism industry, and produce economic, social, and environmental benefits. Tourism resources include natural and human tourism resources that have been exploited and potential natural and human tourism resources with tourism attractiveness to be developed. Tourism resources include the subconcept of cultural tourism resources. In a narrow sense, cultural tourism resources are tourism resources that organically combine culture and tourism; in a broader sense, cultural tourism resources are all tourism resources that can provide cultural experiences for tourists, including cultural relics, buildings, heritage sites, and relics of historical, artistic, or scientific value, as well as oral traditions and expressions, performing arts, social customs, rituals, festivals, practical experiences and knowledge, and handicraft skits.We adopt a broad concept to define cultural tourism resources: things or factors that objectively exist in a certain geographical space and are attractive to tourists primarily due to their cultural values or endowed with cultural elements, which can be used by tourism industry to generate social, economic, and ecological benefits; the tourism resources referred to in this paper exclude natural resources that have tourism attractiveness but need to be developed.


### 2.2. Image Transformation

#### 2.2.1. Image Pyramiding

Before extracting moving objects, the video sequence should be preprocessed first. Image denoising and dimensionality reduction operations can reduce noise interference and computational effort. Image Pyramidtion [6] is a set of results obtained by sampling the same image at different resolutions. It consists of two steps: first, the image is smoothed by a low-pass filter; then the preprocessed image is downsampled (horizontal and vertical 1/2). Then, we will get a series of reduced images. A pyramid is created for the 2D image as shown in Figure 2.Figure 2
A pyramid is created for the 2D image.First, images of different resolutions are realized and labeled asIkx,y, where k = 0, 1, 2 represents the number of layers of the pyramid and (x, y) represents the pixel coordinates. After obtaining the images with different resolutions, three frames of differential operations are performed at the corresponding layers.


### 2.3. Double-Tree Complex Wavelet Transform

The DT-CWT was first proposed by Kingsbury [7, 8]. Unlike the general discrete wavelet transform (DWT), DT-CWT utilizes two independent DT-CWT implemented using two independent wavelet transforms, as shown in Figure 3. A signal is decomposed and constructed by two DWTs, each of which is referred to as a tree. Each of them is called a tree and uses a different set of filters to satisfy the perfect reconstruction (PR) condition. From the real wavelet transform coefficients, these coefficients are formed from each of the formed real wavelet transform coefficients by combining the DWT coefficients to form the complex wavelet coefficients. The complex wavelet coefficients are formed as follows:(1)ya+jybwhere ya and yb are real coefficients from “Tree A″ and “TreeB”, respectively. Equation (1) can be also represented as mejθ, where m is ya2+yb2 and θ is ya/yb.Figure 3
Structure of DT-CWT.The DT-CWT contains two independent wavelet transforms, as shown in Figure3. The two passes filter the input image separately: one is used to generate the real part of the coefficients, and the other is used to generate the imaginary part of the coefficients, which constitute the output of the DT-CWT. By constructing the two filter banks in an approximate Hilbert transform relationship, the DT-CWT has approximate translational invariance.


### 2.4. Gabor Wavelet

Gabor wavelet [9] is a common feature extraction method that has been commonly used in fields such as face recognition and analysis and is a mathematical tool that can extract information at multiple scales and directions.In the spatial domain Gabor filter is a sinusoidal plane wave modulated by a Gaussian function; the two-dimensional Gabor filter is defined as(2)ϕjz=kjσ2exp−kj2z22σ2×expikj⋅z−exp−σ22,where i is the imaginary operator; σ is the filter bandwidth; kj = kv(cosθ, sinθ)T; kv = 2(−v + 2)/2 × π; θ = u × π/K; v corresponds to the scale (frequency) of the Gabor filter; u corresponds to the direction of the Gabor filter.


## 2.1. Concept of Cultural Tourism Resources

The initial definition of resources is the original endowment of nature, which was subsequently expanded to include natural and cultural resources, and the utility characteristics of resources were clarified. Natural resources are the combination of natural environmental factors and conditions that can generate economic value under certain conditions of time and place to enhance the current and future welfare of humans; cultural resources are the material and spiritual products that condense the undifferentiated labor of humans, including the material and immaterial cultural wealth accumulated over the course of history [5]. Tourism resources are the concept of resources based on functional attributes, which refers to the sum of various things and factors that exist in a certain area, can be attractive to tourists, can be used by the tourism industry, and produce economic, social, and environmental benefits. Tourism resources include natural and human tourism resources that have been exploited and potential natural and human tourism resources with tourism attractiveness to be developed. Tourism resources include the subconcept of cultural tourism resources. In a narrow sense, cultural tourism resources are tourism resources that organically combine culture and tourism; in a broader sense, cultural tourism resources are all tourism resources that can provide cultural experiences for tourists, including cultural relics, buildings, heritage sites, and relics of historical, artistic, or scientific value, as well as oral traditions and expressions, performing arts, social customs, rituals, festivals, practical experiences and knowledge, and handicraft skits.We adopt a broad concept to define cultural tourism resources: things or factors that objectively exist in a certain geographical space and are attractive to tourists primarily due to their cultural values or endowed with cultural elements, which can be used by tourism industry to generate social, economic, and ecological benefits; the tourism resources referred to in this paper exclude natural resources that have tourism attractiveness but need to be developed.


## 2.2. Image Transformation

### 2.2.1. Image Pyramiding

Before extracting moving objects, the video sequence should be preprocessed first. Image denoising and dimensionality reduction operations can reduce noise interference and computational effort. Image Pyramidtion [6] is a set of results obtained by sampling the same image at different resolutions. It consists of two steps: first, the image is smoothed by a low-pass filter; then the preprocessed image is downsampled (horizontal and vertical 1/2). Then, we will get a series of reduced images. A pyramid is created for the 2D image as shown in Figure 2.Figure 2
A pyramid is created for the 2D image.First, images of different resolutions are realized and labeled asIkx,y, where k = 0, 1, 2 represents the number of layers of the pyramid and (x, y) represents the pixel coordinates. After obtaining the images with different resolutions, three frames of differential operations are performed at the corresponding layers.


## 2.2.1. Image Pyramiding

Before extracting moving objects, the video sequence should be preprocessed first. Image denoising and dimensionality reduction operations can reduce noise interference and computational effort. Image Pyramidtion [6] is a set of results obtained by sampling the same image at different resolutions. It consists of two steps: first, the image is smoothed by a low-pass filter; then the preprocessed image is downsampled (horizontal and vertical 1/2). Then, we will get a series of reduced images. A pyramid is created for the 2D image as shown in Figure 2.Figure 2
A pyramid is created for the 2D image.First, images of different resolutions are realized and labeled asIkx,y, where k = 0, 1, 2 represents the number of layers of the pyramid and (x, y) represents the pixel coordinates. After obtaining the images with different resolutions, three frames of differential operations are performed at the corresponding layers.


## 2.3. Double-Tree Complex Wavelet Transform

The DT-CWT was first proposed by Kingsbury [7, 8]. Unlike the general discrete wavelet transform (DWT), DT-CWT utilizes two independent DT-CWT implemented using two independent wavelet transforms, as shown in Figure 3. A signal is decomposed and constructed by two DWTs, each of which is referred to as a tree. Each of them is called a tree and uses a different set of filters to satisfy the perfect reconstruction (PR) condition. From the real wavelet transform coefficients, these coefficients are formed from each of the formed real wavelet transform coefficients by combining the DWT coefficients to form the complex wavelet coefficients. The complex wavelet coefficients are formed as follows:(1)ya+jybwhere ya and yb are real coefficients from “Tree A″ and “TreeB”, respectively. Equation (1) can be also represented as mejθ, where m is ya2+yb2 and θ is ya/yb.Figure 3
Structure of DT-CWT.The DT-CWT contains two independent wavelet transforms, as shown in Figure3. The two passes filter the input image separately: one is used to generate the real part of the coefficients, and the other is used to generate the imaginary part of the coefficients, which constitute the output of the DT-CWT. By constructing the two filter banks in an approximate Hilbert transform relationship, the DT-CWT has approximate translational invariance.


## 2.4. Gabor Wavelet

Gabor wavelet [9] is a common feature extraction method that has been commonly used in fields such as face recognition and analysis and is a mathematical tool that can extract information at multiple scales and directions.In the spatial domain Gabor filter is a sinusoidal plane wave modulated by a Gaussian function; the two-dimensional Gabor filter is defined as(2)ϕjz=kjσ2exp−kj2z22σ2×expikj⋅z−exp−σ22,where i is the imaginary operator; σ is the filter bandwidth; kj = kv(cosθ, sinθ)T; kv = 2(−v + 2)/2 × π; θ = u × π/K; v corresponds to the scale (frequency) of the Gabor filter; u corresponds to the direction of the Gabor filter.


## 3. Method

### 3.1. Feature Extraction

#### 3.1.1. DT-CWT Feature Extraction

DT-CWT is implemented by using 1D DT-CWT to filter the rows and columns of the image, and each level of DT-CWT filtering results in 6 different directions of high frequency coefficients matrix at each level [10].In the DT-CWT feature extraction process, the image is first filtered by 3 levels of 2-D DT-CWT to obtain the coefficient matrices of each level, and the meanμ and variance σ of the coefficient matrices of each level are calculated as the feature vector X. The equation of the mean and variance is defined as(3)μ=1M×N∑x=1M∑y=1NIx,y,(4)σ=1M×N∑x=1M∑y=1NIx,y−μ,where M, N are the row and column sizes of the image I(z).The DT-CWT eigenvectors Xr,D of the image at a certain scale r of the Gaussian pyramid decomposition are formed by joining the mean and variance vectors of each subband; i.e.,(5)Xr,D=μ1,15∘,μ1,15∘,μ1,45∘,⋯,μ3,75∘T.


#### 3.1.2. Gabor Feature Extraction

Usually, Gabor filter is used to extract features of multiple scales and multiple directions, and it is widely used in image recognition. However, Gabor increases the computational effort in extracting multidirectional features, so this paper will adopt the rotation-invariant Gabor filter. The rotation-invariant Gabor filter [11] is a filter that extracts features in all directions of a specified image at a certain scale, and to extract its texture features firstly, the rotation-invariant filter is constructed. The kernel function of the rotation-invariant Gabor filter at a certain scale is expressed as(6)hsx,y=∑r=0v−1hs,rx,y,where hs,rx,y denotes the Gabor filter kernel function with scale s and direction r. For a given image I(z), its rotation-invariant Gabor filter image is represented as the convolution of the image I(z) with the rotation-invariant Gabor filter hsx,y, with(7)Gz=Iz⋅hsx,y.Using equations (5) and (6) to calculate the mean and variance of the Gabor image obtained from the Gabor filter kernel function at a certain scale r of the Gaussian pyramid decomposition, the feature vector is obtained as(8)Xr,G=μ,σT.By concatenating the DT-CWT and the Gabor transformed feature vectorsXr,D, Xr,G to obtain Xr as the fused feature vector of the image at scale r, we have(9)Xr=Xr,DT,Xr,GTT.Therefore, the final feature vector of the image after Gaussian pyramid decomposition is(10)X=X1T,X2T,…,XrTTr=1,2,3.


### 3.2. Fusion Processing

Based on the above feature extraction analysis of DT-CWT and Gabor transform, this paper applies DT-CWT and Gabor filter together to the multiscale space through Gaussian pyramid decomposition. After the paper extracts DT-CWT 3-stage filtering, the Gaussian pyramid decomposes 36-dimensional texture features and extracts 2-dimensional features according to a certain proportion of images. In this paper, a total of 38  ×  3 = 114-dimensional feature vectors are extracted for images with 3 levels of Gaussian pyramid decomposition [12]. However, for the extracted features if the serial fusion method of feature first and last is simply used, the dimensionality of the obtained image feature vector is too high, the data volume is huge, and there will be a large amount of redundant information between the features, which is not conducive to the later classification processing and will also lead to the reduction of the efficiency and time of recognition; it is necessary to perform data dimensionality reduction processing on the features.Different levels of features contain different information distribution. The shallow-level features contain rich spatial structure information, but their resolution is higher, so the global context information is weaker. Deep-level features contain rich semantic information, which can effectively pinpoint salient targets; however, their resolution is low and they lack spatial detail information of salient targets. Besides, global contextual feature information can infer the relationship between saliency target and background from a global perspective, which can highlight the region of the target and dilute the interference of background information, so fusing these different levels of feature information can effectively improve the accuracy of saliency detection. Since the deep semantic feature information is gradually faded in the process of top-down transmission, it makes the saliency target lose the guidance of high-level semantic information after being sampled on layer-by-layer convolution, which leads to the degradation of model detection performance [13]. Therefore, this paper adds both fused shallow-level feature information and deep-level feature information to the global contextual feature information when each layer of convolution is upsampled, which can compensate for the fading of semantic information at the deep level, effectively suppress the interference of background information, and achieve the accurate localization of saliency targets at each layer of convolution.First, the feature map extracted by the above multiscale module is subjected to global average pooling (GAP) [14] to obtain the global contextual feature information; then, the global contextual feature map channel information is calibrated to generate a mask with global information; the shallow-level features are multiplied with the mask with global information by convolution operation, and then the output of the feature map is obtained by convolution operation. The fusion of global contextual feature information and shallow feature information makes up for the fading of high-level semantic information, and at the same time, it can suppress the background noise of the shallow layer and achieve the localization of salient targets more accurately. Similar to the above fusion method, the shallow feature information generates the corresponding mask through convolution operation and multiplies with the mask generated by the deep feature information; meanwhile, the deep feature information generates the corresponding mask through convolution operation and multiplies with the mask generated by the shallow feature information; thus, the complementarity of the shallow feature information and the deep feature information is realized, and the useful information between them can be effectively used to generate accurate masks. And cascade these feature information by fusion operation; finally, apply 3 × 3 convolution operation to obtain feature map M1 [15]. In addition, simultaneously cascade shallow feature information, deep feature information, and global contextual feature information; after that, use 3 × 3 convolution operation to further calibrate feature map channel information; then output feature map M2 by 3 × 3 convolution operation. Output the 2 different cascade methods of the feature maps M1 and M2 of the two different cascade methods added together, and the fused feature map channel information is calibrated again to obtain the final output feature map, which is used as input to the next stage of decoding. Thus, the high-resolution significant map is generated gradually through the cascade upsampling. The whole process is calculated as shown in (11) to (16).(11)Mg=CovMlow×MaskMglobal,(12)M1=CovMhigh×MaskMlow,(13)Mh=MaskMhigh×CovMlow,(14)Ma=CMg,M1,Mh,(15)Mb=CCovMglobal,Mlow,Mhigh,(16)Mw=AddCovMa,Mb,where Mlow is the shallow feature information, Mhigh is the deep feature information, Mglobal is the global contextual feature information, Mg is the result of fusing the mask generated by the global contextual information with the shallow features, M1 is the result of fusing the mask generated by the shallow features with the deep features, Mh is the result of fusing the mask generated by the deep features with the shallow features, Cov is the convolution operation, Mask is the mask generated by the different levels of features, and Add is the summation of the corresponding elements of the feature map [16].


### 3.3. Network of Feature Fusion

We achieved feature fusion in our network design by layering features on top of each other. Network with the feature fusion is shown in Figure4.Figure 4
Network with the feature fusion.


#### 3.3.1. Feature Layering Is a Technique for Achieving Multiscale Feature Fusion

The low-level feature layer is in charge of learning and predicting features of smaller-scale targets and objectives, as well as predicting features of smaller-scale targets. When it comes to large-scale targets, the higher-level feature layer is in charge of learning and predicting their characteristics.In this paper, we only consider each target prediction layer in the fusion network as the object of study, and the system learns small-scale detailed feature decompositions from adjacent low-level feature layers as the location sensitive features, the adjacent high-level semantic features as the contextual information, and the features of the corresponding target prediction layer. Small-scale detailed features learned by the adjacent low-level feature layer. The accuracy of the fusion network for multiscale target localization is improved by merging them in an effective manner. Only three-layer feature fusion is effective when taking into consideration the influence of feature scales in each layer. Only three-layer feature fusion is considered due to the influence of the feature scales of each layer on the final result.Figure3 illustrates the architecture of the fusion network. Using feature stacking, the fusion network is capable of anticipating hierarchically multiscale human motion targets. Using feature layering, it is possible to predict hierarchically multiscale human motion targets. The lower feature layer is responsible for learning and predicting the more localized characteristics of targets in addition to predicting the characteristics of smaller-scale targets. The higher-level feature layer is responsible for learning and predicting the characteristics of large-scale targets. The features of the corresponding target prediction layer are used to investigate the small-scale detailed features learned by the adjacent low-level feature layer as the location-sensitive features and the adjacent high-level semantic features as the contextual information. It is possible to improve the precision of fusion networks for multiscale target localization through productive integration. Only three layers of feature fusion are considered for multiple-scaled targets, with the influence of each layer's characteristics taken into account.


## 3.1. Feature Extraction

### 3.1.1. DT-CWT Feature Extraction

DT-CWT is implemented by using 1D DT-CWT to filter the rows and columns of the image, and each level of DT-CWT filtering results in 6 different directions of high frequency coefficients matrix at each level [10].In the DT-CWT feature extraction process, the image is first filtered by 3 levels of 2-D DT-CWT to obtain the coefficient matrices of each level, and the meanμ and variance σ of the coefficient matrices of each level are calculated as the feature vector X. The equation of the mean and variance is defined as(3)μ=1M×N∑x=1M∑y=1NIx,y,(4)σ=1M×N∑x=1M∑y=1NIx,y−μ,where M, N are the row and column sizes of the image I(z).The DT-CWT eigenvectors Xr,D of the image at a certain scale r of the Gaussian pyramid decomposition are formed by joining the mean and variance vectors of each subband; i.e.,(5)Xr,D=μ1,15∘,μ1,15∘,μ1,45∘,⋯,μ3,75∘T.


### 3.1.2. Gabor Feature Extraction

Usually, Gabor filter is used to extract features of multiple scales and multiple directions, and it is widely used in image recognition. However, Gabor increases the computational effort in extracting multidirectional features, so this paper will adopt the rotation-invariant Gabor filter. The rotation-invariant Gabor filter [11] is a filter that extracts features in all directions of a specified image at a certain scale, and to extract its texture features firstly, the rotation-invariant filter is constructed. The kernel function of the rotation-invariant Gabor filter at a certain scale is expressed as(6)hsx,y=∑r=0v−1hs,rx,y,where hs,rx,y denotes the Gabor filter kernel function with scale s and direction r. For a given image I(z), its rotation-invariant Gabor filter image is represented as the convolution of the image I(z) with the rotation-invariant Gabor filter hsx,y, with(7)Gz=Iz⋅hsx,y.Using equations (5) and (6) to calculate the mean and variance of the Gabor image obtained from the Gabor filter kernel function at a certain scale r of the Gaussian pyramid decomposition, the feature vector is obtained as(8)Xr,G=μ,σT.By concatenating the DT-CWT and the Gabor transformed feature vectorsXr,D, Xr,G to obtain Xr as the fused feature vector of the image at scale r, we have(9)Xr=Xr,DT,Xr,GTT.Therefore, the final feature vector of the image after Gaussian pyramid decomposition is(10)X=X1T,X2T,…,XrTTr=1,2,3.


## 3.1.1. DT-CWT Feature Extraction

DT-CWT is implemented by using 1D DT-CWT to filter the rows and columns of the image, and each level of DT-CWT filtering results in 6 different directions of high frequency coefficients matrix at each level [10].In the DT-CWT feature extraction process, the image is first filtered by 3 levels of 2-D DT-CWT to obtain the coefficient matrices of each level, and the meanμ and variance σ of the coefficient matrices of each level are calculated as the feature vector X. The equation of the mean and variance is defined as(3)μ=1M×N∑x=1M∑y=1NIx,y,(4)σ=1M×N∑x=1M∑y=1NIx,y−μ,where M, N are the row and column sizes of the image I(z).The DT-CWT eigenvectors Xr,D of the image at a certain scale r of the Gaussian pyramid decomposition are formed by joining the mean and variance vectors of each subband; i.e.,(5)Xr,D=μ1,15∘,μ1,15∘,μ1,45∘,⋯,μ3,75∘T.


## 3.1.2. Gabor Feature Extraction

Usually, Gabor filter is used to extract features of multiple scales and multiple directions, and it is widely used in image recognition. However, Gabor increases the computational effort in extracting multidirectional features, so this paper will adopt the rotation-invariant Gabor filter. The rotation-invariant Gabor filter [11] is a filter that extracts features in all directions of a specified image at a certain scale, and to extract its texture features firstly, the rotation-invariant filter is constructed. The kernel function of the rotation-invariant Gabor filter at a certain scale is expressed as(6)hsx,y=∑r=0v−1hs,rx,y,where hs,rx,y denotes the Gabor filter kernel function with scale s and direction r. For a given image I(z), its rotation-invariant Gabor filter image is represented as the convolution of the image I(z) with the rotation-invariant Gabor filter hsx,y, with(7)Gz=Iz⋅hsx,y.Using equations (5) and (6) to calculate the mean and variance of the Gabor image obtained from the Gabor filter kernel function at a certain scale r of the Gaussian pyramid decomposition, the feature vector is obtained as(8)Xr,G=μ,σT.By concatenating the DT-CWT and the Gabor transformed feature vectorsXr,D, Xr,G to obtain Xr as the fused feature vector of the image at scale r, we have(9)Xr=Xr,DT,Xr,GTT.Therefore, the final feature vector of the image after Gaussian pyramid decomposition is(10)X=X1T,X2T,…,XrTTr=1,2,3.


## 3.2. Fusion Processing

Based on the above feature extraction analysis of DT-CWT and Gabor transform, this paper applies DT-CWT and Gabor filter together to the multiscale space through Gaussian pyramid decomposition. After the paper extracts DT-CWT 3-stage filtering, the Gaussian pyramid decomposes 36-dimensional texture features and extracts 2-dimensional features according to a certain proportion of images. In this paper, a total of 38  ×  3 = 114-dimensional feature vectors are extracted for images with 3 levels of Gaussian pyramid decomposition [12]. However, for the extracted features if the serial fusion method of feature first and last is simply used, the dimensionality of the obtained image feature vector is too high, the data volume is huge, and there will be a large amount of redundant information between the features, which is not conducive to the later classification processing and will also lead to the reduction of the efficiency and time of recognition; it is necessary to perform data dimensionality reduction processing on the features.Different levels of features contain different information distribution. The shallow-level features contain rich spatial structure information, but their resolution is higher, so the global context information is weaker. Deep-level features contain rich semantic information, which can effectively pinpoint salient targets; however, their resolution is low and they lack spatial detail information of salient targets. Besides, global contextual feature information can infer the relationship between saliency target and background from a global perspective, which can highlight the region of the target and dilute the interference of background information, so fusing these different levels of feature information can effectively improve the accuracy of saliency detection. Since the deep semantic feature information is gradually faded in the process of top-down transmission, it makes the saliency target lose the guidance of high-level semantic information after being sampled on layer-by-layer convolution, which leads to the degradation of model detection performance [13]. Therefore, this paper adds both fused shallow-level feature information and deep-level feature information to the global contextual feature information when each layer of convolution is upsampled, which can compensate for the fading of semantic information at the deep level, effectively suppress the interference of background information, and achieve the accurate localization of saliency targets at each layer of convolution.First, the feature map extracted by the above multiscale module is subjected to global average pooling (GAP) [14] to obtain the global contextual feature information; then, the global contextual feature map channel information is calibrated to generate a mask with global information; the shallow-level features are multiplied with the mask with global information by convolution operation, and then the output of the feature map is obtained by convolution operation. The fusion of global contextual feature information and shallow feature information makes up for the fading of high-level semantic information, and at the same time, it can suppress the background noise of the shallow layer and achieve the localization of salient targets more accurately. Similar to the above fusion method, the shallow feature information generates the corresponding mask through convolution operation and multiplies with the mask generated by the deep feature information; meanwhile, the deep feature information generates the corresponding mask through convolution operation and multiplies with the mask generated by the shallow feature information; thus, the complementarity of the shallow feature information and the deep feature information is realized, and the useful information between them can be effectively used to generate accurate masks. And cascade these feature information by fusion operation; finally, apply 3 × 3 convolution operation to obtain feature map M1 [15]. In addition, simultaneously cascade shallow feature information, deep feature information, and global contextual feature information; after that, use 3 × 3 convolution operation to further calibrate feature map channel information; then output feature map M2 by 3 × 3 convolution operation. Output the 2 different cascade methods of the feature maps M1 and M2 of the two different cascade methods added together, and the fused feature map channel information is calibrated again to obtain the final output feature map, which is used as input to the next stage of decoding. Thus, the high-resolution significant map is generated gradually through the cascade upsampling. The whole process is calculated as shown in (11) to (16).(11)Mg=CovMlow×MaskMglobal,(12)M1=CovMhigh×MaskMlow,(13)Mh=MaskMhigh×CovMlow,(14)Ma=CMg,M1,Mh,(15)Mb=CCovMglobal,Mlow,Mhigh,(16)Mw=AddCovMa,Mb,where Mlow is the shallow feature information, Mhigh is the deep feature information, Mglobal is the global contextual feature information, Mg is the result of fusing the mask generated by the global contextual information with the shallow features, M1 is the result of fusing the mask generated by the shallow features with the deep features, Mh is the result of fusing the mask generated by the deep features with the shallow features, Cov is the convolution operation, Mask is the mask generated by the different levels of features, and Add is the summation of the corresponding elements of the feature map [16].


## 3.3. Network of Feature Fusion

We achieved feature fusion in our network design by layering features on top of each other. Network with the feature fusion is shown in Figure4.Figure 4
Network with the feature fusion.


### 3.3.1. Feature Layering Is a Technique for Achieving Multiscale Feature Fusion

The low-level feature layer is in charge of learning and predicting features of smaller-scale targets and objectives, as well as predicting features of smaller-scale targets. When it comes to large-scale targets, the higher-level feature layer is in charge of learning and predicting their characteristics.In this paper, we only consider each target prediction layer in the fusion network as the object of study, and the system learns small-scale detailed feature decompositions from adjacent low-level feature layers as the location sensitive features, the adjacent high-level semantic features as the contextual information, and the features of the corresponding target prediction layer. Small-scale detailed features learned by the adjacent low-level feature layer. The accuracy of the fusion network for multiscale target localization is improved by merging them in an effective manner. Only three-layer feature fusion is effective when taking into consideration the influence of feature scales in each layer. Only three-layer feature fusion is considered due to the influence of the feature scales of each layer on the final result.Figure3 illustrates the architecture of the fusion network. Using feature stacking, the fusion network is capable of anticipating hierarchically multiscale human motion targets. Using feature layering, it is possible to predict hierarchically multiscale human motion targets. The lower feature layer is responsible for learning and predicting the more localized characteristics of targets in addition to predicting the characteristics of smaller-scale targets. The higher-level feature layer is responsible for learning and predicting the characteristics of large-scale targets. The features of the corresponding target prediction layer are used to investigate the small-scale detailed features learned by the adjacent low-level feature layer as the location-sensitive features and the adjacent high-level semantic features as the contextual information. It is possible to improve the precision of fusion networks for multiscale target localization through productive integration. Only three layers of feature fusion are considered for multiple-scaled targets, with the influence of each layer's characteristics taken into account.


## 3.3.1. Feature Layering Is a Technique for Achieving Multiscale Feature Fusion

The low-level feature layer is in charge of learning and predicting features of smaller-scale targets and objectives, as well as predicting features of smaller-scale targets. When it comes to large-scale targets, the higher-level feature layer is in charge of learning and predicting their characteristics.In this paper, we only consider each target prediction layer in the fusion network as the object of study, and the system learns small-scale detailed feature decompositions from adjacent low-level feature layers as the location sensitive features, the adjacent high-level semantic features as the contextual information, and the features of the corresponding target prediction layer. Small-scale detailed features learned by the adjacent low-level feature layer. The accuracy of the fusion network for multiscale target localization is improved by merging them in an effective manner. Only three-layer feature fusion is effective when taking into consideration the influence of feature scales in each layer. Only three-layer feature fusion is considered due to the influence of the feature scales of each layer on the final result.Figure3 illustrates the architecture of the fusion network. Using feature stacking, the fusion network is capable of anticipating hierarchically multiscale human motion targets. Using feature layering, it is possible to predict hierarchically multiscale human motion targets. The lower feature layer is responsible for learning and predicting the more localized characteristics of targets in addition to predicting the characteristics of smaller-scale targets. The higher-level feature layer is responsible for learning and predicting the characteristics of large-scale targets. The features of the corresponding target prediction layer are used to investigate the small-scale detailed features learned by the adjacent low-level feature layer as the location-sensitive features and the adjacent high-level semantic features as the contextual information. It is possible to improve the precision of fusion networks for multiscale target localization through productive integration. Only three layers of feature fusion are considered for multiple-scaled targets, with the influence of each layer's characteristics taken into account.


## 4. Experimental Results and Analysis

### 4.1. Data Demonstration

To test the efficacy of the proposed algorithm, 4000 images of cultural resources were examined (2,800 positive samples and 1,200 negative samples). Cross-validation was utilized for both training and testing. 1 part of the 2800 images was used as the test set, while the other 4 parts were used as the training set. The average of 5 experiments was used as the performance index. The true positive (TP), true negative (TN), false positive (FP), and false negative (FN) classification results were calculated with reference to the gold standard, and the classification performance of the classifier was measured using equations (17)–(19). In order to ensure the fairness of the experiment, the specific experimental parameters of the method in this paper are consistent with those in [7]:(17)accurary=TP+TNTP+FP+TN+FN,(18)sensitivity=TPTP+FN,(19)specificity=TP+TNTP+FP+TN+FN.To determine which scale of Gabor filter features fuses with DT-CWT features more effectively, the classification performance of five different scales of Gabor filter features after DT-CWT feature fusion and KPCA dimension reduction is compared in Table1.Table 1
Performance comparison of fusing DT-CWT features with Gabor features at different scales.
Gabor filter scaleAccuracySpecificitySensitivityScale 079.4376.4383.54Scale 184.2378.3184.94Scale 282.7079.2185.22Scale 367.9845.7684.61Scale 483.1276.3483.41Table1 reveals that the Gabor filter scale of Scale 1 has a higher recognition rate, specificity, and sensitivity than other scales, indicating that the majority of the texture features for distinguishing benign from malignant thyroid nodules using the Gabor filter are concentrated in this range. If the extracted features from five scales (Scale 0 to Scale 4) are used as feature vectors collectively, the recognition effect is diminished compared to a single scale. Using three filter scales with a higher recognition rate as feature vectors yields a lower recognition rate than using a single scale. Due to the effect of high dimensionality on feature selection after fusion, which decreases the overall recognition rate, the multiscale performance is marginally inferior to that of the single-scale performance. This paper uses DT-CWT features fused with Gabor filter features and a Scale 1 template as the feature extraction method for multiscale images.Table2 provides the classification effect of the fusion of features extracted by the Gabor filter of Scale 1 and DT-CWT with various dimensionality reduction methods in order to test the improved effect of feature fusion after adopting KPCA dimension reduction. According to Table 2, the accuracy of benign and malignant thyroid nodules using PCA and other methods for dimensionality reduction is relatively low, whereas its performance is relatively high when using KPCA. Due to the fact that KPCA can extract the maximum amount of information from the index and retain feature information adequately, this is the case. Moreover, the experiments demonstrated that the performance of the dimensionality reduction method was significantly superior to that of the Scale 1 + DT-CWT method.Table 2
Performance comparison of different dimension reduction methods.
Downscaling methodsAccuracySpecificitySensitivityScale 1 + DT-CWT + PCA69.3521.0583.54Scale 1 + DT-CWT  ＋ LDA69.0340.7174.38Scale 1 + DT-CWT  ＋ LLE62.0435.4375.22Scale 1 + DT-CWT + isomap67.9845.7674.82Scale 1 + DT-CWT + KPCA84.5276.3487.30Scale 1 + DT-CWT57.6256.3156.68To test the effectiveness of the method proposed in this paper, Table3 compares the recognition effects of commonly used texture feature extraction methods on this dataset. The experimental results in Table 3 are obtained by using different algorithms on the same thyroid nodule ultrasound image dataset. Among them, GLCM, LBP, and gradient cooccurrence matrix methods are classical texture feature extraction algorithms, but their performance is not ideal due to the imaging of ultrasound images, and they do not detect changes in texture features well. The methods in [17, 18] are methods that have been applied in recent years to extract texture features in medical images. From the results, it can be seen that its performance is better compared to the traditional methods. But it is slightly worse compared to the fusion method in this paper. And the proposed multiscale DT-CWT and Gabor transform based fusion algorithm in this paper has better performance than DT-CWT and Gabor alone, which further indicates the effectiveness of fusion.Table 3
Performance comparison of different dimension reduction methods.
MethodsAccuracySpecificitySensitivityGLCM56.0034.0072.00Gradient coevolution matrix57.1053.1274.46LBP61.5635.0080.74[17]72.7864.7684.61[18]69.3268.5075.23DT-CWT65.5236.8584.00Gabor68.5256.9082.30Methodology of this paper84.5175.7687.49


## 4.1. Data Demonstration

To test the efficacy of the proposed algorithm, 4000 images of cultural resources were examined (2,800 positive samples and 1,200 negative samples). Cross-validation was utilized for both training and testing. 1 part of the 2800 images was used as the test set, while the other 4 parts were used as the training set. The average of 5 experiments was used as the performance index. The true positive (TP), true negative (TN), false positive (FP), and false negative (FN) classification results were calculated with reference to the gold standard, and the classification performance of the classifier was measured using equations (17)–(19). In order to ensure the fairness of the experiment, the specific experimental parameters of the method in this paper are consistent with those in [7]:(17)accurary=TP+TNTP+FP+TN+FN,(18)sensitivity=TPTP+FN,(19)specificity=TP+TNTP+FP+TN+FN.To determine which scale of Gabor filter features fuses with DT-CWT features more effectively, the classification performance of five different scales of Gabor filter features after DT-CWT feature fusion and KPCA dimension reduction is compared in Table1.Table 1
Performance comparison of fusing DT-CWT features with Gabor features at different scales.
Gabor filter scaleAccuracySpecificitySensitivityScale 079.4376.4383.54Scale 184.2378.3184.94Scale 282.7079.2185.22Scale 367.9845.7684.61Scale 483.1276.3483.41Table1 reveals that the Gabor filter scale of Scale 1 has a higher recognition rate, specificity, and sensitivity than other scales, indicating that the majority of the texture features for distinguishing benign from malignant thyroid nodules using the Gabor filter are concentrated in this range. If the extracted features from five scales (Scale 0 to Scale 4) are used as feature vectors collectively, the recognition effect is diminished compared to a single scale. Using three filter scales with a higher recognition rate as feature vectors yields a lower recognition rate than using a single scale. Due to the effect of high dimensionality on feature selection after fusion, which decreases the overall recognition rate, the multiscale performance is marginally inferior to that of the single-scale performance. This paper uses DT-CWT features fused with Gabor filter features and a Scale 1 template as the feature extraction method for multiscale images.Table2 provides the classification effect of the fusion of features extracted by the Gabor filter of Scale 1 and DT-CWT with various dimensionality reduction methods in order to test the improved effect of feature fusion after adopting KPCA dimension reduction. According to Table 2, the accuracy of benign and malignant thyroid nodules using PCA and other methods for dimensionality reduction is relatively low, whereas its performance is relatively high when using KPCA. Due to the fact that KPCA can extract the maximum amount of information from the index and retain feature information adequately, this is the case. Moreover, the experiments demonstrated that the performance of the dimensionality reduction method was significantly superior to that of the Scale 1 + DT-CWT method.Table 2
Performance comparison of different dimension reduction methods.
Downscaling methodsAccuracySpecificitySensitivityScale 1 + DT-CWT + PCA69.3521.0583.54Scale 1 + DT-CWT  ＋ LDA69.0340.7174.38Scale 1 + DT-CWT  ＋ LLE62.0435.4375.22Scale 1 + DT-CWT + isomap67.9845.7674.82Scale 1 + DT-CWT + KPCA84.5276.3487.30Scale 1 + DT-CWT57.6256.3156.68To test the effectiveness of the method proposed in this paper, Table3 compares the recognition effects of commonly used texture feature extraction methods on this dataset. The experimental results in Table 3 are obtained by using different algorithms on the same thyroid nodule ultrasound image dataset. Among them, GLCM, LBP, and gradient cooccurrence matrix methods are classical texture feature extraction algorithms, but their performance is not ideal due to the imaging of ultrasound images, and they do not detect changes in texture features well. The methods in [17, 18] are methods that have been applied in recent years to extract texture features in medical images. From the results, it can be seen that its performance is better compared to the traditional methods. But it is slightly worse compared to the fusion method in this paper. And the proposed multiscale DT-CWT and Gabor transform based fusion algorithm in this paper has better performance than DT-CWT and Gabor alone, which further indicates the effectiveness of fusion.Table 3
Performance comparison of different dimension reduction methods.
MethodsAccuracySpecificitySensitivityGLCM56.0034.0072.00Gradient coevolution matrix57.1053.1274.46LBP61.5635.0080.74[17]72.7864.7684.61[18]69.3268.5075.23DT-CWT65.5236.8584.00Gabor68.5256.9082.30Methodology of this paper84.5175.7687.49


## 5. Conclusion

In this paper, we use a multiscale fusion method to extract features from the image and video aspects of cultural resources, followed by the fusion and classification of those features, which is essential for analyzing the feature distribution of cultural resources. Combining some characteristics of DT-WT and Gabor wavelets for texture feature extraction, we propose a multiscale fusion technique for feature extraction of cultural resources. We can decompose the image into a multiscale space using a Gaussian pyramid, combine the features of each layer, and then classify them using an SVM classifier with RBF kernel function. Numerous experiments demonstrate that the recognition effect of the method proposed in this paper is superior to that of other feature extraction methods [19].



---

*Source: 5946166-2022-06-17.xml*