Hyperspectral Image Classification Using Custom Spectral Convolutional Neural Networks (CSCNNs)

Authors: stephanie  rouamba
lara  thompson
nian  zhang
max  denis
wagdy  mahmoud
tolessa  deksissa
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-31 05:16:26

Abstract: Custom spectral convolutional neural networks (CSCNNs) combine the strengths of convolutional neural networks with specialized spectral processing, resulting in improved classification accuracy by effectively capturing subtle variations in hyperspectral data. This paper proposes a CSCNNs based approach to classify hyperspectral images. The proposed method leverages the high-dimensional spectral data inherent in hyperspectral images, employing convolutional layers specifically designed to capture spectral-spatial features. By reducing dimensionality through principal component analysis and creating image patches, the model is trained to recognize complex patterns across different spectral bands.  In addition, a comprehensive analysis of CSCNN performance is carried out, focusing on its architecture, key features, and benefits in computational efficiency and spectral representation. Experimental results on datasets such as Salinas-A, Pavia University (Pavia-U), and Indian Pines demonstrate that the CSCNN model surpasses traditional methods, achieving higher classification accuracy and more robust performance metrics like overall accuracy (OA), average accuracy (AA), and Kappa coefficient.

Main paper: 
 introduction: 
 Hyperspectral imaging captures a wide spectrum of light for each pixel in an image, allowing for the identification of materials, objects, and conditions that are not visible to the naked eye or detectable with traditional imaging techniques. In hyperspectral images, each pixel is considered a highdimensional vector, with each component representing spectral reflectance at a specific wavelength. This detailed spectral information allows for the differentiation of subtle spectral variations, making Hyperspectral imaging valuable across numerous applications [1] - [12] . According to recent studies [13] , hyperspectral image classification assigning each pixel to a specific class based on its spectral properties is a highly active research area within the hyperspectral community and has gained significant attention in the remote sensing field.Unlike standard RGB images, which capture light in three bands (red, green, and blue), hyperspectral images can capture hundreds of contiguous spectral bands, offering rich and detailed information about the scene. This highdimensional data has proven valuable in various fields, including agriculture, environmental monitoring, mineral exploration, and military surveillance. However, the vast amount of data in hyperspectral images presents challenges in processing and analysis. In fact, there are two primary challenges in hyperspectral image classification: the high spatial variability of spectral signatures and the imbalance between the limited number of training samples and the high dimensionality of the data [14] . The first challenge arises from several factors, including changes in lighting, environmental conditions, atmospheric effects, and temporal variations. The second challenge creates ill-posed problems for certain methods and reduces the classifiers' ability to generalize effectively.The conventional methods for analyzing hyperspectral images are pixel-based approaches, where each image pixel is classified based on its spectral information [15] . This traditional approach to classification relies on the pixel because the pixel serves as the fundamental unit of satellite imagery.Custom spectral convolutional neural networks (CSCNNs) offer several advantages over traditional pixel-based approaches, particularly in the context of hyperspectral data analysis: 1) Spectral-Spatial Feature Integration: CSCNNs can simultaneously capture both spectral and spatial features, providing a more comprehensive understanding of the data. Traditional pixel-based approaches often treat pixels independently, missing the spatial context that CSCNNs can incorporate. 2) Improved Classification Accuracy: By leveraging deep learning techniques, CSCNNs can model complex relationships within the data, leading to higher classification accuracy compared to traditional methods that rely solely on spectral information. 3) Noise Reduction: CSCNNs can better handle noise in hyperspectral images. Traditional pixel-based methods may classify noisy or outlier pixels incorrectly, while CSCNNs can mitigate this by considering spatial information, leading to more robust classifications. 4) Dimensionality Reduction: CSCNNs can effectively reduce the dimensionality of hyperspectral data by learning lower-dimensional representations, making the processing more efficient without losing critical information. Traditional pixel-based methods may struggle with the high dimensionality of hyperspectral data. 5) Scalability and Efficiency: CSCNNs are designed to handle large-scale data efficiently, often requiring fewer computational resources as they can exploit the spectral structure more effectively than traditional methods, which may require more intensive processing for similar tasks. 6) Enhanced Spectral Representation: Custom spectral convolutions in CSCNNs are tailored to better capture the unique spectral signatures of different materials, leading to more precise and contextaware classifications than pixel-based methods, which may not fully utilize the spectral richness of the data. Therefore, it is imperative to develop a novel Custom Spectral Convolutional Neural Networks (CSCNNs) method for hyperspectral image classification.The remainder of the paper is organized as follows: Section II details the proposed methodology. Section III delves into time series prediction using the LSTM network. Section IV provides the experimental results. Section V concludes the paper.Hyperspectral image classification is a complex task due to the high dimensionality of the data and the variability of spectral signatures across different materials. To address these challenges, we propose a Spectral Convolutional Neural Network (CSCNN), to effectively learn and classify hyperspectral images.Networks (CNNs) with the characteristics of spectral domain processing. Its basic architecture typically includes multiple convolutional layers, activation layers, pooling layers, and fully connected layers. One of the key features of CSCNN includes spectral feature extraction. CSCNN performs convolution operations in the spectral domain, effectively capturing subtle variations and features in hyperspectral images. Compared to traditional spatial domain processing, this approach better preserves spectral information. Another key feature of CSCNN is the layer count and parameter tuning. CSCNN 's layer count can be flexibly adjusted based on requirements, such as using a custom 17-layer network. The parameter settings of each layer directly influence the network's learning ability and classification performance.The proposed network architecture for hyperspectral image classification leverages a custom Spectral Convolutional Neural Network (CSCNN) specifically designed to handle the high-dimensional data inherent in hyperspectral images (HSI), as shown in Fig. 1 . The CSCNN architecture consists of several layers that are tailored to extract spectral and spatial features effectively, enabling the classification of each pixel in an HSI into a specific class based on its spectral signature. The network begins with a 3D input layer that accepts hyperspectral image patches. The input size is defined as [25, 25, 30] , where 25 x 25 represents the spatial dimensions of the image patch and 30 represents the number of spectral bands after dimensionality reduction using techniques like PCA.The network includes a series of 3D convolutional layers that apply 3D filters to the input data. These layers are designed to learn spatial-spectral features by convolving across both spatial dimensions (height and width) and the spectral dimension (depth). The network typically includes multiple convolutional layers with varying filter sizes, such as:• First 3D convolutional layer with a filter size of [3, 3, 7] and 8 filters, followed by a ReLU activation function. • Second 3D convolutional layer with a filter size of [3, 3, 5 ] and 16 filters, followed by a ReLU activation function.• Subsequent 3D convolutional layers further refine the feature extraction with progressively smaller filter sizes, capturing detailed spatial-spectral relationships.• Following the convolutional layers, the network includes fully connected layers to aggregate the features extracted from the convolutional layers. These layers transform the 3D feature maps into a 1D feature vector for classification purposes. Typically, there are multiple fully connected layers with a decreasing number of neurons: • The first fully connected layer with 256 neurons, followed by a ReLU activation. • A second fully connected layer with 128 neurons, incorporating dropout to prevent overfitting. • A final fully connected layer corresponding to the number of classes in the dataset, followed by a softmax activation function to produce class probabilities.The output layer of the network uses a softmax function to generate a probability distribution across all the classes, allowing for the classification of each input patch into one of the predefined classes.Dropout layers are interspersed between fully connected layers to mitigate overfitting by randomly setting a fraction of input units to zero during each forward pass, which helps improve the model's generalization capability.In this section, we present the experimental results obtained from using the Convolutional Spectral Convolutional Neural Network (CSCNN), for hyperspectral image classification. We evaluate the effectiveness of our proposed method on the widely used hyperspectral datasets, including Indian Pines, Salinas, and Pavia University datasets.In the experiments, we assess the proposed CSCNN network on three widely used hyperspectral scenes named Salinas-A [16] , PaviaU [17] , and Indian Pines [18] . Each hyperspectral image was normalized and divided into smaller patches of size 25x25 pixels to ensure uniform input to the deep learning model. Dimensionality reduction was performed using PCA to reduce the number of spectral bands to 30, maintaining the most significant features. In addition, the network was trained using the Adam optimizer with an initial learning rate of 0.001, batch size of 256, and a maximum of 100 epochs. Moreover, the datasets were randomly split into training (70%) and test (30%) subsets to evaluate the model's performance on unseen data.The performance is evaluated by the Overall Accuracy (OA), Average Accuracy (AA), and Kappa Coefficient [19] - [22] .The OA is calculated as the average of Producer's Accuracy and User's Accuracy.Average Accuracy (AA) is a performance metric used in Average Accuracy (AA) is a performance metric used in classification tasks that calculates the average of accuracies obtained for each class in a dataset. AA is computed by taking the mean of individual class accuracies, providing an overall measure of how well the model performs across all classes. It helps to identify if the model is performing well across all classes or if it is biased towards one or more classes with higher representation. Where N is the total number of classes.The Kappa Coefficient, also known as Cohen's Kappa, is a statistical measure that evaluates the agreement between the predicted and actual classifications while accounting for the possibility of random agreement. The formula for the Kappa Coefficient is: The comparison of the performance of the proposed CSCNN model with other state-of-the-art 3D Convolutional Neural Network (3D-CNN) [23] , Support Vector Machine (SVM) [24] , and Graph Convolutional Network (GCN) [25] on the Salinas-A, Pavia University (Pavia-U), and Indian Pines datasets is demonstrated in From TABLE IV, we observed that the CSCNN model achieved high overall accuracy on all datasets, demonstrating its effectiveness in capturing the spectral-spatial features of hyperspectral images. In addition, the model's average accuracy is close to the overall accuracy, indicating balanced performance across different land cover classes. Moreover, the CSCNN model showed robustness to spectral variability within the datasets, achieving high Kappa coefficients, which indicate a strong agreement between the predicted and actual classes. Furthermore, the proposed CSCNN method outperformed other state-of-the-art methods.This paper proposes custom spectral convolutional neural networks (CSCNNs) to classify hyperspectral images. The experimental results demonstrated that the CSCNN model achieved high overall accuracy across all datasets, demonstrating its effectiveness in capturing the spectralspatial features of hyperspectral images. Additionally, the model's average accuracy closely aligns with its overall accuracy, indicating balanced performance across various land cover classes. The CSCNN model also exhibited robustness to spectral variability within the datasets, achieving high Kappa coefficients that indicate strong agreement between predicted and actual classes. Furthermore, the proposed CSCNN method outperformed other state-ofthe-art approaches.Potential improvements could include the integration of spatial information. Combining spatial features with spectral features can significantly improve classification performance [26] . CSCNNs can be designed to effectively extract spatial-spectral features, using spatial context information to enhance classification accuracy. 
 
 Appendix:  



Reviews: review: title: title: Accpet
main_review: main_review: This paper investigates Custom spectral convolutional neural networks (CSCNNs) subject to image classification. After my review of this article, this article can be published in its current form. But there are still some suggestions:
1. There seems to be a formatting problem in the first paragraph of the introduction.
2. This paper presents several advantages over traditional pixel-based approaches. The authors can represent them more concisely.
title: title: Hyperspectral Image Classification Using Custom Spectral Convolutional Neural Networks (CSCNNs)
main_review: main_review: This paper proposes a CSCNNs-based approach to classify hyperspectral images. The experimental results demonstrated that the CSCNN model achieved high overall accuracy across all datasets, demonstrating its effectiveness in capturing the spectral spatial features of hyperspectral images. The results in this paper is new and interesting. Some minor suggestions are proposed below. 1) Please highlight the main contributions for reading. 2) The motivation of this paper should be clarified.
title: title: Accept
main_review: main_review: This paper proposes a hyperspectral image classification method based on CSCNNs. The method leverages the high-dimensional spectral data inherent in hyperspectral images, employing specially designed convolutional layers to capture spectral-spatial features. Extensive experiments on relevant datasets validate the performance of CSCNNs, achieving higher classification accuracy and stronger performance metrics. The paper is innovative and well-structured, but the following comments could further enhance its quality:
1.The paper has some noticeable formatting issues, such as inconsistencies in the identification of the corresponding author and improper alignment of the first line of the text.
2.The literature review is relatively sparse.
3.Table 1 is too large, and Figure 1 has a low resolution.
4.There is excessive white space on the fifth page.