pwc-archive/papers-with-abstracts huggingface dataset:
Prediction of brain tumor recurrence location based on multi-modal fusion and nonlinear correlation learning
Authors: Tongxue Zhou, Alexandra Noeuveglise, Romain Modzelewski, Fethi Ghazouani, Sébastien Thureau, Maxime Fontanilles, Su Ruan
Date: 2023-04-11 00:00:00
Tasks: Brain Tumor Segmentation, Transfer Learning, Tumor Segmentation

Brain tumor is one of the leading causes of cancer death. The high-grade brain tumors are easier to recurrent even after standard treatment. Therefore, developing a method to predict brain tumor recurrence location plays an important role in the treatment planning and it can potentially prolong patient's survival time. There is still little work to deal with this issue. In this paper, we present a deep learning-based brain tumor recurrence location prediction network. Since the dataset is usually small, we propose to use transfer learning to improve the prediction. We first train a multi-modal brain tumor segmentation network on the public dataset BraTS 2021. Then, the pre-trained encoder is transferred to our private dataset for extracting the rich semantic features. Following that, a multi-scale multi-channel feature fusion model and a nonlinear correlation learning module are developed to learn the effective features. The correlation between multi-channel features is modeled by a nonlinear equation. To measure the similarity between the distributions of original features of one modality and the estimated correlated features of another modality, we propose to use Kullback-Leibler divergence. Based on this divergence, a correlation loss function is designed to maximize the similarity between the two feature distributions. Finally, two decoders are constructed to jointly segment the present brain tumor and predict its future tumor recurrence location. To the best of our knowledge, this is the first work that can segment the present tumor and at the same time predict future tumor recurrence location, making the treatment planning more efficient and precise. The experimental results demonstrated the effectiveness of our proposed method to predict the brain tumor recurrence location from the limited dataset.
Ensemble CNNs for Breast Tumor Classification
Authors: Muhammad Umar Farooq, Zahid Ullah, Jeonghwan Gwak
Date: 2023-04-11 00:00:00
Tasks: Classification

To improve the recognition ability of computer-aided breast mass classification among mammographic images, in this work we explore the state-of-the-art classification networks to develop an ensemble mechanism. First, the regions of interest (ROIs) are obtained from the original dataset, and then three models, i.e., XceptionNet, DenseNet, and EfficientNet, are trained individually. After training, we ensemble the mechanism by summing the probabilities outputted from each network which enhances the performance up to 5%. The scheme has been validated on a public dataset and we achieved accuracy, precision, and recall 88%, 85%, and 76% respectively.
A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
Authors: Dewant Katare, Diego Perino, Jari Nurmi, Martijn Warnier, Marijn Janssen, Aaron Yi Ding
Date: 2023-04-13 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Survey

Autonomous driving services rely heavily on sensors such as cameras, LiDAR, radar, and communication modules. A common practice of processing the sensed data is using a high-performance computing unit placed inside the vehicle, which deploys AI models and algorithms to act as the brain or administrator of the vehicle. The vehicular data generated from average hours of driving can be up to 20 Terabytes depending on the data rate and specification of the sensors. Given the scale and fast growth of services for autonomous driving, it is essential to improve the overall energy and environmental efficiency, especially in the trend towards vehicular electrification (e.g., battery-powered). Although the areas have seen significant advancements in sensor technologies, wireless communications, computing and AI/ML algorithms, the challenge still exists in how to apply and integrate those technology innovations to achieve energy efficiency. This survey reviews and compares the connected vehicular applications, vehicular communications, approximation and Edge AI techniques. The focus is on energy efficiency by covering newly proposed approximation and enabling frameworks. To the best of our knowledge, this survey is the first to review the latest approximate Edge AI frameworks and publicly available datasets in energy-efficient autonomous driving. The insights and vision from this survey can be beneficial for the collaborative driving service development on low-power and memory-constrained systems and also for the energy optimization of autonomous vehicles.
GPU accelerated matrix factorization of large scale data using block based approach
Authors: Prasad Bhavana, Vineet Padmanabhan
Date: 2023-01-02 00:00:00
Tasks: CPU, GPU

Matrix Factorization (MF) on large scale data takes substantial time on a Central Processing Unit (CPU). While Graphical Processing Unit (GPU)s could expedite the computation of MF, the available memory on a GPU is finite. Leveraging GPUs require alternative techniques that allow not only parallelism but also address memory limitations. Synchronization between computation units, isolation of data related to a computational unit, sharing of data between computational units and identification of independent tasks among computational units are some of the challenges while leveraging GPUs for MF. We propose a block based approach to matrix factorization using Stochastic Gradient Descent (SGD) that is aimed at accelerating MF on GPUs. The primary motivation for the approach is to make it viable to factorize extremely large data sets on limited hardware without having to compromise on results. The approach addresses factorization of large scale data by identifying independent blocks, each of which are factorized in parallel using multiple computational units. The approach can be extended to one or more GPUs and even to distributed systems. The RMSE results of the block based approach are with in acceptable delta in comparison to the results of CPU based variant and multi-threaded CPU variant of similar SGD kernel implementation. The advantage, of the block based variant, in-terms of speed are significant in comparison to other variants.
Lane Change Intention Recognition and Vehicle Status Prediction for Autonomous Vehicles
Authors: Renteng Yuan, Mohamed Abdel-Aty, Xin Gu, Ou Zheng, Qiaojun Xiang
Date: 2023-04-25 00:00:00
Tasks: Autonomous Vehicles, Intent Detection, Multi-Task Learning

Accurately detecting and predicting lane change (LC)processes of human-driven vehicles can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes, first developing a temporal convolutional network with an attention mechanism (TCN-ATM) model to recognize LC intention. Considering the intrinsic relationship among output variables, the Multi-task Learning (MTL)framework is employed to simultaneously predict multiple LC vehicle status indicators. Furthermore, a unified modeling framework for LC intention recognition and driving status prediction (LC-IR-SP) is developed. The results indicate that the classification accuracy of LC intention was improved from 96.14% to 98.20% when incorporating the attention mechanism into the TCN model. For LC vehicle status prediction issues, three multi-tasking learning models are constructed based on MTL framework. The results indicate that the MTL-LSTM model outperforms the MTL-TCN and MTL-TCN-ATM models. Compared to the corresponding single-task model, the MTL-LSTM model demonstrates an average decrease of 26.04% in MAE and 25.19% in RMSE.
Neural Field Conditioning Strategies for 2D Semantic Segmentation
Authors: Martin Gromniak, Sven Magg, Stefan Wermter
Date: 2023-04-12 00:00:00
Tasks: 2D Semantic Segmentation, Decoder, Segmentation, Semantic Segmentation

Neural fields are neural networks which map coordinates to a desired signal. When a neural field should jointly model multiple signals, and not memorize only one, it needs to be conditioned on a latent code which describes the signal at hand. Despite being an important aspect, there has been little research on conditioning strategies for neural fields. In this work, we explore the use of neural fields as decoders for 2D semantic segmentation. For this task, we compare three conditioning methods, simple concatenation of the latent code, Feature Wise Linear Modulation (FiLM), and Cross-Attention, in conjunction with latent codes which either describe the full image or only a local region of the image. Our results show a considerable difference in performance between the examined conditioning strategies. Furthermore, we show that conditioning via Cross-Attention achieves the best results and is competitive with a CNN-based decoder for semantic segmentation.
Small Sample Hyperspectral Image Classification Based on the Random Patches Network and Recursive Filtering
Authors: Denis Uchaev, Dmitry Uchaev
Date: 2023-02-23 00:00:00
Proceeding: Sensors 2023 2
Tasks: Classification, Dimensionality Reduction, Few-Shot Image Classification, Few-Shot Learning, Hyperspectral Image Classification, image-classification, Image Classification

In recent years, different deep learning frameworks were introduced for hyperspectral image (HSI) classification. However, the proposed network models have a higher model complexity, and do not provide high classification accuracy if few-shot learning is used. This paper presents an HSI classification method that combines random patches network (RPNet) and recursive filtering (RF) to obtain informative deep features. The proposed method first convolves image bands with random patches to extract multi-level deep RPNet features. Thereafter, the RPNet feature set is subjected to dimension reduction through principal component analysis (PCA), and the extracted components are filtered using the RF procedure. Finally, the HSI spectral features and the obtained RPNet–RF features are combined to classify the HSI using a support vector machine (SVM) classifier. In order to test the performance of the proposed RPNet–RF method, some experiments were performed on three widely known datasets using a few training samples for each class, and classification results were compared with those obtained by other advanced HSI classification methods adopted for small training samples. The comparison showed that the RPNet–RF classification is characterized by higher values of such evaluation metrics as overall accuracy and Kappa coefficient.
Generative Adversarial Networks Based on Transformer Encoder and Convolution Block for Hyperspectral Image Classification
Authors: Jing Bai, Jiawei Lu, Zhu Xiao, Zheng Chen, Licheng Jiao
Date: 2022-07-16 00:00:00
Proceeding: Remote Sensing 2022 7
Tasks: Classification, Classification Of Hyperspectral Images, Few-Shot Image Classification, Few-Shot Learning, Generative Adversarial Network, Hyperspectral Image Classification, image-classification, Image Classification

Nowadays, HSI classification can reach a high classification accuracy when given sufficient labeled samples as training set. However, the performances of existing methods decrease sharply when trained on few labeled samples. Existing methods in few-shot problems usually require another dataset in order to improve the classification accuracy. However, the cross-domain problem exists in these methods because of the significant spectral shift between target domain and source domain. Considering above issues, we propose a new method without requiring external dataset through combining a Generative Adversarial Network, Transformer Encoder and convolution block in a unified framework. The proposed method has both a global receptive field provided by Transformer Encoder and a local receptive field provided by convolution block. Experiments conducted on Indian Pines, PaviaU and KSC datasets demonstrate that our method exceeds the results of existing deep learning methods for hyperspectral image classification in the few-shot learning problem.
Deep Metric Learning-Based Feature Embedding for Hyperspectral Image Classification
Authors: Bin Deng, Sen Jia, Daming Shi
Date: 2019-10-30 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2019 10
Tasks: Domain Adaptation, Few-Shot Image Classification, Hyperspectral Image Classification, image-classification, Image Classification, Metric Learning, Scene Classification, Unsupervised Domain Adaptation

Learning from a limited number of labeled samples (pixels) remains a key challenge in the hyperspectral image (HSI) classification. To address this issue, we propose a deep metric learning-based feature embedding model, which can meet the tasks both for same- and cross-scene HSI classifications. In the first task, when only a few labeled samples are available, we employ ideas from metric learning based on deep embedding features and make a similarity learning between pairs of samples. In this case, the proposed model can learn well to compare whether two samples belong to the same class. In another task, when an HSI image (target scene) that needs to be classified is not labeled at all, the embedding model can learn from another similar HSI image (source scene) with sufficient labeled samples and then transfer to the target model by using an unsupervised domain adaptation technique, which not only employs the adversarial approach to make the embedding features from the source and target samples indistinguishable but also encourages the target scene's embeddings to form similar clusters with the source scene one. After the domain adaptation between the HSIs of the two scenes is finished, any traditional HSI classifier can be used. In a simple manner, the nearest neighbor (NN) algorithm is selected as the classifier for the classification tasks throughout this article. The experimental results from a series of popular HSIs demonstrate the advantages of the proposed model both in the same- and cross-scene classification tasks.
Hyperspectral Image Classification of Convolutional Neural Network Combined with Valuable Samples
Authors: Lixin Hu, Xiaobo Luo, Yufan Wei
Date: 2020-06-01 00:00:00
Proceeding: Journal of Physics: Conference Series 2020 6
Tasks: Active Learning, Classification, Few-Shot Image Classification, Hyperspectral Image Classification, image-classification, Image Classification

Aiming at the problem that the manual labeling of samples in the hyperspectral image classification is expensive and laborious, a large number of unlabeled samples are not effectively utilized and the classification results are not ideal. A method which can provide valuable samples and employ convolutional neural network to extract spectral spatial features for classification is proposed. Active learning method is used to construct a valuable training sample set by iteratively selecting the most uncertain samples through support vector machine which performs well in small sample classification, and labeling them. Then the 3D convolutional neural network is used to extract the spectral spatial features of hyperspectral image. The experimental results of the hyperspectral classification on Indian Pines and PaviaU datasets show that the proposed method (3D VS-CNN) is better than traditional classification methods.
Graph Information Aggregation Cross-Domain Few-Shot Learning for Hyperspectral Image Classification
Authors: Yuxiang Zhang, Wei Li, Mengmeng Zhang, Shuai Wang, Ran Tao, Qian Du
Date: 2022-06-30 00:00:00
Proceeding: IEEE Transactions on Neural Networks and Learning Systems 2022 6
Tasks: Cross-Domain Few-Shot, cross-domain few-shot learning, Domain Adaptation, Few-Shot Image Classification, Few-Shot Learning, Hyperspectral Image Classification, image-classification, Image Classification

Most domain adaptation (DA) methods in cross-scene hyperspectral image classification focus on cases where source data (SD) and target data (TD) with the same classes are obtained by the same sensor. However, the classification performance is significantly reduced when there are new classes in TD. In addition, domain alignment, as one of the main approaches in DA, is carried out based on local spatial information, rarely taking into account nonlocal spatial information (nonlocal relationships) with strong correspondence. A graph information aggregation cross-domain few-shot learning (Gia-CFSL) framework is proposed, intending to make up for the above-mentioned shortcomings by combining FSL with domain alignment based on graph information aggregation. SD with all label samples and TD with a few label samples are implemented for FSL episodic training. Meanwhile, intradomain distribution extraction block (IDE-block) and cross-domain similarity aware block (CSA-block) are designed. The IDE-block is used to characterize and aggregate the intradomain nonlocal relationships and the interdomain feature and distribution similarities are captured in the CSA-block. Furthermore, feature-level and distribution-level cross-domain graph alignments are used to mitigate the impact of domain shift on FSL. Experimental results on three public HSI datasets demonstrate the superiority of the proposed method. The codes will be available from the website: https://github.com/YuxiangZhang-BIT/IEEE_TNNLS_Gia-CFSL.
Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification
Authors: Jie Feng, Xueliang Feng, Jiantong Chen, Xianghai Cao, Xiangrong Zhang, Licheng Jiao, Tao Yu
Date: 2020-04-03 00:00:00
Proceeding: Remote Sensing 2020 4
Tasks: Few-Shot Image Classification, Generative Adversarial Network, Hard Attention, Hyperspectral Image Classification, image-classification, Image Classification

Classifying hyperspectral images (HSIs) with limited samples is a challenging issue. The generative adversarial network (GAN) is a promising technique to mitigate the small sample size problem. GAN can generate samples by the competition between a generator and a discriminator. However, it is difficult to generate high-quality samples for HSIs with complex spatial–spectral distribution, which may further degrade the performance of the discriminator. To address this problem, a symmetric convolutional GAN based on collaborative learning and attention mechanism (CA-GAN) is proposed. In CA-GAN, the generator and the discriminator not only compete but also collaborate. The shallow to deep features of real multiclass samples in the discriminator assist the sample generation in the generator. In the generator, a joint spatial–spectral hard attention module is devised by defining a dynamic activation function based on a multi-branch convolutional network. It impels the distribution of generated samples to approximate the distribution of real HSIs both in spectral and spatial dimensions, and it discards misleading and confounding information. In the discriminator, a convolutional LSTM layer is merged to extract spatial contextual features and capture long-term spectral dependencies simultaneously. Finally, the classification performance of the discriminator is improved by enforcing competitive and collaborative learning between the discriminator and generator. Experiments on HSI datasets show that CA-GAN obtains satisfactory classification results compared with advanced methods, especially when the number of training samples is limited.
HSI-BERT: Hyperspectral Image Classification Using the Bidirectional Encoder Representation From Transformers
Authors: Ji He, Lina Zhao, HongWei Yang, Mengmeng Zhang, Wei Li
Date: 2019-09-04 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2019 9
Tasks: Classification, Few-Shot Image Classification, Hyperspectral Image Classification, image-classification, Image Classification

Deep learning methods have been widely used in hyperspectral image classification and have achieved state-of-the-art performance. Nonetheless, the existing deep learning methods are restricted by a limited receptive field, inflexibility, and difficult generalization problems in hyperspectral image classification. To solve these problems, we propose HSI-BERT, where BERT stands for bidirectional encoder representations from transformers and HSI stands for hyperspectral imagery. The proposed HSI-BERT has a global receptive field that captures the global dependence among pixels regardless of their spatial distance. HSI-BERT is very flexible and enables the flexible and dynamic input regions. Furthermore, HSI-BERT has good generalization ability because the jointly trained HSI-BERT can be generalized from regions with different shapes without retraining. HSI-BERT is primarily built on a multihead self-attention (MHSA) mechanism in an MHSA layer. Moreover, several attentions are learned by different heads, and each head of the MHSA layer encodes the semantic context-aware representation to obtain discriminative features. Because all head-encoded features are merged, the resulting features exhibit spatial-spectral information that is essential for accurate pixel-level classification. Quantitative and qualitative results demonstrate that HSI-BERT outperforms any other CNN-based model in terms of both classification accuracy and computational time and achieves state-of-the-art performance on three widely used hyperspectral image data sets.
Hyperspectral image classification via a random patches network
Authors: Yonghao Xu, Bo Du, Fan Zhang, Liangpei Zhang
Date: 2018-05-22 00:00:00
Proceeding: ISPRS Journal of Photogrammetry and Remote Sensing 2018 5
Tasks: Classification, Few-Shot Image Classification, Hyperspectral Image Classification, image-classification, Image Classification

Due to the remarkable achievements obtained by deep learning methods in the fields of computer vision, an increasing number of researches have been made to apply these powerful tools into hyperspectral image (HSI) classification. So far, most of these methods utilize a pre-training stage followed by a fine-tuning stage to extract deep features, which is not only tremendously time-consuming but also depends largely on a great deal of training data. In this study, we propose an efficient deep learning based method, namely, Random Patches Network (RPNet) for HSI classification, which directly regards the random patches taken from the image as the convolution kernels without any training. By combining both shallow and deep convolutional features, RPNet has the advantage of multi-scale, which possesses a better adaption for HSI classification, where different objects tend to have different scales. In the experiments, the proposed method and its two variants RandomNet and RPNet–single are tested on three benchmark hyperspectral data sets. The experimental results demonstrate the RPNet can yield a competitive performance compared with existing methods.