pwc-archive/papers-with-abstracts huggingface dataset:
VFP290K: A Large-Scale Benchmark Dataset for Vision-based Fallen Person Detection
Authors: Jaeju An, Jeongho Kim, Hanbeen Lee, Jinbeom Kim, Junhyung Kang, Saebyeol Shin, Minha Kim, Donghee Hong, Simon S. Woo
Date: 2022-01-14 00:00:00
Proceeding: NeurIPS 2021 Track Datasets and Benchmarks 2022 1
Tasks: Anomaly Detection In Surveillance Videos, Human Detection, object-detection, Object Detection

Detection of fallen persons due to, for example, health problems, violence, or accidents, is a critical challenge. Accordingly, detection of these anomalous events is of paramount importance for a number of applications, including but not limited to CCTV surveillance, security, and health care. Given that many detection systems rely on a comprehensive dataset comprising fallen person images collected under diverse environments and in various situations is crucial. However, existing datasets are limited to only specific environmental conditions and lack diversity. To address the above challenges and help researchers develop more robust detection systems, we create a novel, large-scale dataset for the detection of fallen persons composed of fallen person images collected in various real-world scenarios, with the support of the South Korean government. Our Vision-based Fallen Person (VFP290K) dataset consists of 294,713 frames of fallen persons extracted from 178 videos, including 131 scenes in 49 locations. We empirically demonstrate the effectiveness of the features through extensive experiments analyzing the performance shift based on object detection models. In addition, we evaluate our VFP290K dataset with properly divided versions of our dataset by measuring the performance of fallen person detecting systems. We ranked first in the first round of the anomalous behavior recognition track of AI Grand Challenge 2020, South Korea, using our VFP290K dataset, which can be found here. Our achievement implies the usefulness of our dataset for research on fallen person detection, which can further extend to other applications, such as intelligent CCTV or monitoring systems. The data and more up-to-date information have been provided at our VFP290K site.
FOR THE SAKE OF PRIVACY: SKELETON-BASED SALIENT BEHAVIOR RECOGNITION
Authors: Thomas Golda, Johanna Thiemich, Mickael Cormier, Jurgen Beyerer
Date: 2022-10-18 00:00:00
Proceeding: 2022 IEEE International Conference on Image Processing (ICIP) 2022 10
Tasks: Anomaly Detection In Surveillance Videos

Authorities as well as emergency and rescue services have an increasing interest in smart support systems to ensure public safety which includes in particular behavioral analysis of pedestrians by using video surveillance systems. In order to accommodate concerns of citizens regarding their personal rights, the demand for data privacy friendly approaches, using as few information as possible, arises. In this paper, we examine existing approaches tackling the recognition of anomalous or salient behavior based solely on person pose information within the context of real-world surveillance applications. Particularly, we chose two existing state-of-the-art approaches and evaluate them on two public and an internal dataset in order to examine the overall performance of these methods for the desired task. Furthermore, we present our
own approach achieving comparable results to these methods. Finally, we extend the aforementioned methods with a memory extension for modeling normal behavior, which yields on average a 4.3% higher recognition performance.
Predicting Soil Properties from Hyperspectral Satellite Images
Authors: Rıdvan Salih Kuzu, Frauke Albrecht, Caroline Arnold, Roshni Kamath
Date: 2022-10-18 00:00:00
Proceeding: Conference 2022 10
Tasks: Seeing Beyond the Visible

The AI4EO HYPERVIEW challenge seeks machine learning
methods that predict agriculturally relevant soil parameters
(K, Mg, P2O5, pH) from airborne hyperspectral images.
We present a hybrid model fusing Random Forest and K-
nearest neighbor regressors that exploit the average spectral
reflectance, as well as derived features such as gradients,
wavelet coefficients, and Fourier transforms. The solution is
computationally lightweight and improves upon the challenge
baseline by 21.9%, with the first place on the public leader-
board. In addition, we discuss neural network architectures
and potential future improvements.
DGSSC: A Deep Generative Spectral-Spatial Classifier for Imbalanced Hyperspectral Imagery
Authors: Bobo Xi, Jiaojiao Li, Yan Diao, Yunsong Li, Zan Li, Yan Huang, Jocelyn Chanussot
Date: 2022-10-19 00:00:00
Proceeding: IEEE Transactions on Circuits and Systems for Video Technology 2022 10
Tasks: Data Augmentation, Decoder, Hyperspectral Image Classification, image-classification, Image Classification

In recent years, hyperspectral image classification (HSIC) has achieved impressive progress with emerging studies on deep learning models. However, the classification performance downgrades due to the limited number of annotated samples, especially for minority classes. Notably, the imbalanced data dilemma is familiar in remote sensing hyperspectral image because the ground objects are commonly distributed without evenness. Therefore, this paper proposes a novel deep generative spectral-spatial classifier (DGSSC) for addressing the issues of imbalanced HSIC. Specifically, the DGSSC comprises three components, a two-stage encoder, a decoder, and a classifier, which are trained in an end-to-end manner. In particular, to exploit the abundant spectral-spatial features with relatively low computational complexity, the first stage of the encoder comprises successive three-dimensional (3D) and two-dimensional (2D) convolutions, exploring the spectral-spatial and deep spatial information. In addition, the second stage involves the deep latent variable model to achieve minority-class data augmentation. Furthermore, a patch distance-based reconstruction loss function is designed to facilitate the outputs of the decoder being more similar to the input 3D patch samples. The proposed DGSSC can outperform the state-of-the-art methods on three benchmark datasets, especially with its more robust prediction results. For instance, the DGSSC achieves a remarkable 97.85% mean overall accuracy with 0.24% standard deviation over ten independent runs with randomly selected imbalanced 1% training samples on the University of Pavia dataset.
CTGAN : Cloud Transformer Generative Adversarial Network
Authors: Gi-Luen Huang, Pei-Yuan Wu
Date: 2022-10-18 00:00:00
Proceeding: 2022 International Conference on Image Processing (ICIP 2022) 2022 10
Tasks: Generative Adversarial Network, Land Cover Classification

Cloud occlusions obstruct some applications of remote sensing imagery, such as environment monitoring, land cover classification, and poverty prediction. In this paper, we propose the Cloud Transformer Generative Adversarial Network (CTGAN), taking three temporal cloudy images as input and generating a corresponding cloud-free image. Unlike previous work using generative networks, we design the feature extractor to maintain the weight of the cloudless region while reducing the weight of the cloudy region, and we pass the extracted features to a conformed module to find the most critical representations. Meanwhile, to address the lack of datasets, we collected a new dataset named Sen2 MTC from the Sentinel-2 satellite and manually labeled each cloudy and cloud-free image. Finally, we conducted extensive experiments on FS-2, the STGAN dataset, and Sen2 MTC. Our proposed CTGAN demonstrates higher qualitative and quantitative performance than the previous work and achieves state-of-the-art performance on these three datasets. The code is available at https://github.com/come880412/CTGAN
Decoding finger movements from ECoG signals using switching linear models
Authors: Rémi Flamary, Alain Rakotomamonjy
Date: 2012-03-06 00:00:00
Proceeding: Front. Neurosci., Sec. Neuroprosthetics 2012 3
Tasks: Brain Decoding

One of the most interesting challenges in ECoG-based Brain-Machine Interface is movement prediction. Being able to perform such a prediction paves the way to high-degree precision command for a machine such as a robotic arm or robotic hands. As a witness of the BCI community increasing interest toward such a problem, the fourth BCI Competition provides a dataset which aim is to predict individual finger movements from ECoG signals. The difficulty of the problem relies on the fact that there is no simple relation between ECoG signals and finger movements. We propose in this paper, to estimate and decode these finger flexions using switching models controlled by an hidden state. Switching models can integrate prior knowledge about the decoding problem and helps in predicting fine and precise movements. Our model is thus based on a first block which estimates which finger is moving and another block which, knowing which finger is moving, predicts the movements of all other fingers. Numerical results that have been submitted to the Competition show that the model yields high decoding performances when the hidden state is well estimated. This approach achieved the second place in the BCI competition with a correlation measure between real and predicted movements of 0.42.
G2L: A Global to Local Alignment Method for Unsupervised Domain Adaptive Semantic Segmentation
Authors: Nguyen Viet Manh, Kieu Dang Nam, Dinh Viet Sang, Thi-Oanh Nguyen
Date: 2022-09-07 00:00:00
Proceeding: KES 2022 9
Tasks: Denoising, Domain Adaptation, Image-to-Image Translation, Semantic Segmentation, Synthetic-to-Real Translation, Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer knowledge from a source dataset with dense pixel-level annotations to an unlabeled target dataset. However, the performance of UDA methods often suffers from the domain shift, which is the discrepancy between the feature distributions of the two domains. There have been several attempts to match these distributions at the image level marginally. However, due to the so-called category-level domain shift, such global alignments do not guarantee a good separability of deep features extracted from different categories in the target domain. As a result, the generated pseudo-labels can be noisy and thus poison the learning process on the target domain. Some recent methods focus on denoising the pseudo-labels online using category-wise information. This paper introduces a novel UDA method called Global-to-Local alignment (G2L) that leverages fine-grained adversarial training and a newly proposed chromatic Fourier transform to address the image-level domain shift from a global perspective. Next, our method deals with the category-level domain shift under a local view. Specifically, we propose a long-tail category rating strategy as well as apply dynamic confidence thresholds and category-wise priority weights when generating and denoising the pseudo-labels to favor rare categories. Finally, self-distillation is used to boost the final segmentation results. Experiments on popular benchmarks GTA5 → Cityscapes and SYNTHIA → Cityscapes show that our method yields superior accuracy performance than other state-of-the-art methods.
Decoding finger flexion from band-specific ECoG signals in humans
Authors: Nanying Liang, Laurent Bougrain
Date: 2012-06-28 00:00:00
Proceeding: Frontiers in Neuroscience, Section Neuroprosthetics 2012 6
Tasks: Brain Computer Interface, Brain Decoding, EEG, Electroencephalogram (EEG)

This article presents the method that won the brain-computer interface (BCI) competition IV addressed to the prediction of the finger flexion from electrocorticogram (ECoG) signals. ECoG-based BCIs have recently drawn the attention from the community. Indeed, ECoG can provide higher spatial resolution and better signal quality than classical EEG recordings. It is also more suitable for long-term use. These characteristics allow to decode precise brain activities and to realize efficient ECoG-based neuroprostheses. Signal processing is a very important task in BCIs research for translating brain signals into commands. Here, we present a linear regression method based on the amplitude modulation of band-specific ECoG including a short-term memory for individual finger flexion prediction. The effectiveness of the method was proven by achieving the highest value of correlation coefficient between the predicted and recorded finger flexion values on data set 4 during the BCI competition IV.
A library of human electrocorticographic data and analyses
Authors: Miller, Kai Joshua
Date: 2019-10-03 00:00:00
Proceeding: Nature 2019 10
Tasks: Anatomy

Electrophysiological data from implanted electrodes in the human brain are rare, and therefore scientific access to such data has remained somewhat exclusive. Here we present a freely available curated library of implanted electrocorticographic data and analyses for 16 behavioural experiments, with 204 individual datasets from 34 patients recorded with the same amplifiers and at the same settings. For each dataset, electrode positions were carefully registered to brain anatomy. A large set of fully annotated analysis scripts with which to interpret these data is embedded in the library alongside them. All data, anatomical locations and analysis files (MATLAB code) are provided in a shared file structure at https://searchworks.stanford.edu/view/zk881ps0522.
Segmentation of patchy areas in biomedical images based on local edge density estimation
Authors: Aleksandr M. Sinitca, Airat R. Kayumov, Pavel V. Zelenikhin, Andrey G. Porfiriev, Dmitrii I. Kaplun, Mikhail I. Bogachev
Date: 2022-09-22 00:00:00
Proceeding: Biomedical Signal Processing and Control 2022 9
Tasks: Density Estimation, Image Segmentation, Segmentation

We suggest an effective approach for the semi-automated segmentation of biomedical images according to their patchiness based on local edge density estimation. Our approach does not require any preliminary learning or tuning, although a couple of free parameters directly controllable by the end user adjust the analysis resolution and sensitivity, respectively. We show explicitly that the local edge density exhibits excellent correlations with the cell monolayer density obtained by manual domain-expert based assessment, characterized by correlation coefficients . Our results indicate that the proposed algorithm is capable of an efficient segmentation and quantification of patchy areas in various biomedical microscopic images. In particular, the proposed algorithm achieves 95 to 99% median accuracy in the segmentation of image areas covered by the cell monolayer in an in vitro scratch assay. Moreover, the proposed algorithm effectively distinguishes between the native and regenerated tissue fragments in microscopic images of histological sections, indicated by nearly three-fold discrepancy between the local edge densities in the corresponding image areas. We believe that the local edge density estimate could be further applicable as a surrogate image channel characterizing its patchiness either as a substitute or as a complementary source to the conventional cell- or tissue-specific fluorescent staining, in some cases either avoiding or limiting the use of complex experimental protocols. We implemented a simple open-source software tool with for on-the-fly visualization allowing for a straightforward feedback by a domain expert without any specific expertise in image analysis techniques. Our tool is freely available online at https://gitlab.com/digiratory/biomedimaging/bcanalyzer.
Decoding of finger trajectory from ECoG using deep learning
Authors: Ziqian Xie, Odelia Schwartz, Abhishek Prasad
Date: 2018-02-28 00:00:00
Proceeding: Journal of Neural Engineering 2018 2
Tasks: Brain Decoding, Deep Learning, feature selection, regression

Objective. Conventional decoding pipeline for brain–machine interfaces (BMIs) consists of chained different stages of feature extraction, time-frequency analysis and statistical learning models. Each of these stages uses a different algorithm trained in a sequential manner, which makes it difficult to make the whole system adaptive. The goal was to create an adaptive online system with a single objective function and a single learning algorithm so that the whole system can be trained in parallel to increase the decoding performance. Here, we used deep neural networks consisting of convolutional neural networks (CNN) and a special kind of recurrent neural network (RNN) called long short term memory (LSTM) to address these needs. Approach. We used electrocorticography (ECoG) data collected by Kubanek et al. The task consisted of individual finger flexions upon a visual cue. Our model combined a hierarchical feature extractor CNN and a RNN that was able to process sequential data and recognize temporal dynamics in the neural data. CNN was used as the feature extractor and LSTM was used as the regression algorithm to capture the temporal dynamics of the signal. Main results. We predicted the finger trajectory using ECoG signals and compared results for the least angle regression (LARS), CNN-LSTM, random forest, LSTM model (LSTM_HC, for using hard-coded features) and a decoding pipeline consisting of band-pass filtering, energy extraction, feature selection and linear regression. The results showed that the deep learning models performed better than the commonly used linear model. The deep learning models not only gave smoother and more realistic trajectories but also learned the transition between movement and rest state. Significance. This study demonstrated a decoding network for BMI that involved a convolutional and recurrent neural network model. It integrated the feature extraction pipeline into the convolution and pooling layer and used LSTM layer to capture the state transitions. The discussed network eliminated the need to separately train the model at each step in the decoding pipeline. The whole system can be jointly optimized using stochastic gradient descent and is capable of online learning.
Interpreting wide-band neural activity using convolutional neural networks
Authors: Markus Frey, Sander Tanni, Catherine Perrodin, Alice O'Leary, Matthias Nau, Jack Kelly, Andrea Banino, Daniel Bendor, Julie Lefort, Christian F Doeller, Caswell Barry
Date: 2021-08-02 00:00:00
Proceeding: eLife 2021 8
Tasks: Brain Decoding, Hippocampus

Rapid progress in technologies such as calcium imaging and electrophysiology has seen a dramatic increase in the size and extent of neural recordings. Even so, interpretation of this data often depends on manual operations and requires considerable knowledge about the nature of the representation. Decoding provides a means to infer the information content of such recordings but typically requires highly processed data and prior knowledge of the encoding scheme. Here, we
developed a deep-learning-framework able to decode sensory and behavioural variables directly
from wide-band neural data. The network requires little user input and generalizes across stimuli,
behaviours, brain regions, and recording techniques. Once trained, it can be analysed to determine
elements of the neural code that are informative about a given variable. We validated this approach
using data from rodent auditory cortex and hippocampus, identifying a novel representation of
head direction encoded by putative CA1 interneurons.
Decoding and interpreting cortical signals with a compact convolutional neural network
Authors: Artur Petrosyan, Mikhail Sinkin, Mikhail Lebedev, Alexei Ossadtchi
Date: 2021-03-02 00:00:00
Proceeding: Journal of Neural Engineering 2021 3
Tasks: Brain Decoding, EEG, Electroencephalogram (EEG), Feature Engineering, Motor Imagery

Objective. Brain–computer interfaces (BCIs) decode information from neural activity and send it to external devices. The use of Deep Learning approaches for decoding allows for automatic feature engineering within the specific decoding task. Physiologically plausible interpretation of the network parameters ensures the robustness of the learned decision rules and opens the exciting opportunity for automatic knowledge discovery. Approach. We describe a compact convolutional network-based architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also propose a novel theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time. The obtained spatial and frequency patterns characterizing the neuronal populations pivotal to the specific decoding task can then be interpreted by fitting appropriate spatial and dynamical models. Main results. We first tested our solution using realistic Monte-Carlo simulations. Then, when applied to the ECoG data from Berlin BCI competition IV dataset, our architecture performed comparably to the competition winners without requiring explicit feature engineering. Using the proposed approach to the network weights interpretation we could unravel the spatial and the spectral patterns of the neuronal processes underlying the successful decoding of finger kinematics from an ECoG dataset. Finally we have also applied the entire pipeline to the analysis of a 32-channel EEG motor-imagery dataset and observed physiologically plausible patterns specific to the task. Significance. We described a compact and interpretable CNN architecture derived from the basic principles and encompassing the knowledge in the field of neural electrophysiology. For the first time in the context of such multibranch architectures with factorized spatial and temporal processing we presented theoretically justified weights interpretation rules. We verified our recipes using simulations and real data and demonstrated that the proposed solution offers a good decoder and a tool for investigating motor control neural mechanisms.
Fast and accurate decoding of finger movements from ECoG through Riemannian features and modern machine learning techniques
Authors: Lin Yao, Bingzhao Zhu, Mahsa Shoaran
Date: 2022-02-25 00:00:00
Proceeding: Journal of Neural Engineering 2022 2
Tasks: Brain Computer Interface, Brain Decoding

Objective.Accurate decoding of individual finger movements is crucial for advanced prosthetic control. In this work, we introduce the use of Riemannian-space features and temporal dynamics of electrocorticography (ECoG) signal combined with modern machine learning (ML) tools to improve the motor decoding accuracy at the level of individual fingers.
Approach. We selected a set of informative biomarkers that correlated with finger movements and evaluated the performance of state-of-the-art ML algorithms on the brain-computer interface (BCI) competition IV dataset (ECoG, three subjects) and a second ECoG dataset with a similar recording paradigm (Stanford, nine subjects). We further explored the temporal concatenation of features to effectively capture the history of ECoG signal, which led to a significant improvement over single-epoch decoding in both classification (p < 0.01) and regression tasks (p < 0.01).
Main results. Using feature concatenation and gradient boosted trees (the top-performing model), we achieved a classification accuracy of 77.0% in detecting individual finger movements (six-class task, including rest state), improving over the state-of-the-art conditional random fields by 11.7% on the three BCI competition subjects. In continuous decoding of movement trajectory, our approach resulted in an average Pearson's correlation coefficient (r) of 0.537 across subjects and fingers, outperforming both the BCI competition winner and the state-of-the-art approach reported on the same dataset (CNN + LSTM). Furthermore, our proposed method features a low time complexity, with only<17.2 s required for training and<50 ms for inference. This enables about 250× speed-up in training compared to previously reported deep learning method with state-of-the-art performance.
Significance.The proposed techniques enable fast, reliable, and high-performance prosthetic control through minimally-invasive cortical signals.