Reviewerly/exHarmony huggingface dataset:
Product Graph Gaussian Processes for Multi-Domain Data Imputation and Active Learning

In this work, we consider the problem of imputing signals defined on the nodes of a product graph from the subset of observations. To this end, we focus on learning their predictive probability distribution function (PDF) based Gaussian processes. In particular, we propose a product graph Gaussian process model, which incorporates the product graph structure in the Gaussian process kernel via a product graph filter. When the observed graph signals are real-valued, the mean and variance of the predictive PDF can be computed in closed form. Further, the variance captures the model uncertainty, which we use for active learning to obtain subsequent observations. We demonstrate the efficacy of the proposed method on multi-domain data imputation and active learning tasks on synthetic and real-world datasets.
A Scene-Adaptive Framework for Pose-Oriented Abnormal Event Detection

For intelligent surveillance systems, abnormal event detection (AED) automatically analyses monitoring video sequences and detects abnormal objects or strange human actions at the frame level. Due to the shortage of labelled data, most approaches for AED are based on reconstruction or prediction models in a semi-surprised manner. However, these methods may not generalize well to an unseen scene context. To address this, we present a pose-oriented scene-adaptive frame-work for AED. In this framework, we propose synergistic pose estimation and object detection, which integrates human poses and object detection information well to improve pose information accuracy. Subsequently, the enhanced pose sequences are taken into a spatial-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify the type of actions and calculate the normality scores. For evaluation, the proposed framework is tested on video sequences with unseen scene context across from UCSD PED1 & PED2 and ShanghaiTech Campus datasets. The performance analysis and the results compared with other state-of-the-art works confirm the robustness and effectiveness of our proposed framework for cross-scene AED.
Statistical Prior Knowledge for Robust Medical Image Segmentation by Level Set

Region-based active contours give interesting results when applied for objects detection with poor contrasting edges and noise, but for medical image segmentation, where objects of interest are often present with inhomogeneity, these models fail. In [16], we presented an original approach that integrate prior knowledge on the target shape into active contours with free registration based on a complete and stable set of invariant descriptors. In this work, we intend to present a novel method that incorporate statistical prior knowledge into level set model for robust medical image segmentation resulting from machine learning algorithms. In fact, for medical imaging, in many situations like tumors segmentation, prior knowledge on shape is insufficient or unavailable because it vary from a patient to another. The adopted approach is demonstrated with two unsupervised learning methods. The first one use the Fuzzy C-Means (FCM) and the second one is based on the Hidden Markov Random Field Expectation-Maximization algorithm (HMRF-EM). Both of the two are used to classify the pixels of the target image into two classes: the object and the background to construct our reference shape to be used as prior knowledge. A weighting schema is then considered to guide the active contour model under both forces: the traditional and the proposed one. We validate the robustness and the accuracy of our approach using ground truths given from experts segmentation and standard data bases. Experiments and comparative studies with several recent works are highlighted and show that the integrated method achieves good performance results.
Signed Graph Balancing with Graph Cut

Signed graphs–graphs with both positive and negative edge weights–are useful to specify pairwise dissimilarities as well as similarities in data. However, unlike graph variation operators (e.g., adjacency and graph Laplacian matrices) for unsigned graphs, the spectra of signed graph variation operators are not well understood in general. The lone exception is balanced signed graphs: there exists a one-to-one mapping of eigen-pairs between a balanced signed graph and its corresponding unsigned positive graph, which means that spectral filters for well-studied positive graphs can be reused if signed graphs are balanced. In this paper, we propose a simple yet effective method to balance a signed graph. Specifically, we balance a signed graph by removing carefully chosen edges, while the cuts of positive / negative edges are minimized / maximized, respectively. Experimental results on graph signal denoising and interpolation show that our signed graph balancing algorithms achieved promising results.
Sensor Data Representation with Transformer-Based Contrastive Learning for Human Action Recognition and Detection

Feature extraction is an important process in human activity recognition (HAR) with wearable sensors. Recent studies have shown that learned features are more effective than manually engineered features in related fields. However, the scarcity and expensiveness of labeled data are limiting the development of sensor data representation learning. Our work focuses on this issue and introduces a self-supervised learning method that uses unlabeled data to improve the quality of learned sensor representations. We hypothesize that unlabeled wearable sensor data in human activities have long-term and short-term temporal contextual correlations and exploit such correlations with Transformer and Contrastive Predictive Coding (CPC) framework. The learned representation is evaluated on human activity recognition and detection tasks in real-life scenarios. The experiments show that our method outperforms previous state-of-the-art methods on MotionSense and MobiAct datasets on the HAR task and gets a remarkable performance on the EVARS dataset on the action detection task.
Assessment of a Two-Step Integration Method as an Optimizer for Deep Learning

It is a known fact that accelerated (non-stochastic) optimization methods can be understood as multi-step integration ones: e.g. the heavy ball's Polyak and Nesterov accelerations can be derived as particular instances of a two-step integration method applied to the gradient flow. However, in the stochastic context, to the best of our knowledge, multi-step integration methods have not been exploited as such, only as some particular instances, i.e. SGD (stochastic gradient descent) with momentum or with the Nesterov acceleration. In this paper we propose to directly use a two-step (TS) integration method in the stochastic context. Furthermore, we assess the computational effectiveness of selecting the TS method's weights after considering its lattice representation. Our experiments includes several well-known multiclass classification architectures (AlexNet, VGG16 and EfficientNetV2) as well as several established stochastic optimizer e.g. SGD along with momentum/Nesterov acceleration and ADAM. The TS based method attains a better test accuracy than the first two, whereas it is competitive with to a well-tuned ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\epsilon /$</tex> learning rate) ADAM.
AirGNNs: Graph Neural Networks over the Air

Graph neural networks (GNNs) are information processing architectures that model representations from networked data and allow for decentralized implementation through localized communications. Existing GNN architectures often assume ideal communication links and ignore channel effects, such as fading and noise, leading to performance degradation in real-world implementation. This paper proposes graph neural networks over the air (AirGNNs), a novel GNN architecture that incorporates the communication model into the architecture. AirGNN modifies the graph convolutional operation that shifts graph signals over random communication graphs to take into account channel fading and noise when aggregating features from neighbors, thus, improving the architecture robustness to channel impairments during testing. We propose a stochastic gradient descent based method to train the AirGNN, and show that the training procedure converges to a stationary solution. Numerical simulations on decentralized source localization and multi-robot flocking corroborate theoretical findings and show superior performance of the AirGNN over wireless communication channels.
Quantitative Evaluation of Video Explainability Methods via Anomaly Localization

This paper presents a study of explainable AI methods applied to video anomaly detection. Specifically, we put forward a multidimensional evaluation protocol to evaluate attribution methods by considering the correctness of the explanations, their plausibility with respect to ground-truth anomaly data, and the robustness of explanations across multiple time frames. We evaluate these metrics on common gradient-based and perturbation-based explanation techniques, which we use to explain a 3D convolutional neural network trained on real video data. Our results show that using specific methods generally leads to trade-offs between the explanation performance and the higher computational cost related to video data. In particular, gradient-based methods achieve higher robustness across multiple frames, whereas perturbation methods achieve higher model fidelity scores.
Low-Resource Text-to-Speech Using Specific Data and Noise Augmentation

Many neural text-to-speech architectures can synthesize nearly natural speech from text inputs. These architectures must be trained with tens of hours of annotated and high- quality speech data. Compiling such large databases for every new voice requires a lot of time and effort. In this paper, we describe a method to extend the popular Tacotron-2 architecture and its training with data augmentation to enable single-speaker synthesis using a limited amount of specific training data. In contrast to elaborate augmentation methods proposed in the literature, we use simple stationary noises for data augmentation. Our extension is easy to implement and adds almost no computational overhead during training and inference. Using only two hours of training data, our approach was rated by human listeners to be on par with the baseline Tacotron-2 trained with 23.5 hours of LJSpeech data. In addition, we tested our model with a semantically unpredictable sentences test, which showed that both models exhibit similar intelligibility levels.
Simulating Print/Scan Textures for Morphing Attack Detection

Morphing Attack Detection is a relevant topic aiming to detect attempts of unauthorised individuals who want access to a “valid” identity. One of the main scenarios is printing morphed images and submitting the respective print in a passport application process. In order to improve the detection capabilities and spot such morphing attacks, it will be necessary to have a more realistic data set representing the passport application scenario with the diversity of devices and the resulting printed scanned or compressed images. Creating training data representing the diversity of attacks is a very demanding task because the training material is developed manually or semiautomatically. This paper proposes a transfer-style pixel-wise network for a general-purpose method to automatically create digital print/scan face images and use such images in the training of a Morphing Attack Detection (MAD) method. Our proposal can reach an Equal Error Rate (EER) of 5.13% with Random Forest and 3.17% using MobileNetV2 on the FRGCv2 database between manual print/scan and synthetics print/scan with 600 dpi. This method opens a new insight into developing attack datasets easily and time efficiently.
MUSE: A Multi-view Synthesis Enhancer

In this paper, we introduce the MUSE (MUlti-view Synthesis Enhancer) method, which is an evolution of our previously proposed HDSB method and is based on a hybrid algorithmic-learning-based scheme. MUSE generates novel views of a scene using an autoencoder specifically optimized for refining pre-synthesized views that have been derived from actual observations. Since the subjective test is the ultimate test of the visual rendering quality, we evaluate our proposed method by two subjective tests. Experimental results show that the MUSE brings a global gain compared to two tested state-of-the-art methods.
Motion Deblurring of RAW Mosaic Image Using Coded Exposure Photography

In this paper, a new motion deblurring method is investigated for reconstructing a latent image from a raw mosaic image that has been blurred by motion. Since conventional motion deblurring methods for color images often suffer from significant errors due to nonlinear processing in color image processing pipeline, we devise a new deblurring technique by generating an invertible point spread function for deblurring of raw mosaic image by using coded exposure photography. The effectiveness of this method is demonstrated using a real dataset.
Multi-target tracking pipeline for MIMO-FMCW radars based on modified GM-PHD

Multitarget tracking (MTT) with radar is challenging due to the radar's low-resolution noisy clutter-prone observation data. Classical approaches employ data association filters to deal with clutter and interference, but they are highly complex. This paper presents an improved Gaussian mixture probability hypothesis density (GM-PHD) multi-target tracking pipeline. The proposed GM-PHD solution considers (i) radar bounding-box- based measurements, (ii) a clutter model for radar observations, and (iii) robust target identification. Using both simulated and experimental scenarios, these contributions are proven to improve the tracking performance in terms of robustness against noise and clutter, as well as target identification. The significance of the presented MTT scheme lies in its low complexity since it does not need any data association. This makes it applicable for intense high-level applications for which MTT is used.
Uncertainty-Informed On-Device Personalisation Using Early Exit Networks on Sensor Signals

As their computational capabilities improve, attention has turned towards deploying deep learning models on edge devices to process the locally generated sensor signals. While these devices remain comparatively resource-constrained, early exit approaches have been shown to reduce the computational demands of on-device model personalisation training, which improves the accuracy and latency of a generalised model by fitting it to a specific use scenario. However, existing methods provide no mechanism to select the most informative signals for training. This work aims to improve prior approaches by interpreting the early exits as an ensemble of models trained with a joint loss function, retaining prior approaches' energy and latency savings while improving the accuracy. Additionally, it provides a principled mechanism to choose the signals that introduce higher uncertainty to the prediction due to the distributional shift and include them in the personalisation procedure, reducing energy consumption and latency. The key findings are a 42 % energy saving with exit-only retraining versus a standard (without intermediate exits) model, which increases up to 79 % when a subset of training samples was chosen according to the uncertainty estimation, alongside a 4.23pp increase in F1 score.
Fairness-Aware Dimensionality Reduction

In this era of digitalization, the massive increase in available data leads to great potential for advancing various domains. However, the available data, such as images, videos, and speech signals, generally lie in high-dimensional space, which calls for efficient dimensionality reduction techniques to process them efficiently. Furthermore, while the fairness of algorithms is essential for their deployment in real-world systems, the effect of dimensionality reduction on fairness is an under-explored research area. Motivated by this, this paper puts forth a fairness-aware dimensionality reduction framework that is capable of properly compressing the data while mitigating bias. Specifically, our design targets at reducing the correlation between the compressed data and sensitive attributes, while projecting the data into a new coordinate system where most of its variation can be described. Experimental results on the CelebA dataset demonstrate that the proposed dimensionality reduction framework can improve group fairness measures for image classification while providing comparable utility to the conventional techniques.
Adapting Pretrained Models for Adult to Child Voice Conversion

Due to widespread lack of parallel data for adult to child voice conversion (VC), non parallel VC techniques have grown in popularity. Methods, such as encoder-decoder model, have achieved good performance in adult-to-adult VC. It provides flexibility by either training each module separately or exploit pretrained models. These pretrained models are only available for adult speech. In case of children speech, we do not have enough data to train all the modules of a robust encoder-decoder based VC system. In a limited data scenario, we can only train the decoder module using target speech. Specifically, we find that adult to child VC using a pretrained encoder and trained decoder with child speech does not yield spectral variability of a child speech. The reason being gross spectral mismatch between adult and child speech. We address this mismatch by exploiting a warping mechanism to modify the acoustic attributes based on child speech. We conduct objective and subjective evaluations on CMU and CSLU kids corpus and one adult actress data. Results show that the proposed method reduces MCD and F0 RMSE by 0.67 and 0.03 respectively. For subjective evaluations we observe a relative MOS improvement of 10.7% for naturalness and 18.23% for similarity.
Applying Speech Derived Breathing Patterns to Automatically Classify Human Confidence

Non-verbal expressions of speech are used to understand a spectrum of human behaviour parameters; one of them being confidence. Several speech representation techniques, from hand-crafted features to auto-encoder representations, are explored for mining such information. We introduce a deep network trained with 100 speakers' data for the extraction of breathing patterns from the speech signals. This network gives an average Pearson's correlation coefficient of 0.61 and a breaths-per-minute error of 2.5 across 100 speakers. In this paper, we propose the novel use of speech-derived breathing patterns as the feature set for the binary classification of confidence levels. The classification model trained with the data from 51 interview candidates gives an average AUC of 76 % in classifying the confident speakers from the non-confident ones using breathing patterns as the feature set. On comparing this performance with that of Mel frequency cepstral coefficients and auto-encoder representations, we observe an absolute improvement of 8 % and 5 % respectively.
Are Explainability Tools Gender Biased? A Case Study on Face Presentation Attack Detection

Face recognition (FR) systems continue to spread in our daily lives with an increasing demand for higher explain-ability and interpretability of FR systems that are mainly based on deep learning. While bias across demographic groups in FR systems has already been studied, the bias of explainability tools has not yet been investigated. As such tools aim at steering further development and enabling a better understanding of computer vision problems, the possible existence of bias in their outcome can lead to a chain of biased decisions. In this paper, we explore the existence of bias in the outcome of explainability tools by investigating the use case of face presentation attack detection. By utilizing two different explainability tools on models with different levels of bias, we investigate the bias in the outcome of such tools. Our study shows that these tools show clear signs of gender bias in the quality of their explanations.
Automated Automotive Radar Calibration with Intelligent Vehicles

While automotive radar sensors are widely adopted and have been used for automatic cruise control and collision avoidance tasks, their application outside of vehicles is still limited. As they have the ability to resolve multiple targets in 3D space, radars can also be used for improving environment perception. This application, however, requires a precise calibration’ which is usually a time-consuming and labor-intensive task. We, therefore, present an approach for automated and geo-referenced extrinsic calibration of automotive radar sensors that is based on a novel hypothesis filtering scheme. Our method does not require external modifications of a vehicle and instead uses the location data obtained from automated vehicles. This location data is then combined with filtered sensor data to create calibration hypotheses. Subsequent filtering and optimization recovers the correct calibration. Our evaluation on data from a real testing site shows that our method can correctly calibrate infrastructure sensors in an automated manner, thus enabling cooperative driving scenarios.
Correlation-Based Graph Smoothness Measures In Graph Signal Processing

Graph smoothness is an important prior used for designing sampling strategies for graph signals as well as for regularizing the problem of graph learning. Additionally, smoothness is an appropriate assumption for graph signal processing (GSP) tasks such as filtering or signal recovery from samples. The most popular measure of smoothness is the quadratic form of the Laplacian, which naturally follows from the factor analysis approach. This paper presents a novel smoothness measure based on the graph correlation. The proposed measure enhances the applicability of graph smoothness measures across a variety of GSP tasks, by facilitating interoperability and generalizing across shift operators.
Voice or Content? — Exploring Impact of Speech Content on Age Estimation from Voice

To investigate the impact of speech content on age estimation accuracy from voice data, we created a corpus of speech utterances featuring identical content spoken by individuals of varying ages. Subsequently, we analyzed the age estimation outcomes derived from this dataset. Previous studies have identified biases in age-labeled speech corpora regarding speaker age and vocabulary usage. Given that speech content typically varies with the speaker's age during conversations, it's plausible that age estimation results could be influenced by speech content. To address this concern, we developed a dataset in which speakers of different ages delivered speech content that was consistent across all speakers and tailored to the characteristics of each age group. We estimated the speakers' ages both manually, through crowdsourcing, and automatically and then conducted a significance test to assess whether speech content affected the age estimation results. Our findings indicated that neither the automatic age estimation system employed nor the manual age estimation outcomes were significantly impacted by speech content.