Reviewerly/exHarmony huggingface dataset:
Semantic Segmentation of Built-UP Areas in SAR Images Based on Improved YOLOv7 Model

With the continuous development of synthetic aperture radar technology, image segmentation has evolved into a pivotal method for identifying and classifying land cover. Traditional segmentation methods usually suffer from poor segmentation performance and low accuracy when dealing with dense and small targets. To address these issues, this paper proposed an improved YOLOv7 model for building object segmentation. The model was improved as follows: (1) The loss function of the original YOLOv7 model was replacd by SIoU (Scally Intersection over Union). (2) The attention mechanisms was introduced into the network. Through comparison with original YOLOv7 model, and Mask R-CNN model, the improved model achieves significant improvements in accuracy. Furthermore, the improved model outperforms other models in terms of runtime speed and power consumption.
Software Defect Detection Based on Feature Fusion and Alias Analysis

As the scale of software systems continues to increase, predicting program defects in a quick and efficient manner has become a significant research area. Recent studies have introduced deep learning models that use neural networks to extract code features and build classifiers for defect prediction. However, most existing research focuses on extracting code features at a single granularity and single level, resulting in a lack of rich code features and low prediction accuracy. To address this issue, this paper proposes a defect prediction framework based on feature fusion and alias analysis for predicting the presence of non-inferable aliases and vulnerabilities in programs. The proposed approach parses the program into two different program representations, namely Abstract Syntax Tree (AST) and Program dependency Graph (PDG), and extracts code features for feature fusion using Long Short-Term Memory (LSTM) networks and Graph Convolutional Networks (GCN), respectively. To evaluate the effectiveness of the proposed approach, the Software Assurance Reference Dataset (SARD) from the National Institute of Standards and Technology (NIST) is chosen as the experimental dataset. The classifiers constructed using the proposed approach are used to predict the presence of non-inferable aliases and vulnerabilities in programs. The experimental results demonstrate the effectiveness of the proposed approach in predicting program defects.
Variational Gaussian Filtering via Wasserstein Gradient Flows

We present a novel approach to approximate Gaussian and mixture-of-Gaussians filtering. Our method relies on a variational approximation via a gradient-flow representation. The gradient flow is derived from a Kullback-Leibler discrepancy minimization on the space of probability distributions equipped with the Wasserstein metric. We outline the general method and show its competitiveness in posterior representation and parameter estimation on two state-space models for which Gaussian approximations typically fail: systems with multiplicative noise and multi-modal state distributions.
A Statistical Model for Predicting Generalization in Few-Shot Classification

The estimation of the generalization error of classifiers often relies on a validation set. Such a set is hardly available in few-shot learning scenarios, a highly disregarded shortcoming in the field. In these scenarios, it is common to rely on features extracted from pre-trained neural networks combined with distance-based classifiers such as nearest class mean. In this work, we introduce a Gaussian model of the feature distribution. By estimating the parameters of this model, we are able to predict the generalization error on new classification tasks with few samples. We observe that accurate distance estimates between class-conditional densities are the key to accurate estimates of the generalization performance. Therefore, we propose an unbiased estimator for these distances and integrate it in our numerical analysis. We empirically show that our approach outperforms alternatives such as the leave-one-out cross-validation strategy.
A New End-to-End Sar Super-Resolution Imaging Model

The resolution of SAR image plays a crucial role in representing the system's ability to distinguish two adjacent targets in space, making it an essential parameter for measuring imaging system performance. The necessity to enhance the resolution of SAR images after imaging has become imperative, primarily due to hardware constraints and the absence of comprehensive prior knowledge. In recent years, a surge in image super-reconstruction methodologies driven by deep learning has been witnessed. Given that SAR images are complex images with both amplitude and phase information, this paper adopts a novel end-to-end model. It divides the input of the network into two channels, one for the real part and the other for the imaginary part, preserving both the amplitude and phase information of the SAR image.
Active Face Recognition Through View Synthesis

Active vision exploits the ability of robots to interact with their environment, towards increasing the quantity / quality of information obtained through their sensors and, therefore, improving their performance in perception tasks. Active face recognition is largely understudied in recent literature. In this paper, we propose an active approach that utilizes facial views produced by facial image rendering. The robot that performs face recognition selects the best candidate rotation around the person of interest by simulating the results of such movements through view synthesis. This is achieved by passing to the robot's face recognizer a real world facial image acquired in the current position, generating synthesized views that differ by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\pm\theta^{\circ}$</tex> from the current view. Then, it decides, on the basis of the confidence of the recognizer, whether to stay in place or move to the position that corresponds to one of the two synthesized views, so as to to acquire a new real image. Experimental results in two datasets verify the superior performance of the proposed method compared to the respective static approach and an approach based on the same face recognizer that involves face frontalization with synthesized views.
Attentions for Short Duration Speech Classification

Neural attention mechanisms have gained significant popularity and widespread adoption in various applications. These attention mechanisms can be applied to different models, including sequential and spatial models. Recently, the Transformer model, based on multi-head self-attention, and the ResNeSt model, utilizing split-attention, have been developed for sequential and spatial tasks, respectively. However, many attempts to leverage state-of-the-art attention methodologies for speech-related classification problems have been made without thorough analysis. In this paper, we conduct an extensive analysis of various attention-based methods by performing experiments on infant cry classification and speech emotion recognition tasks. Additionally, we evaluate the proposed models on different durations of audio clips from the Baby Chillanto and CREMA-D databases. Our results demonstrate that the Transformer (encoder-only) model significantly outperforms ResNeSt. However, for longer audio clips, ResNeSt exhibits greater robustness compared to the Transformer. Furthermore, both Transformer and ResNeSt surpass the previous state-of-the-art in infant cry classification, achieving recall improvements of 10.9% and 4.3%, respectively.
Generating Salient Scene Graphs with Weak Language Supervision

Scene Graph Generation (SGG), given an image, is the task of building directed graphs where edges represent predicted <subject - predicate - object> triplets. Most SGG models struggle to identify important and descriptive relations in images flooding the graph with triplets like <window - on - building>. This is not due to training problems but rather the lack of saliency in fully supervised SGG datasets. Hence, observing that annotators describing an image naturally omit background relations and encode image saliency we (i) introduce a generalized method for training SGG models with weak supervision using image captions, (ii) introduce two variations of the Recall@N metric which can quantify the saliency of SGG models and (iii) perform quantitative and qualitative comparisons with related literature in VG200, where we achieve up to 35 % improvement compared to re-implementation of the SOTA.
Test-Time Adversarial Detection and Robustness for Localizing Humans Using Ultra Wide Band Channel Impulse Responses

Keyless entry systems in cars are adopting neural networks for localizing its operators. Using test-time adversarial defences equip such systems with the ability to defend against adversarial attacks without prior training on adversarial samples. We propose a test-time adversarial example detector which detects the input adversarial example through quantifying the localized intermediate responses of a pretrained neural network and confidence scores of an auxiliary softmax layer. Furthermore, in order to make the network robust, we extenuate the non-relevant features by non-iterative input sample clipping. Using our approach, mean performance over 15 levels of adversarial perturbations is increased by 53.3% for the fast gradient sign method and 60.9% for both the basic iterative method and the projected gradient method when compared to adversarial training.
Deep Equilibrium Models Meet Federated Learning

In this study the problem of Federated Learning (FL) is explored under a new perspective by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL, such as the communication overhead due to the sharing large models and the ability to incorporate heterogeneous edge devices with significantly different computation capabilities. Additionally, a weighted average fusion rule is proposed at the server-side of the FL framework to account for the different qualities of models from heterogeneous edge devices. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning, contributing to the development of an efficient and effective FL framework. Finally, promising initial experimental results are presented, demonstrating the potential of this approach in addressing challenges of FL.
A Privacy-Preserving Method Using Secret Key for Convolutional Neural Network-Based Speech Classification

In this paper, we propose a privacy-preserving method with a secret key for convolutional neural network (CNN)-based speech classification tasks. Recently, many methods related to privacy preservation have been developed in image classification research fields. In contrast, in speech classification research fields, little research has considered these risks. To promote research on privacy preservation for speech classification, we provide an encryption method with a secret key in CNN - based speech classification systems. The encryption method is based on a random matrix with an invertible inverse. The encrypted speech data with a correct key can be accepted by a model with an encrypted kernel generated using an inverse matrix of a random matrix. Whereas the encrypted speech data is strongly distorted, the classification tasks can be correctly performed when a correct key is provided. Additionally, in this paper, we evaluate the difficulty of reconstructing the original information from the encrypted spectrograms and waveforms. In our experiments, the proposed encryption methods are performed in automatic speech recognition (ASR) and automatic speaker verification (ASV) tasks. The results show that the encrypted data can be used completely the same as the original data when a correct secret key is provided in the transformer-based ASR and x-vector-based ASV with self-supervised front-end systems. The robustness of the encrypted data against reconstruction attacks is also illustrated.
A Novel Graph Neural Network based Approach for Human Activity Recognition

Human activity recognition (HAR) is an important research area that involves detecting and classifying human activities using various sensors. Recently, radar-based HAR systems have attracted the attention of researchers due to their superiority over other techniques. However, one of the main challenges in HAR is capturing the spatial dependencies among the micro-Doppler features, as the spatial arrangement of the activities and their features are necessary for accurate recognition. This paper introduces a novel graph neural network model for classifying human activities using superpixel gray-scale images constructed from the range and velocity profile obtained from human activity data. Our approach utilizes a k-NN (k-nearest neighbor) graph structure to represent the superpixels as nodes, enabling the model to learn spatial dependencies among the image features. We evaluate the performance of our model with a publicly available dataset using different GNN (Graph Neural Network) techniques, including GCN (Graph Convolution Neural Network), GAT (Graph Attention Network), and SGC (Simplifying Graph Convolution Network), and conduct a comparative study to determine the most effective approach for this task. The results indicate that GAT outperforms the other state-of-the-art techniques in accurately classifying human activities with competitive accuracy.
LPC-GAN for Speech Super-Resolution

Up to today telephone speech lacks of perceptual quality and intelligibility due to bandwidth removal and quantisation artefacts in the encoding process. Super-resolution artificially regenerates this missing frequency content and thus improves the perceptual quality and intelligibility. This work proposes a novel approaches for super-resolution based on generative adversarial networks with convolutional architectures. Motivated by the source-filter model of the human speech production, the proposed system decomposes the speech signal into spectral envelope and excitation signal. The missing frequency-content of envelope and excitation are restored with dedicated networks. The network restoring the excitation signal is trained such that there is no mismatch between the excitation signal and the envelope. By this, we achieve better perceptual quality at lower computational complexity.
Pose-aware Disentangled Multiscale Transformer for Pose Guided Person Image Generation

Pose Guided Person Image Generation (PGPIG) is the task that transforms the pose of a person's image from the source image, its pose information, and the target pose information. Most existing PGPIG methods require additional pose information or tasks, which limits their application. Moreover, they use CNNs, which can only extract features from neighboring pixels and cannot consider the consistency of the entire image. This paper proposes a PGPIG network solving these problems by using a module containing Axial Transformers with large receptive field. The proposed method disentangles the PGPIG task into two subtasks: “rough pose transformation” and “detailed texture generation”. In the former task, low-resolution feature maps are transformed by blocks containing Axial Transformer. The latter task uses a CNN network with Adaptive Instance Normalization. Experiments show that the proposed method has competitive performance with other state-of-the-art methods. Furthermore, despite achieving excellent performance, the proposed network has a significantly fewer parameters than existing methods.
Deep Learning-based F0 Synthesis for Speaker Anonymization

Voice conversion for speaker anonymization is an emerging concept for privacy protection. In a deep learning setting, this is achieved by extracting multiple features from speech, altering the speaker identity, and waveform synthesis. However, many existing systems do not modify fundamental frequency (F0) trajectories, which convey prosody information and can reveal speaker identity. Moreover, mismatch between F0 and other features can degrade speech quality and intelligibility. In this paper, we formally introduce a method that synthesizes F0 trajectories from other speech features and evaluate its reconstructional capabilities. Then we test our approach within a speaker anonymization framework, comparing it to a baseline and a state-of-the-art F0 modification that utilizes speaker information. The results show that our method improves both speaker anonymity, measured by the equal error rate, and utility, measured by the word error rate.
The Discrete Orthogonal Stockwell Transforms for Infinite-Length Signals and Their Real-Time Implementations

In recent literature, the discrete Stockwell Transform (DST) for infinite length signals has been introduced along with its fast implementation. This method allows for low computational cost and enables processing of an infinite-length or large-size signal segment-by-segment while overcoming the boundary effects produced by conventional DST. The algorithm also preserves the absolute-reference phase, making it suitable for real-time signal processing. In this paper, we propose a new formulation of the discrete Orthogonal Stockwell Transform for infinite length signals. Based on the definition, we implement its fast algorithm using FFT. Our proposed scheme can process an infinite signal segment-by-segment, eliminating boundary effects and preserving the absolute-reference phase. Compared to the DST for infinite length signals, the DOST version significantly reduces computational complexity, making it more practical for real-time signal processing.
A Recursive Newton Method for Smoothing in Nonlinear State Space Models

In this paper, we use the optimization formulation of nonlinear Kalman filtering and smoothing problems to develop second-order variants of iterated Kalman smoother (IKS) meth-ods. We show that Newton's method corresponds to a recursion over affine smoothing problems on a modified state-space model augmented by a pseudo measurement. The first and second derivatives required in this approach can be efficiently computed with widely available automatic differentiation tools. Further-more, we show how to incorporate line-search and trust-region strategies into the proposed second-order IKS algorithm in order to regularize updates between iterations. Finally, we provide numerical examples to demonstrate the method's efficiency in terms of runtime compared to its batch counterpart.
Towards Robust Identification of Nonstationary Systems

The article proposes a fast, two-stage method for the identification of nonstationary systems. The method uses iterative reweighting to robustify the identification process against the outliers in the measurement noise and against the numerical errors that may occur at the first stage of identification. We also propose an adaptive algorithm to optimize the values of the hyperparameters that are crucial for this new method.
Towards Model-Agnostic Federated Learning over Networks

We present a model-agnostic federated learning method for decentralized data with an intrinsic network structure. The network structure reflects similarities between the (statistics of) local datasets and, in turn, their associated local (“personal”) models. Our method is an instance of empirical risk minimization, with the regularization term derived from the network structure of data. In particular, we require well-connected local models, forming clusters, to yield similar predictions on a common test set. The proposed method allows for a wide range of local models. The only restriction put on these local models is that they allow for efficient implementation of regularized empirical risk minimization (training). Such implementations might be available in the form of high-level programming frameworks such as scikit-learn, Keras or PyTorch.
Underwater Image Dehazing via Red-Channel Recovery

Due to light scattering and absorption while traveling through water, underwater images become hazy and loose critical information resulting in poor contrast and weak color performance. Hence, it is difficult to see the difference between foreground colors and items, and to differentiate the background in these images. To solve these issues, this study proposes a novel technique for single underwater image enhancement which relies on the recovery of the lost red-channel via a weighted multi-scale fusion. Firstly, three color balance algorithms are applied to the input image to gain more information about the scene. Then, five weight maps are extracted from these balanced versions of the input image to emphasize fine-details. Finally, the enhanced output is obtained with the new red-channel, white balanced green and blue channels followed by gamma correction to maintain contrast of the image. The developed method produces higher-quality underwater images that can be evaluated qualitatively and quantitatively when compared to state-of-the-art approaches.
Domain and Modality Adaptation Using Multi-Kernel Matching

In this paper, we propose a new method for domain and modality adaptation using multi-kernel matching. Our method is based on the representation of the source and target sets with several local kernels centered at a small number of apriori known corresponding samples. We propose to match the local kernels and, in turn, aggregate the local matches and find a mapping between the source and target sets. We showcase the applicability of our method on simulations and real-world data sets that include EEG recordings for mental arithmetic identification and single-cell multi-omics. In these applications, we demonstrate the advantages of our method over recent competing schemes.
Oral Fluency Classification for Speech Assessment

Automatic speech quality assessment finds importance in evaluating the quality of spoken speech, especially by L2 speakers. Goodness of pronunciation, stressing on the right syllable in a multi-syllable word, and oral fluency are a few main components which are assessed for a speaker. While gauging pronunciation and identifying the syllable stress is relatively standard, oral fluency assessment has large variation in the rubrics used in addition to qualitative dimension to measure the quality of fluency. In this paper, we explore, using Statistical Machine Leaning (SML) and Deep Learning (DL) models, to classify oral fluency using two publicly available datasets, namely, Avalinguo Audio Dataset (AAD) and SpeechOcean762 (SO762). We introduce pre-trained DeepSpeech model embeddings in conjunction with known speech features like Mel- Frequency Cepstral Features (MFCC) and Fluency Features (FF) to correctly predict the fluency class. The best classification accuracy obtained for AAD was 95.04%, while the same for SO762 was 77.12%.
Mask-CTC-Based Encoder Pre-Training for Streaming End-to-End Speech Recognition

Achieving high accuracy with low latency has always been a challenge in streaming end-to-end automatic speech recognition (ASR) systems. By attending to more future contexts, a streaming ASR model achieves higher accuracy but results in larger latency, which hurts the streaming performance. In the Mask-CTC framework, an encoder network is trained to learn the feature representation that anticipates long-term contexts, which is desirable for streaming ASR. Mask-CTC-based encoder pre-training has been shown beneficial in achieving low latency and high accuracy for triggered attention-based ASR. However, the effectiveness of this method has not been demonstrated for various model architectures, nor has it been verified that the encoder has the expected look-ahead capability to reduce latency. This study, therefore, examines the effectiveness of Mask-CTC-based pre-training for models with different architectures, such as Transformer-Transducer and contextual block streaming ASR. We also discuss the effect of the proposed pre-training method on obtaining accurate output spike timings, which contributes to the latency reduction in streaming ASR.