Reviewerly/exHarmony huggingface dataset:
A Novel Approach for Data Encryption Based on Vector-Matrix Keys

The foundation of cryptography is number theory, which is crucial to data security. The majority of commonly used encryption techniques use prime integers, making it challenging to identify specific prime values (keys). The suggested approach employs matrices and vectors as keys, making it harder to identify the individual keys and using vectors to represent the data. Now a days, one method for providing data security safeguards is encryption. The right encryption technique protects digital data from unauthorized access, data corruption, e-piracy, e-theft, and other threats. Data security is the main benefit of utilizing this method. Here, we have used the symmetric key encryption procedure to generate keys from two uneven matrices. Seven different keys in matrix format have been chosen to perform encryption and decryption. With the help of an example, the techniques for encryption and decryption have been explained.
Neighborhood Information-Based Label Refinement for Person Re-Identification with Label Noise

The existing excellent person re-identification (Re-ID) model is still affected by the samples with the incorrect labels. It is difficult to accurately annotate person images in the real scene, resulting in label noise. To avoid fitting to the noisy labels, a common solution in Re-ID is to replace the original label with the label predicted by the deep model. Unfortunately, similar samples of different identities with the same label are due to label noise, which is challenging for the model to distinguish them. Neighborhood information can optimize noisy labels through neighborhood labels and similarity between samples. This paper proposes a label refinement module based on neighborhood information (LRNI) for person Re-ID with label noise. Specifically, we first use the pre-trained model to extract features and calculate the similarity between samples. Rather than treating samples as isolated, the similarity used as label propagation weight and neighborhood labels are combined to optimize noisy labels. To further reduce the influence of label noise, we design a hard sample re-weighting (HSR) strategy to balance the learning of noisy and boundary samples. Experimental results under different noise settings demonstrate our method's effectiveness in the person Re-ID task.
UAV Local Path Planning Based on Improved Proximal Policy Optimization Algorithm

Recently, more and more researchers have used deep reinforcement learning (DRL) to solve the UAV local path planning problem. However, existing DRL didn’t consider the importance of recent experience for path planning, such as proximal policy optimization (PPO). Moreover, the Actor-Critic framework of PPO suffers from the problem of high variance. To address the above issues, we proposed a Delayed-policy-update PPO with a Prioritized Reply of Recent experience (DPPO-PR2) for local path planning. Firstly, we designed an adaptive parameter to calculate the probability of resampling. By limiting the range of resampling, the possibility of resampling the recent experience is increased. Secondly, a parameter experimentally is picked to make the Actor Network have fewer updates than the Critic Network. Finally, we verified our algorithm has better convergence results and faster execution in six test scenarios.
Learning Scene Flow from 3d Point Clouds with Cross-Transformer and Global Motion Cues

Scene flow estimation is critical for real-world vision problems such as autonomous driving and augmented reality. Due to the popularity of 3D LiDAR sensors, scene flow estimation from 3D point clouds arouses increasing attention. Existing methods usually use a flow embedding-based layer to find correspondences between point pairs. However, only using a flow embedding-based layer is not enough to model the global mutual relationship between two features due to local matching. In this paper, we introduce a cross-transformer to capture more reliable dependencies for point pairs. Moreover, a global motion-aware module is adopted to learn large displacements with a non-local approach. The experimental results demonstrate that the proposed method achieves comparable performance on public datasets and confirm the effectiveness of exploiting the cross-transformer and global motion cues for scene flow estimation.
Drone-vs-Bird: Drone Detection Using YOLOv7 with CSRT Tracker

Drone detection has become a critical component of surveillance, but detecting small drones remains a challenge. In this work, we introduce a new drone detection scheme that uses a customized YOLOv7 combined with a tracker. Using simple YOLOv7 has some limitations, including false positives and difficulty detecting drones in complex environments. To address these issues, we propose two solutions. First, we use a simple method based on the probability or confidence level of detected objects to reduce the number of false positives. Second, we integrate YOLOv7 with object tracking to detect drones in complex environments. Our proposed method was successful in achieving a top-5 ranking in the 6 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> edition of the Drone-vs-Bird challenge, which was organized by the ICASSP Signal Processing Grand Challenge in 2023.
Evaluation of the Robustness, Transparency, Reliability and Safety of AI Systems

Advances in artificial intelligence have been promising in recent years. AI-related activities such as international conferences, academic research, and technological challenges have proliferated over the globe. The technologies and applications are always being improved and expanded in the modern age. In addition, humans’ studies, job, and personal lives have been greatly improved by the ongoing development of intelligent devices. Efforts have been dedicated to identify some of these issues and to present policymakers with possible remedies, which focus on reconciling the valid expectations of AI in terms of resilience and interpretability with the existing scientific environment of AI on these aspects. In this paper, the individual aims are to present an objective picture of the current environment of Artificial Intelligence (AI), emphasizing on the issues of robustness and explanability, transparency, reliability and safety. AI's present cybersecurity, security, and information security problems are discussed in detail, as well as the scientific approaches that are now being developed in the AI world to address these risks.
Client Selection for Generalization in Accelerated Federated Learning: A Bandit Approach

Federated learning (FL) is an emerging machine learning (ML) paradigm used to train models across multiple nodes (i.e., clients) holding local data sets, without explicitly exchanging the data. It has attracted a growing interest in recent years due to its advantages in terms of privacy considerations, and communication resources. In FL, selected clients train their local models and send a function of the models to the server, which consumes a random processing and transmission time. The server updates the global model and broadcasts it back to the clients. The client selection (CS) problem in FL is to schedule a subset of the clients for training and transmission at each given time so as to optimize the learning performance. In this paper, we present a novel multi-armed bandit (MAB)-based approach for CS to minimize the training latency without harming the ability of the model to generalize, i.e., to give reliable predictions for new observations. We develop a novel algorithm to achieve this goal, dubbed Bandit Scheduling for FL (BSFL). We analyze BSFL theoretically, and show that it achieves a logarithmic regret, defined as the loss of BSFL as compared to a genie that has complete knowledge about the latency means of all clients. Furthermore, simulation results using synthetic and real datasets demonstrate that BSFL is superior to existing methods.
Signal Analysis-Synthesis Using the Quantum Fourier Transform

This paper presents the development of Quantum Fourier transform (QFT) education tools in the object-oriented Java-DSP (J-DSP) simulation environment. More specifically, QFT and Inverse QFT (IQFT) user-friendly J-DSP functions are developed to expose undergraduate students to quantum computing. These functions provide opportunities to examine QFT resolution, precision (qubits), and the effects of quantum measurement noise. In our study, we also describe a laboratory exercise on QFT-based speech analysis-synthesis which has been deployed in our senior-level DSP class and in our NSF workforce development programs. The software and the laboratory exercise are evaluated using formative and summative assessments.
Multilingual Word Error Rate Estimation: E-Wer3

The success of the multilingual automatic speech recognition systems empowered many voice-driven applications. However, measuring the performance of such systems remains a major challenge, due to its dependency on manually transcribed speech data in both mono- and multilingual scenarios. In this paper, we propose a novel multilingual framework – eWER3 – jointly trained on acoustic and lexical representation to estimate word error rate. We demonstrate the effectiveness of eWER3 to (i) predict WER without using any internal states from the ASR and (ii) use the multilingual shared latent space to push the performance of the close-related languages. We show our proposed multilingual model outperforms the previous monolingual word error rate estimation method (eWER2) by an absolute 9% increase in Pearson correlation coefficient (PCC), with better overall estimation between the predicted and reference WER.
A Gaussian Latent Variable Model for Incomplete Mixed Type Data

In many machine learning problems, one has to work with data of different types, including continuous, discrete, and categorical data. Further, it is often the case that many of these data are missing from the database. This paper proposes a Gaussian process framework that efficiently captures the information from mixed numerical and categorical data that effectively incorporates missing variables. First, we propose a generative model for the mixed-type data. The generative model exploits Gaussian processes with kernels constructed from the latent vectors. We also propose a method for inference of the unknowns, and in its implementation, we rely on a sparse spectrum approximation of the Gaussian processes and variational inference. We demonstrate the performance of the method for both supervised and unsupervised tasks. First, we investigate the imputation of missing variables in an unsupervised setting, and then we show the results of joint imputation and classification on IBM employee data.
A Probabilistic Framework for Pruning Transformers Via a Finite Admixture of Keys

Pairwise dot product-based self-attention is key to the success of transformers which achieve state-of-the-art performance across a variety of applications in language and vision, but are costly to compute. It has been shown that most attention scores and keys in transformers are redundant and can be removed without loss of accuracy. In this paper, we develop a novel probabilistic framework for pruning attention scores and keys in transformers. We first formulate an admixture model of attention keys whose input data to be clustered are attention queries. We show that attention scores in self-attention correspond to the posterior distribution of this model when attention keys admit a uniform prior distribution. We then relax this uniform prior constraint and let the model learn these priors from data, resulting in a new Finite Admixture of Keys (FiAK). The learned priors are used for pruning away redundant attention scores and keys in the baseline transformers, improving the diversity of attention patterns that the models capture. We corroborate the efficiency of transformers pruned with FiAK on the ImageNet object classification and WikiText-103 language modeling tasks. Our experiments demonstrate that transformers pruned with FiAK yield similar or better accuracy than the baseline dense transformers while being much more efficient in terms of memory and computational cost.
BATT: Backdoor Attack with Transformation-Based Triggers

Deep neural networks (DNNs) are vulnerable to backdoor attacks. The backdoor adversaries intend to maliciously control the predictions of attacked DNNs by injecting hidden backdoors that can be activated by adversary-specified trigger patterns during the training process. One recent research revealed that most of the existing attacks failed in the real physical world since the trigger contained in the digitized test samples may be different from that of the one used for training. Accordingly, users can adopt spatial transformations as the image pre-processing to deactivate hidden backdoors. In this paper, we explore the previous findings from another side. We exploit classical spatial transformations (i.e., rotation and translation) with the specific parameter as trigger patterns to design a simple yet effective poisoning-based backdoor attack. For example, only images rotated to a particular angle can activate the embedded backdoor of attacked DNNs. Extensive experiments are conducted, verifying the effectiveness of our attack under both digital and physical settings and its resistance to existing backdoor defenses.
A Simulation-Based Framework for Urban Traffic Accident Detection

Urban traffic accident detection in surveillance videos is an essential task in intelligent traffic monitoring. The current deep-learning based approaches suffer from the lack of large-scale traffic accident video datasets due to the privacy concerns. In this paper, we propose a framework to synthesize traffic videos containing both normal traffic and accident events by simulating the real urban traffic scenarios. By introducing various weather conditions, viewpoints of surveillance cameras, and traffic statuses, we can simulate the complex real-world traffic scenarios. The vehicles are automatically controlled to behave normally or trigger accidents randomly. Based on our framework, we automatically generate a synthetic dataset containing 1,100 distinct traffic videos with video-level annotations. For accident detection, we observe that the existing reconstruction-based models produce many false alarms by predicting the pseudo accidents (non-accident anomalous traffic events) as real accidents. To address this ambiguity problem, we propose an improved approach by employing multiple instance learning to refine the detection results. Our experiments demonstrate the great potentials of our proposed framework in boosting the performance of accident detection on real-world datasets. We make our code and dataset publicly available at https://github.com/hankluo2/UrbanTrafficAccidentDetection.
Structure-Aware Multi-Feature Co-Learning for Dual Branch Face Super Resolution

Recently, face super-resolution has achieved pleasing performance. Numerous works have shown that texture features and structural information play a crucial role for super-resolution reconstruction. However, effective co-learning of both has been limiting the performance improvement of existing state-of-the-art methods. Therefore, we focus on the texture and structure of images in this paper, and design a two-branch network containing a texture network (T-Net) and a structure network (S-Net) to jointly explore texture and structure information for co-learning. T-Net serves as the backbone network to learn both texture and structure information for reconstruction, while the S-Net serves as the auxiliary network that can effectively exploit the multi-scale information of the T-Net encoder to recover the structure. To better facilitate the co-learning of the two branches, two co-learning modules deal with the information flow interaction between the encoder and decoder of the two branches, respectively, thus explicitly guiding the structure-aware image reconstruction. Additionally, a dense feature enhancement module investigates the channel and spatial correlation of features and enhances the representation capability of the network. Extensive experiments on the CelebA and Helen datasets show that our proposed approach outperforms state-of-the-art methods.
Higher-Order Sparse Convolutions in Graph Neural Networks

Graph Neural Networks (GNNs) have been applied to many problems in computer sciences. Capturing higher-order relationships between nodes is crucial to increase the expressive power of GNNs. However, existing methods to capture these relationships could be infeasible for large-scale graphs. In this work, we introduce a new higher-order sparse convolution based on the Sobolev norm of graph signals. Our Sparse Sobolev GNN (S-SobGNN) computes a cascade of filters on each layer with increasing Hadamard powers to get a more diverse set of functions, and then a linear combination layer weights the embeddings of each filter. We evaluate S-SobGNN in several applications of semi-supervised learning. S-SobGNN shows competitive performance in all applications as compared to several state-of-the-art methods.
Spoofed Training Data for Speech Spoofing Countermeasure Can Be Efficiently Created Using Neural Vocoders

A good training set for speech spoofing countermeasures requires diverse TTS and VC spoofing attacks, but generating TTS and VC spoofed trials for a target speaker may be technically demanding. Instead of using full-fledged TTS and VC systems, this study uses neural-network-based vocoders to do copy-synthesis on bona fide utterances. The output data can be used as spoofed data. To make better use of pairs of bona fide and spoofed data, this study introduces a contrastive feature loss that can be plugged into the standard training criterion. On the basis of the bona fide trials from the ASVspoof 2019 logical access training set, this study empirically compared a few training sets created in the proposed manner using a few neural non-autoregressive vocoders. Results on multiple test sets suggest good practices such as fine-tuning neural vocoders using bona fide data from the target domain. The results also demonstrated the effectiveness of the contrastive feature loss. Combining the best practices, the trained CM achieved overall competitive performance. Its EERs on the ASVspoof 2021 hidden subsets also outperformed the top-1 challenge submission.
Interpretable Multi-Scale Neural Network for Granger Causality Discovery

We propose a novel multi-scale neural network for Granger causality discovery (MSNGC) in multivariate time series. Compared with existing counterparts, our model avoids the explicit data segmentation between series and between time lags for the first time. By extracting diverse causal information from the data with different delay ranges and then integrating them effectively via the learned attention weights, it can capture the complete causal relationships between all the series and provide accurate weighted adjacency matrix estimation. Thus, further interpretable inference can be better supported. Specifically, we propose a consistency-based thresholding algorithm for binary causal structure inference, an effect sign detection method to distinguish the positive causal effects from the negative ones, and a self-adaptive lag discovery algorithm to identify the lagged time points. Experiments on multiple benchmarks demonstrate that our model significantly outperforms the state-of-the-art methods.
Invariant Adversarial Imitation Learning From Visual Inputs

Generalization across environments is critical when using imitation learning algorithms in real-world applications. In this paper, we propose an invariant model-based adversarial imitation learning (IMAIL) method to improve generalization. IMAIL develops a variational dynamics model providing rich auxiliary objectives for efficiently learning compact state representations. The latent representations are then regularized using mutual information constraints, guaranteeing that they are insensitive to environmental changes. Based on such representations, we utilize model-based adversarial imitation learning to mimic expert behavior in the latent space. As a result, the learned policies are well generalized in unseen environments. We conduct experiments with several vision-based control tasks to demonstrate the performance of IMAIL. Experimental results show that IMAIL significantly outperforms existing baselines and successfully achieves expert-level performance in all unseen test environments.
OAFormer: Learning Occlusion Distinguishable Feature for Amodal Instance Segmentation

The Amodal Instance Segmentation (AIS) task aims to infer the complete mask of occluded instance. Under many circumstances, existing methods treat occluded objects as unoccluded ones, and vice versa, leading to inaccurate predictions. This is because existing AIS methods do not explicitly utilize the occlusion rates of each object as supervision. However, occlusion information is critical for the methods to recognize whether the target objects are occluded. Hence we believe it is vital for the method to be distinguishable about the degree of occlusion for each instance. In this paper, a simple yet effective Occlusion-aware transformer-based model, OAFormer, is proposed for accurate amodal instance segmentation. The goal of OAFormer is to learn the occlusion discriminative features. Novel components are proposed to enable OAFormer to be occlusion distinguishable. We conduct extensive experiments on two challenging AIS datasets to evaluate the effectiveness of our method. OAFormer outperforms state-of-the-art methods by large margins.
Boosting Semi-Supervised Federated Learning with Model Personalization and Client-Variance-Reduction

Recently, federated learning (FL) has been increasingly appealing in distributed signal processing and machine learning. Nevertheless, the practical challenges of label deficiency and client heterogeneity form a bottleneck to its wide adoption. Although numerous efforts have been devoted to semi- supervised FL, most of the adopted algorithms follow the same spirit as FedAvg, thus heavily suffering from the adverse effects caused by client heterogeneity. In this paper, we boost the semi-supervised FL by addressing the issue using model personalization and client-variance-reduction. In particular, we propose a novel and unified problem formulation based on pseudo-labeling and model interpolation. We then propose an effective algorithm, named FedCPSL, which judiciously adopts the schemes of a novel momentum-based client- variance-reduction and normalized averaging. Convergence property of FedCPSL is analyzed and shows that FedCPSL is resilient to client heterogeneity and obtains a sublinear convergence rate. Experimental results on image classification tasks are also presented to demonstrate the efficacy of FedCPSL over the benchmark algorithms.
Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies

Gradient compression (GC) is a promising approach to addressing the communication bottleneck in distributed deep learning (DDL). It saves the communication time, but also incurs additional computation overheads. The training throughput of compression-enabled DDL is determined by the compression strategy, including whether to compress each tensor, the type of compute resources (e.g., CPUs or GPUs) for compression, the communication schemes for compressed tensor, and so on. However, it is challenging to find the optimal compression strategy for applying GC to DDL because of the intricate interactions among tensors. To fully unleash the benefits of GC, two questions must be addressed: 1) How to express any compression strategies and the corresponding interactions among tensors of any DDL training job? 2) How to quickly select a near-optimal compression strategy?