Reviewerly/exHarmony huggingface dataset:
Self-Healing Through Error Detection, Attribution, and Retraining

Negative feedback received from users of voice agents can provide valuable training signal to their underlying ML systems. However, such systems tend to have complex inference pipelines consisting of multiple model-based and deterministic components. Therefore, when negative feedback is received, it can be difficult to attribute the system error to a specific sub-component. In this work, we address this challenge by building a system for error attribution and correction. We prototype attributing errors to the ML models used for do-main classification (DC) in the NLU component of an assistant’s pipeline, using a combination of a model and rule based system. We propose a simple method to add these detected errors directly to offline DC model training, and study our system’s effectiveness on a challenging test set of low-frequency utterances. Our experiments on nine domains suggest that augmenting DC training data with our method significantly improves performance on a majority of them.
Passive Acoustic Tracking of Whales in 3-D

Passive acoustic monitoring (PAM) is a nonintrusive approach to studying behaviors of vocalizing marine organisms underwater that otherwise would remain unexplored. In this paper, we propose a data processing chain that can detect and track multiple whales in 3-D from passively recorded underwater acoustic signals. In particular, time-difference-of-arrival (TDOA) measurements of echolocation clicks are extracted from a volumetric hydrophone array’s acoustic data by using a noise-whitening cross-correlation. For multi-target tracking, the TDOA measurements are then processed by a Bayesian inference engine consisting of two stages that is based on the sum-product algorithm (SPA). Particle flow is embedded in the SPA to make tracking computationally feasible in the considered nonlinear and high-dimensional scenario. The capability to track multiple whales without human intervention is demonstrated in scenarios with simulated and real data.
Similarity Relation Preserving Cross-Modal Learning for Multispectral Pedestrian Detection Against Adversarial Attacks

Although multispectral pedestrian detection studies have shown remarkable detection performances, they are still vulnerable to adversarial attacks. We see the similarity relations between object candidates were not maintained because of the adversarial attacks, resulting in performance degradation. In this paper, we introduce a new method that can preserve the similarity relation between candidates against adversarial attacks using multispectral knowledge. First, we propose Similarity Relation Generation (SRG) module to generate the optimal similarity relation between clean candidates by referring to the two modalities (color and thermal). Second, we propose Adversarial Similarity Relation Preserving (ASRP) module to guide the similarity relation between adversarial candidates to be similar to that of the clean candidates. By maintaining the relationship between candidates, our multispectral detector can distinguish between pedestrian/background classes even in adversarial attacks. Comprehensive experimental results show that our method conspicuously improves the adversarial robustness.
PAGE: A Position-Aware Graph-Based Model for Emotion Cause Entailment in Conversation

Conversational Causal Emotion Entailment (C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> E <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> ) is a task that aims at recognizing the causes corresponding to a target emotion in a conversation. The order of utterances in the conversation affects the causal inference. However, most current position encoding strategies ignore the order relation among utterances and speakers. To address the issue, we devise a novel position-aware graph to encode the entire conversation, fully modeling causal relations among utterances. The comprehensive experiments show that our method consistently achieves state-of-the-art performance on two challenging test sets, proving the effectiveness of our model. Our source code is available on Github <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Filter Pruning Via Filters Similarity in Consecutive Layers

Filter pruning is widely adopted to compress and accelerate the Convolutional Neural Networks (CNNs), but most previous works ignore the relationship between filters and channels in different layers. Processing each layer independently fails to utilize the collaborative relationship across layers. In this paper, we intuitively propose a novel pruning method by explicitly leveraging the Filters Similarity in Consecutive Layers (FSCL). FSCL compresses models by pruning filters whose corresponding features are more worthless in the model. The extensive experiments demonstrate the effectiveness of FSCL, and it yields remarkable improvement over state-of-the-art on accuracy, FLOPs and parameter reduction on several benchmark models and datasets.
Graph Neural Networks for Object Type Classification Based on Automotive Radar Point Clouds and Spectra

Object type classification (OTC) is an integral part of automotive radar perception. In this work, we propose graph neural networks (GNN) for radar OTC, which jointly process the radar reflection list and spectra. Combining the full set of features available in reflections and rich object representation in radar spectra in a graph structure allows a notable performance improvement, reducing the misclassification rate from 8.2% to 2.87%. With respect to implementation efficiency, we propose an approach that appends the learned spectral features directly into the reflection list, enabling a reduction of number of model parameters.
Modeling Turn-Taking in Human-To-Human Spoken Dialogue Datasets Using Self-Supervised Features

Self-supervised pre-trained models have consistently delivered state-of-art results in the fields of natural language and speech processing. However, we argue that their merits for modeling Turn-Taking for spoken dialogue systems still need further investigation. Due to that, in this paper we intro-duce a modular End-to-End system based on an Upstream + Downstream architecture paradigm, which allows easy use/integration of a large variety of self-supervised features to model the specific Turn-Taking task of End-of-Turn Detection (EOTD). Several architectures to model the EOTD task using audio-only, text-only and audio+text modalities are presented, and their performance and robustness are carefully evaluated for three different human-to-human spoken dialogue datasets. The proposed model not only achieves SOTA results for EOTD, but also brings light to the possibility of powerful and well fine-tuned self-supervised models to be successfully used for a wide variety Turn-Taking tasks.
Guide and Select: A Transformer-Based Multimodal Fusion Method for Points of Interest Description Generation

The task of Points of Interest (POI) description generation aims to generate an objective and informative description for a given POI based on POI-related information. High-quality descriptions can better guide users and improve the performance of POI-related recommendation systems. A practical POI description generation model should have effective multimodal fusion and information encoding methods suitable for various data forms. However, due to model structure and data utilization limitations, the previous method is challenging to meet the above requirements. We propose a novel Guide-Select multimodal fusion method that combines the guiding and selecting process to fuse various POI-related information efficiently. In addition, we propose a reasonable review encoding method and a category encoding method that has strong generalization ability. We integrate these methods into our Guide-Select Generation Model (GSGM). Experimental results demonstrate that our model significantly outperforms the state-of-the-art model while having a strong generalization ability on category information.
Semantic Centralized Contrastive Learning for Unsupervised Hashing

Contrastive learning has shown its potential in many unsupervised tasks, including hashing. However, the representations obtained by contrastive learning generally fail to produce no-table margins between semantic classes. Different semantic samples around the boundary are likely to collide into the same hash code. In this paper, we propose a novel Semantic Centralized Contrastive Hashing (SCCH) to allow the learned features closer to their semantic centers and more applicable to hashing. Specifically, a semantic centralization strategy is proposed by pulling strongly augmented samples towards weakly augmented ones since the weak are closer to semantic centers than the strong. Moreover, quantization directly after contrastive learning would damage the learned similarity relationship. We provide a solution to eliminate the mismatch of similarity metrics between contrastive learning and hashing mapping. Extensive experiments on three benchmark datasets demonstrate that SCCH outperforms the existing state-of-the-art methods.
Federated Semi-Supervised Learning for Object Detection in Autonomous Driving

One of the main challenges in designing deep learning networks for autonomous driving is the lack of labeled data. Recent trends that address this problem involve the use of unlabeled data. In this paper, we propose a unified semi-supervised and federated learning (FL) approach that is designed to offer cost efficient and practical training of deep learning object detection models for autonomous driving. In our implementation, we assume that each vehicle is given some well-labeled image data which are coupled with unlabeled image data captured by its cameras. Each of the vehicles has a local object detection model, which will be trained leveraging a semi-supervised learning method with both labeled and unlabeled data. The local model parameters are uploaded to a cloud server and aggregated to update a global FL model which in turn is shared with all the vehicles involved. Performance evaluations showed that our proposed approach is a promising solution as it allows continuous training and thus improved performance in autonomous driving.
Robustness-Preserving Lifelong Learning Via Dataset Condensation

Lifelong learning (LL) aims to improve a predictive model as the data source evolves continuously. Most work in this learning paradigm has focused on resolving the problem of ‘catastrophic forgetting,’ which refers to a notorious dilemma between improving model accuracy over new data and retaining accuracy over previous data. Yet, it is also known that machine learning (ML) models can be vulnerable in the sense that even tiny, adversarial input perturbations can deceive the models into producing erroneous predictions. This motivates the research objective of this paper – specification of a new LL framework that can salvage model robustness (against adversarial attacks) from catastrophic forgetting. Specifically, we propose a new memory-replay LL strategy that leverages modern bi-level optimization techniques to determine the ‘coreset’ of the current data (i.e., a small amount of data to be memorized) for ease of preserving adversarial robustness over time. We term the resulting LL framework ‘Data-Efficient Robustness-Preserving LL’ (DERPLL). The effectiveness of DERPLL is evaluated for class-incremental image classification using ResNet-18 over the CIFAR-10 dataset. Experimental results show that DERPLL outperforms the conventional coreset-guided LL baseline and achieves a substantial improvement in both standard accuracy and robust accuracy.
A Physically Explainable Framework for Human-Related Anomaly Detection

Due to the complexity in understanding human behaviors under limited observations and insufficient training data, video anomaly detection is challenging. Most of existing approaches solely rely on visual clues and suffer from noisy observations which easily lead to unreasonable predictions. In this paper, we introduce physically explainable dynamics to enhance visual representations. Firstly, a Physical Intuition (PI) module is proposed to be combined with a Visual Representation (VR) module in estimating the forces applied on subjects. Secondly, a hierarchical structure is proposed to facilitate PI module in achieving physically plausible descriptions of human movements while maintaining the consistency with visual representations. Thirdly, a novel anomaly score is proposed considering the distributions of forces. Extensive experimental results on five benchmark datasets show that state-of-the-art performance can be achieved by the proposed framework with a strong robustness.
Saba: Rethinking Datacenter Network Allocation from Application's Perspective

Today's datacenter workloads increasingly comprise distributed data-intensive applications, including data analytics, graph processing, and machine-learning training. These applications are bandwidth-hungry and often congest the datacenter network, resulting in poor network performance, which hurts application completion time. Efforts made to address this problem generally aim to achieve max-min fairness at the flow or application level. We observe that splitting the bandwidth equally among workloads is sub-optimal for aggregate application-level performance because various workloads exhibit different sensitivity to network bandwidth: for some workloads, even a small reduction in the available bandwidth yields a significant increase in completion time; for others, the completion time is largely insensitive to the available bandwidth.
EMC<sup>2</sup>-Net: Joint Equalization and Modulation Classification Based on Constellation Network

Modulation classification (MC) is the first step performed at the receiver side unless the modulation type is explicitly indicated by the transmitter. Machine learning techniques have been widely used for MC recently. In this paper, we propose a novel MC technique dubbed as Joint Equalization and Modulation Classification based on Constellation Network (EMC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net). Unlike prior works that considered the constellation points as an image, the proposed EMC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net directly uses a set of 2D constellation points to perform MC. In order to obtain clear and concrete constellation despite multipath fading channels, the proposed EMC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net consists of equalizer and classifier having separate and explainable roles via novel three-phase training and noise-curriculum pretraining. Numerical results with linear modulation types under different channel models show that the proposed EMC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net achieves the performance of state-of-the-art MC techniques with significantly less complexity.
Weakly- and Semi-Supervised Object Localization

Weakly supervised object localization deals with the lack of location-level labels to train localization models. Recently a new evaluation protocol is proposed in which full supervision is available but limited to only a small validation set. It motives us to explore semi-supervised learning for addressing this problem. In particular, the localization model is developed via self-training: we use a small amount of data with full supervision to train a class-agnostic detector, and use it to generate pseudo bounding boxes for data with weak supervision. Furthermore, we propose a selection algorithm to discover high-quality pseudo labels, and deal with data imbalance caused by pseudo labeling. We demonstrate the superiority of the proposed method with performance on par with the state of the art on two benchmarks.
Enhanced GM-PHD Filter for Real Time Satellite Multi-Target Tracking

We present a real-time multi-object tracker using an enhanced version of the Gaussian mixture probability hypothesis density (GM-PHD) filter to track detections of a state-of-the-art convolutional neural network (CNN). This approach adapts the GM-PHD filter to a real-world scenario to recover target trajectories in remote sensing videos. Our GM-PHD filter uses a measurement-driven birth, considers past tracked objects, and uses CNN information to propose better hypotheses initialization. Additionally, we present a label tracking solution for the GM-PHD filter to improve identity propagation given target path uncertainties. Our results show competitive scores against other trackers while obtaining real-time performance. Code is available at https://github.com/Ayana-Inria/RFS-filters-for-satellite-videos.
Aggregate VM: Why Reduce or Evict VM's Resources When You Can Borrow Them From Other Nodes?

Hardware resource fragmentation is a common issue in data centers. Traditional solutions based on migration or overcommitment are unacceptably slow, and modern commercial or research solutions like Spot VM may reduce or evict VM's resources anytime. We propose an alternative solution that does not suffer from these drawbacks, the Aggregate VM. We introduce a new distributed hypervisor design, the resource-borrowing hypervisor, which creates Aggregate VMs: distributed VMs that temporarily aggregate fragmented resources belonging to different host machines, which require mobility of virtual CPUs, memory and IO devices. We implement a prototype, FragVisor, which runs guest software transparently. We also propose minimal modifications to the guest OS that can enable significant performance gains. We evaluate FragVisor over a set of microbenchmarks and IaaS-style real applications. Although Aggregate VMs are not a perfect fit for every type of applications, some workloads enjoy significant speedups compared to overcommitted scenarios (up to 3.9x with 4 distributed vCPUs). We further demonstrate that FragVisor is faster than a state-of-the-art competitor, GiantVM (up to 2.5x).
TreeXGNN: can gradient-boosted decision trees help boost heterogeneous graph neural networks?

Graph neural networks are a promising deep learning method that can apply graph structures to various tasks. In real-world scenarios, we often have heterogeneous graphs, wherein different node and edge types capture complex interactions between nodes. High-dimensional node features provide rich information about the target nodes. Conventional heterogeneous graph neural networks (HGNN) focus more on graph structures than node features and may have difficulties extracting knowledge from complex node features. In this study, we propose a novel framework, the tree-boosted heterogeneous graph neural network abbreviated as TreeXGNN, which could efficiently and automatically extract target node features via gradient-boosted decision trees (GBDT). It integrates community structure information with proper fusion modules and a shared feature space design on HGNN. We achieved state-of-the-art performance on the three well-known heterogeneous graph benchmark datasets, IMDB, DBLP, and ACM, and significantly improved performance compared to previous studies. Our work paves the foundation for integrating tree-based models to boost HGNNs for general community analysis.
Ultra Real-Time Portrait Matting via Parallel Semantic Guidance

Most existing portrait matting models either require expensive auxiliary information or try to decompose the task into sub-tasks that are usually resource-hungry. These challenges limit its application on low-power computing devices. In this paper, we propose an ultra-light-weighted portrait matting network via parallel semantic guidance (PSGNet) for real-time portrait matting without any auxiliary inputs. PSGNet leverages parallel multi-level semantic information to efficiently guide the feature representations to replace traditional sequential semantic hints from objective decomposition. We also introduce an efficient fusion module to effectively combine parallel branches of PSGNet to minimize the representation redundancy. Comprehensive experiments demonstrate that our PSGNet can achieve remarkable performance on both synthetic and real-world images. Our PSGNet is capable to process at 100fps thanks to its ultra-small number of parameters, which makes it deployable on low-power computing devices without compromising on the performance of real-time portrait matting.
Color Guided Depth Map Super-Resolution with Nonlocla Autoregres-Sive Modeling

Depth map captured by 3D cameras usually suffers from low resolution and insufficient quality, which limits its applications in real world. Thus, it is an essential task to develop efficient and effective techniques to handle various depth degradations. In this paper, we propose a color guided depth map super-resolution method with nonlocal autoregressive modeling. Considering that textures in depth map demonstrate distinct geometry direction, we exploit the multi-directional dictionary which is effective in recovering subtle structures of depth patches. We further introduce two regularization terms into the sparse representation framework. Firstly, a patch based autoregressive model is introduced to represent the local patterns in a small area. Secondly, inspired by the structure consistence between depth map and color image, we propose a color guided nonlocal similarity to provide nonlocal constraint to the local structures, which is very helpful in preserving local structures and suppressing noise. Experimental results demonstrate the superior of our method compared with state-of-the-art methods.
Mitigating Unintended Memorization in Language Models Via Alternating Teaching

Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. We employ a teacher-student framework and propose a novel approach called alternating teaching to mitigate unintended memorization in sequential modeling. In our method, multiple teachers are trained on disjoint training sets whose privacy one wishes to protect, and teachers’ predictions supervise the training of a student model in an alternating manner at each time step. Experiments on LibriSpeech datasets show that the proposed method achieves superior privacy-preserving results than other counterparts. In comparison with no prevention for unintended memorization, the accuracy loss is small when training records are sufficient.
Uncertainty-Aware Few-Shot Class-Incremental Learning

In a real-world setting, machine needs to continuously recognize new categories without forgetting. However, the number of new categories may be small. For some difficult categories, even humans cannot recognize only based on few-shot examples. To address the above issues, an innovative uncertainty-aware few-shot class incremental learning method (UACL) is proposed, which allows the model to continuously recognize new classes with few-shot examples and identify the classes it cannot recognize currently. Besides, in order to imitate the cognitive way of human beings and improve the continuous representation ability, we propose a pseudo-incremental task construction mechanism based on uncertainty estimation, where the machine learn to recognize from simple to difficult. Further, a large-scale pre-training model is used as an expert system to guide the model to recognize difficult classes. We evaluate our method on three popular benchmark datasets, showing that UACL is state-of-the-art.