pwc-archive/papers-with-abstracts huggingface dataset:
MixFormer: End-to-End Tracking with Iterative Mixed Attention
Authors: Yutao Cui, Cheng Jiang, LiMin Wang, Gangshan Wu
Date: 2022-03-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: Semi-Supervised Video Object Segmentation, Video Object Tracking, Visual Object Tracking

Tracking often uses a multi-stage pipeline of feature extraction, target information integration, and bounding box estimation. To simplify this pipeline and unify the process of feature extraction and target information integration, we present a compact tracking framework, termed as MixFormer, built upon transformers. Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration. This synchronous modeling scheme allows to extract target-specific discriminative features and perform extensive communication between target and search area. Based on MAM, we build our MixFormer tracking framework simply by stacking multiple MAMs with progressive patch embedding and placing a localization head on top. In addition, to handle multiple target templates during online tracking, we devise an asymmetric attention scheme in MAM to reduce computational cost, and propose an effective score prediction module to select high-quality templates. Our MixFormer sets a new state-of-the-art performance on five tracking benchmarks, including LaSOT, TrackingNet, VOT2020, GOT-10k, and UAV123. In particular, our MixFormer-L achieves NP score of 79.9% on LaSOT, 88.9% on TrackingNet and EAO of 0.555 on VOT2020. We also perform in-depth ablation studies to demonstrate the effectiveness of simultaneous feature extraction and information integration. Code and trained models are publicly available at https://github.com/MCG-NJU/MixFormer.
An Introduction to Matrix factorization and Factorization Machines in Recommendation System, and Beyond
Authors: Yuefeng Zhang
Date: 2022-03-12 00:00:00
Tasks: Recommendation Systems

This paper aims at a better understanding of matrix factorization (MF), factorization machines (FM), and their combination with deep algorithms' application in recommendation systems. Specifically, this paper will focus on Singular Value Decomposition (SVD) and its derivations, e.g Funk-SVD, SVD++, etc. Step-by-step formula calculation and explainable pictures are displayed. What's more, we explain the DeepFM model in which FM is assisted by deep learning. Through numerical examples, we attempt to tie the theory to real-world problems.
Imitation Learning for Neural Network Autopilot in Fixed-Wing Unmanned Aerial Systems
Authors: Daksh Shukla, Shawn Keshmiri, Nicole Beckage
Date: 2020-09-04 00:00:00
Proceeding: International Conference on Unmanned Aircraft Systems (ICUAS) 2020 9
Tasks: Imitation Learning

This research identifies the feasibility of training an artificial neural network (ANN) autopilot, using supervised learning techniques, including an imitation learning framework known as the data aggregation set (DAgger) algorithm. The ANN autopilot aims to mimic a unified guidance, navigation & control (GNC) system to fly a fixed-wing UAS. Utilizing high-fidelity nonlinear 6-DOF aircraft simulations, it is shown that several modifications in existing imitation learning techniques must be considered. DAgger algorithm when applied sequentially to augment data along the desired flight trajectory to train the ANN autopilot is unable to generalize across turning and straight-line maneuvers, and hence cannot learn to fly stably, eventually terminating the simulation. Monte-Carlo methods when incorporated into the DAgger algorithm allow for sampling of random data along the desired flight trajectory, proved to be effective to train the ANN autopilot to fly indefinitely with acceptable tracking errors, however undesirable low frequency oscillations were observed in control input and aircraft states. The oscillatory behavior is dealt with by introducing a time-based moving window along the trajectory for data addition in conjunction with the standard DAgger algorithm. Different variations of DAgger algorithms are tested and compared using closed-loop 6-DOF flight simulations. 3D trajectory tracking and stability in aircraft states are evaluated with evidence supporting the idea that an ANN autopilot can behave as a unified GNC system and fly a fixed-wing UAS.
Controllable energy angular spectrum method
Authors: Fan Wang, Tomoyoshi Shimobaba, Takashi Kakue, Tomoyoshi Ito
Date: 2022-03-18 00:00:00
Tasks: valid

A controllable energy method, which considers the undersampling issue of the transfer function and valid spectral energy of a source signal, is proposed to implement angular spectrum diffraction calculation in near and far fields. The proposed method provides an optimized frequency boundary $f_{CE}$ within which it always keeps controllable energy to be diffracted. The controllable energy angular spectrum method significantly reduces the number of samples while having the same accuracy as previous angular spectrum methods, implying a higher calculation efficiency. The new perspective of analyzing spectral energy is shown to improve the performance of relevant diffraction calculations.
General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining
Authors: Qi Lv, Ziqiang Cao, Lei Geng, Chunhui Ai, Xu Yan, Guohong Fu
Date: 2022-03-21 00:00:00
Tasks: Domain Adaptation, token-classification, Token Classification

The lack of label data is one of the significant bottlenecks for Chinese Spelling Check (CSC). Existing researches use the method of automatic generation by exploiting unlabeled data to expand the supervised corpus. However, there is a big gap between the real input scenario and automatic generated corpus. Thus, we develop a competitive general speller ECSpell which adopts the Error Consistent masking strategy to create data for pretraining. This error consistency masking strategy is used to specify the error types of automatically generated sentences which is consistent with real scene. The experimental result indicates our model outperforms previous state-of-the-art models on the general benchmark. Moreover, spellers often work within a particular domain in real life. Due to lots of uncommon domain terms, experiments on our built domain specific datasets show that general models perform terribly. Inspired by the common practice of input methods, we propose to add an alterable user dictionary to handle the zero-shot domain adaption problem. Specifically, we attach a User Dictionary guided inference module (UD) to a general token classification based speller. Our experiments demonstrate that ECSpell$^{UD}$, namely ECSpell combined with UD, surpasses all the other baselines largely, even approaching the performance on the general benchmark.
Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression
Authors: Xiaosu Zhu, Jingkuan Song, Lianli Gao, Feng Zheng, Heng Tao Shen
Date: 2022-03-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: Image Compression, Quantization

Modeling latent variables with priors and hyperpriors is an essential problem in variational image compression. Formally, trade-off between rate and distortion is handled well if priors and hyperpriors precisely describe latent variables. Current practices only adopt univariate priors and process each variable individually. However, we find inter-correlations and intra-correlations exist when observing latent variables in a vectorized perspective. These findings reveal visual redundancies to improve rate-distortion performance and parallel processing ability to speed up compression. This encourages us to propose a novel vectorized prior. Specifically, a multivariate Gaussian mixture is proposed with means and covariances to be estimated. Then, a novel probabilistic vector quantization is utilized to effectively approximate means, and remaining covariances are further induced to a unified mixture and solved by cascaded estimation without context models involved. Furthermore, codebooks involved in quantization are extended to multi-codebooks for complexity reduction, which formulates an efficient compression procedure. Extensive experiments on benchmark datasets against state-of-the-art indicate our model has better rate-distortion performance and an impressive $3.18\times$ compression speed up, giving us the ability to perform real-time, high-quality variational image compression in practice. Our source code is publicly available at \url{https://github.com/xiaosu-zhu/McQuic}.
g2pW: A Conditional Weighted Softmax BERT for Polyphone Disambiguation in Mandarin
Authors: Yi-Chang Chen, Yu-Chuan Chang, Yen-Cheng Chang, Yi-Ren Yeh
Date: 2022-03-20 00:00:00
Tasks: Part-Of-Speech Tagging, Polyphone disambiguation, POS, POS Tagging

Polyphone disambiguation is the most crucial task in Mandarin grapheme-to-phoneme (g2p) conversion. Previous studies have approached this problem using pre-trained language models, restricted output, and extra information from Part-Of-Speech (POS) tagging. Inspired by these strategies, we propose a novel approach, called g2pW, which adapts learnable softmax-weights to condition the outputs of BERT with the polyphonic character of interest and its POS tagging. Rather than using the hard mask as in previous works, our experiments show that learning a soft-weighting function for the candidate phonemes benefits performance. In addition, our proposed g2pW does not require extra pre-trained POS tagging models while using POS tags as auxiliary features since we train the POS tagging model simultaneously with the unified encoder. Experimental results show that our g2pW outperforms existing methods on the public CPP dataset. All codes, model weights, and a user-friendly package are publicly available.
Subspace Identification of Linear Time-Periodic Systems with Periodic Inputs
Authors: Mingzhou Yin, Andrea Iannelli, Roy S. Smith
Date: 2020-06-06 00:00:00

This paper proposes a new methodology for subspace identification of linear time-periodic (LTP) systems with periodic inputs. This method overcomes the issues related to the computation of frequency response of LTP systems by utilizing the frequency response of the time-lifted system with linear time-invariant structure instead. The response is estimated with an ensemble of input-output data with periodic inputs. This allows the frequency-domain subspace identification technique to be extended to LTP systems. The time-aliased periodic impulse response can then be estimated and the order-revealing decomposition of the block-Hankel matrix is formulated. The consistency of the proposed method is proved under mild noise assumptions. Numerical simulation shows that the proposed method performs better than multiple widely-used time-domain subspace identification methods when an ensemble of periodic data is available.
Cluster & Tune: Boost Cold Start Performance in Text Classification
Authors: Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, Noam Slonim
Date: 2022-03-20 00:00:00
Proceeding: ACL 2022 5
Tasks: Classification, Clustering, text-classification, Text Classification

In real-world scenarios, a text classification task often begins with a cold start, when labeled data is scarce. In such cases, the common practice of fine-tuning pre-trained models, such as BERT, for a target classification task, is prone to produce poor performance. We suggest a method to boost the performance of such models by adding an intermediate unsupervised classification task, between the pre-training and fine-tuning phases. As such an intermediate task, we perform clustering and train the pre-trained model on predicting the cluster labels. We test this hypothesis on various data sets, and show that this additional classification phase can significantly improve performance, mainly for topical classification tasks, when the number of labeled instances available for fine-tuning is only a couple of dozen to a few hundred.
Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation
Authors: Antonin Vobecky, David Hurych, Oriane Siméoni, Spyros Gidaris, Andrei Bursuc, Patrick Pérez, Josef Sivic
Date: 2022-03-21 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Unsupervised Semantic Segmentation

This work investigates learning pixel-wise semantic image segmentation in urban scenes without any manual annotation, just from the raw non-curated data collected by cars which, equipped with cameras and LiDAR sensors, drive around a city. Our contributions are threefold. First, we propose a novel method for cross-modal unsupervised learning of semantic image segmentation by leveraging synchronized LiDAR and image data. The key ingredient of our method is the use of an object proposal module that analyzes the LiDAR point cloud to obtain proposals for spatially consistent objects. Second, we show that these 3D object proposals can be aligned with the input images and reliably clustered into semantically meaningful pseudo-classes. Finally, we develop a cross-modal distillation approach that leverages image data partially annotated with the resulting pseudo-classes to train a transformer-based model for image semantic segmentation. We show the generalization capabilities of our method by testing on four different testing datasets (Cityscapes, Dark Zurich, Nighttime Driving and ACDC) without any finetuning, and demonstrate significant improvements compared to the current state of the art on this problem. See project webpage https://vobecant.github.io/DriveAndSegment/ for the code and more.
Optimizing Camera Placements for Overlapped Coverage with 3D Camera Projections
Authors: Akshay Malhotra, Dhananjay Singh, Tushar Dadlani, Luis Yoichi Morales
Date: 2022-03-20 00:00:00
Tasks: Combinatorial Optimization

This paper proposes a method to compute camera 6Dof poses to achieve a user defined coverage. The camera placement problem is modeled as a combinatorial optimization where given the maximum number of cameras, a camera set is selected from a larger pool of possible camera poses. We propose to minimize the squared error between the desired and the achieved coverage, and formulate the non-linear cost function as a mixed integer linear programming problem. A camera lens model is utilized to project the cameras view on a 3D voxel map to compute a coverage score which makes the optimization problem in real environments tractable. Experimental results in two real retail store environments demonstrate the better performance of the proposed formulation in terms of coverage and overlap for triangulation compared to existing methods.
Federated Spatial Reuse Optimization in Next-Generation Decentralized IEEE 802.11 WLANs
Authors: Francesc Wilhelmi, Jernej Hribar, Selim F. Yilmaz, Emre Ozfatura, Kerem Ozfatura, Ozlem Yildiz, Deniz Gündüz, Hao Chen, Xiaoying Ye, Lizhao You, Yulin Shao, Paolo Dini, Boris Bellalta
Date: 2022-03-20 00:00:00
Tasks: Federated Learning

As wireless standards evolve, more complex functionalities are introduced to address the increasing requirements in terms of throughput, latency, security, and efficiency. To unleash the potential of such new features, artificial intelligence (AI) and machine learning (ML) are currently being exploited for deriving models and protocols from data, rather than by hand-programming. In this paper, we explore the feasibility of applying ML in next-generation wireless local area networks (WLANs). More specifically, we focus on the IEEE 802.11ax spatial reuse (SR) problem and predict its performance through federated learning (FL) models. The set of FL solutions overviewed in this work is part of the 2021 International Telecommunication Union (ITU) AI for 5G Challenge.
Unidirectional Thin Adapter for Efficient Adaptation of Deep Neural Networks
Authors: Han Gyel Sun, Hyunjae Ahn, Hyungyu Lee, Injung Kim
Date: 2022-03-20 00:00:00

In this paper, we propose a new adapter network for adapting a pre-trained deep neural network to a target domain with minimal computation. The proposed model, unidirectional thin adapter (UDTA), helps the classifier adapt to new data by providing auxiliary features that complement the backbone network. UDTA takes outputs from multiple layers of the backbone as input features but does not transmit any feature to the backbone. As a result, UDTA can learn without computing the gradient of the backbone, which saves computation for training significantly. In addition, since UDTA learns the target task without modifying the backbone, a single backbone can adapt to multiple tasks by learning only UDTAs separately. In experiments on five fine-grained classification datasets consisting of a small number of samples, UDTA significantly reduced computation and training time required for backpropagation while showing comparable or even improved accuracy compared with conventional adapter models.
Partitioning Image Representation in Contrastive Learning
Authors: Hyunsub Lee, Heeyoul Choi
Date: 2022-03-20 00:00:00
Tasks: Contrastive Learning, Data Augmentation, Few-Shot Learning

In contrastive learning in the image domain, the anchor and positive samples are forced to have as close representations as possible. However, forcing the two samples to have the same representation could be misleading because the data augmentation techniques make the two samples different. In this paper, we introduce a new representation, partitioned representation, which can learn both common and unique features of the anchor and positive samples in contrastive learning. The partitioned representation consists of two parts: the content part and the style part. The content part represents common features of the class, and the style part represents the own features of each sample, which can lead to the representation of the data augmentation method. We can achieve the partitioned representation simply by decomposing a loss function of contrastive learning into two terms on the two separate representations, respectively. To evaluate our representation with two parts, we take two framework models: Variational AutoEncoder (VAE) and BootstrapYour Own Latent(BYOL) to show the separability of content and style, and to confirm the generalization ability in classification, respectively. Based on the experiments, we show that our approach can separate two types of information in the VAE framework and outperforms the conventional BYOL in linear separability and a few-shot learning task as downstream tasks.
On the Computation of Necessary and Sufficient Explanations
Authors: Adnan Darwiche, Chunxi Ji
Date: 2022-03-20 00:00:00
Tasks: counterfactual

The complete reason behind a decision is a Boolean formula that characterizes why the decision was made. This recently introduced notion has a number of applications, which include generating explanations, detecting decision bias and evaluating counterfactual queries. Prime implicants of the complete reason are known as sufficient reasons for the decision and they correspond to what is known as PI explanations and abductive explanations. In this paper, we refer to the prime implicates of a complete reason as necessary reasons for the decision. We justify this terminology semantically and show that necessary reasons correspond to what is known as contrastive explanations. We also study the computation of complete reasons for multi-class decision trees and graphs with nominal and numeric features for which we derive efficient, closed-form complete reasons. We further investigate the computation of shortest necessary and sufficient reasons for a broad class of complete reasons, which include the derived closed forms and the complete reasons for Sentential Decision Diagrams (SDDs). We provide an algorithm which can enumerate their shortest necessary reasons in output polynomial time. Enumerating shortest sufficient reasons for this class of complete reasons is hard even for a single reason. For this problem, we provide an algorithm that appears to be quite efficient as we show empirically.
Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design
Authors: Won Joon Yun, Yunseok Kwak, Jae Pyoung Kim, Hyunhee Cho, Soyi Jung, Jihong Park, Joongheon Kim
Date: 2022-03-20 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In recent years, quantum computing (QC) has been getting a lot of attention from industry and academia. Especially, among various QC research topics, variational quantum circuit (VQC) enables quantum deep reinforcement learning (QRL). Many studies of QRL have shown that the QRL is superior to the classical reinforcement learning (RL) methods under the constraints of the number of training parameters. This paper extends and demonstrates the QRL to quantum multi-agent RL (QMARL). However, the extension of QRL to QMARL is not straightforward due to the challenge of the noise intermediate-scale quantum (NISQ) and the non-stationary properties in classical multi-agent RL (MARL). Therefore, this paper proposes the centralized training and decentralized execution (CTDE) QMARL framework by designing novel VQCs for the framework to cope with these issues. To corroborate the QMARL framework, this paper conducts the QMARL demonstration in a single-hop environment where edge agents offload packets to clouds. The extensive demonstration shows that the proposed QMARL framework enhances 57.7% of total reward than classical frameworks.
Interpretability of Fine-grained Classification of Sadness and Depression
Authors: Tiasa Singha Roy, Priyam Basu, Aman Priyanshu, Rakshit Naidu
Date: 2022-03-20 00:00:00
Tasks: Classification, Federated Learning

While sadness is a human emotion that people experience at certain times throughout their lives, inflicting them with emotional disappointment and pain, depression is a longer term mental illness which impairs social, occupational, and other vital regions of functioning making it a much more serious issue and needs to be catered to at the earliest. NLP techniques can be utilized for the detection and subsequent diagnosis of these emotions. Most of the open sourced data on the web deal with sadness as a part of depression, as an emotion even though the difference in severity of both is huge. Thus, we create our own novel dataset illustrating the difference between the two. In this paper, we aim to highlight the difference between the two and highlight how interpretable our models are to distinctly label sadness and depression. Due to the sensitive nature of such information, privacy measures need to be taken for handling and training of such data. Hence, we also explore the effect of Federated Learning (FL) on contextualised language models.
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation
Authors: Qingkai Fang, Rong Ye, Lei LI, Yang Feng, Mingxuan Wang
Date: 2022-03-20 00:00:00
Proceeding: ACL 2022 5
Tasks: Machine Translation, Speech-to-Text, Speech-to-Text Translation

How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data? Existing techniques often attempt to transfer powerful machine translation (MT) capabilities to ST, but neglect the representation discrepancy across modalities. In this paper, we propose the Speech-TExt Manifold Mixup (STEMM) method to calibrate such discrepancy. Specifically, we mix up the representation sequences of different modalities, and take both unimodal speech sequences and multimodal mixed sequences as input to the translation model in parallel, and regularize their output predictions with a self-learning framework. Experiments on MuST-C speech translation benchmark and further analysis show that our method effectively alleviates the cross-modal representation discrepancy, and achieves significant improvements over a strong baseline on eight translation directions.