pwc-archive/papers-with-abstracts huggingface dataset:
Controllable Summarization with Constrained Markov Decision Process
Authors: Hou Pong Chan, Lu Wang, Irwin King
Date: 2021-08-07 00:00:00
Tasks: Attribute, Text Summarization

We study controllable text summarization which allows users to gain control on a particular attribute (e.g., length limit) of the generated summaries. In this work, we propose a novel training framework based on Constrained Markov Decision Process (CMDP), which conveniently includes a reward function along with a set of constraints, to facilitate better summarization control. The reward function encourages the generation to resemble the human-written reference, while the constraints are used to explicitly prevent the generated summaries from violating user-imposed requirements. Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness, as we devise specific constraints for each of these aspects. Extensive experiments on popular benchmarks show that our CMDP framework helps generate informative summaries while complying with a given attribute's requirement.
On Measures of Biases and Harms in NLP
Authors: Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
Date: 2021-08-07 00:00:00

Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. To create interventions and mitigate these biases and associated harms, it is vital to be able to detect and measure such biases. While existing works propose bias evaluation and mitigation methods for various tasks, there remains a need to cohesively understand the biases and the specific harms they measure, and how different measures compare with each other. To address this gap, this work presents a practical framework of harms and a series of questions that practitioners can answer to guide the development of bias measures. As a validation of our framework and documentation questions, we also present several case studies of how existing bias measures in NLP -- both intrinsic measures of bias in representations and extrinsic measures of bias of downstream applications -- can be aligned with different harms and how our proposed documentation questions facilitates more holistic understanding of what bias measures are measuring.
Tiny Neural Models for Seq2Seq
Authors: Arun Kandoor
Date: 2021-08-07 00:00:00
Tasks: Decoder, Semantic Parsing

Semantic parsing models with applications in task oriented dialog systems require efficient sequence to sequence (seq2seq) architectures to be run on-device. To this end, we propose a projection based encoder-decoder model referred to as pQRNN-MAtt. Studies based on projection methods were restricted to encoder-only models, and we believe this is the first study extending it to seq2seq architectures. The resulting quantized models are less than 3.5MB in size and are well suited for on-device latency critical applications. We show that on MTOP, a challenging multilingual semantic parsing dataset, the average model performance surpasses LSTM based seq2seq model that uses pre-trained embeddings despite being 85x smaller. Furthermore, the model can be an effective student for distilling large pre-trained models such as T5/BERT.
Towards Zero-shot Language Modeling
Authors: Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell, Roi Reichart, Anna Korhonen
Date: 2021-08-06 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Language Modeling, Language Modelling

Can we construct a neural model that is inductively biased towards learning human languages? Motivated by this question, we aim at constructing an informative prior over neural weights, in order to adapt quickly to held-out languages in the task of character-level language modeling. We infer this distribution from a sample of typologically diverse training languages via Laplace approximation. The use of such a prior outperforms baseline models with an uninformative prior (so-called "fine-tuning") in both zero-shot and few-shot settings. This shows that the prior is imbued with universal phonological knowledge. Moreover, we harness additional language-specific side information as distant supervision for held-out languages. Specifically, we condition language models on features from typological databases, by concatenating them to hidden states or generating weights with hyper-networks. These features appear beneficial in the few-shot setting, but not in the zero-shot setting. Since the paucity of digital texts affects the majority of the world's languages, we hope that these findings will help broaden the scope of applications for language technology.
Facebook AI WMT21 News Translation Task Submission
Authors: Chau Tran, Shruti Bhosale, James Cross, Philipp Koehn, Sergey Edunov, Angela Fan
Date: 2021-08-06 00:00:00
Tasks: All, Reranking, Translation

We describe Facebook's multilingual model submission to the WMT2021 shared task on news translation. We participate in 14 language directions: English to and from Czech, German, Hausa, Icelandic, Japanese, Russian, and Chinese. To develop systems covering all these directions, we focus on multilingual models. We utilize data from all available sources --- WMT, large-scale data mining, and in-domain backtranslation --- to create high quality bilingual and multilingual baselines. Subsequently, we investigate strategies for scaling multilingual model size, such that one system has sufficient capacity for high quality representations of all eight languages. Our final submission is an ensemble of dense and sparse Mixture-of-Expert multilingual translation models, followed by finetuning on in-domain news data and noisy channel reranking. Compared to previous year's winning submissions, our multilingual system improved the translation quality on all language directions, with an average improvement of 2.0 BLEU. In the WMT2021 task, our system ranks first in 10 directions based on automatic evaluation.
Guaranteeing Safety for Neural Network-Based Aircraft Collision Avoidance Systems
Authors: Kyle D. Julian, Mykel J. Kochenderfer
Date: 2019-12-15 00:00:00
Tasks: Collision Avoidance

The decision logic for the ACAS X family of aircraft collision avoidance systems is represented as a large numeric table. Due to storage constraints of certified avionics hardware, neural networks have been suggested as a way to significantly compress the data while still preserving performance in terms of safety. However, neural networks are complex continuous functions with outputs that are difficult to predict. Because simulations evaluate only a finite number of encounters, simulations are not sufficient to guarantee that the neural network will perform correctly in all possible situations. We propose a method to provide safety guarantees when using a neural network collision avoidance system. The neural network outputs are bounded using neural network verification tools like Reluplex and Reluval, and a reachability method determines all possible ways aircraft encounters will resolve using neural network advisories and assuming bounded aircraft dynamics. Experiments with systems inspired by ACAS X show that neural networks giving either horizontal or vertical maneuvers can be proven safe. We explore how relaxing the bounds on aircraft dynamics can lead to potentially unsafe encounters and demonstrate how neural network controllers can be modified to guarantee safety through online costs or lowering alerting cost. The reachability method is flexible and can incorporate uncertainties such as pilot delay and sensor error. These results suggest a method for certifying neural network collision avoidance systems for use in real aircraft.
Hidden Markov Model: Tutorial
Authors: Benyamin Ghojogh, Fakhri Karray, Mark Crowley
Date: 2019-12-10 00:00:00
Proceeding: engrXiv 2019 12
Tasks: Action Recognition, Bayesian Inference, Bayesian Optimisation, model, Probabilistic Time Series Forecasting, Sequential Bayesian Inference, Time Series Analysis, Time Series Forecasting, Time Series Prediction

This is a tutorial paper for Hidden Markov Model (HMM). First, we briefly review the background on Expectation Maximization (EM), Lagrange multiplier, factor graph, the sum-product algorithm, the max-product algorithm, and belief propagation by the forward-backward procedure. Then, we introduce probabilistic graphical models including Markov random field and Bayesian network. Markov property and Discrete Time Markov Chain (DTMC) are also introduced. We, then, explain likelihood estimation and EM in HMM in technical details. We explain evaluation in HMM where direct calculation and the forward-backward belief propagation are both explained. Afterwards, estimation in HMM is covered where both the greedy approach and the Viterbi algorithm are detailed. Then, we explain how to train HMM using EM and the Baum-Welch algorithm. We also explain how to use HMM in some applications such as speech and action recognition.
Development and evaluation of intraoperative ultrasound segmentation with negative image frames and multiple observer labels
Authors: Liam F Chalcroft, Jiongqi Qu, Sophie A Martin, Iani JMB Gayo, Giulio V Minore, Imraj RD Singh, Shaheer U Saeed, Qianye Yang, Zachary MC Baum, Andre Altmann, Yipeng Hu
Date: 2021-07-28 00:00:00
Tasks: Segmentation

When developing deep neural networks for segmenting intraoperative ultrasound images, several practical issues are encountered frequently, such as the presence of ultrasound frames that do not contain regions of interest and the high variance in ground-truth labels. In this study, we evaluate the utility of a pre-screening classification network prior to the segmentation network. Experimental results demonstrate that such a classifier, minimising frame classification errors, was able to directly impact the number of false positive and false negative frames. Importantly, the segmentation accuracy on the classifier-selected frames, that would be segmented, remains comparable to or better than those from standalone segmentation networks. Interestingly, the efficacy of the pre-screening classifier was affected by the sampling methods for training labels from multiple observers, a seemingly independent problem. We show experimentally that a previously proposed approach, combining random sampling and consensus labels, may need to be adapted to perform well in our application. Furthermore, this work aims to share practical experience in developing a machine learning application that assists highly variable interventional imaging for prostate cancer patients, to present robust and reproducible open-source implementations, and to report a set of comprehensive results and analysis comparing these practical, yet important, options in a real-world clinical application.
Aspect-based Sentiment Analysis in Document -- FOMC Meeting Minutes on Economic Projection
Authors: Sarah-Yifei-Wang
Date: 2021-08-09 00:00:00
Tasks: Aspect-Based Sentiment Analysis, Sentiment Analysis

The Federal Open Market Committee within the Federal Reserve System is responsible for managing inflation, maximizing employment, and stabilizing interest rates. Meeting minutes play an important role for market movements because they provide the birds eye view of how this economic complexity is constantly re-weighed. Therefore, There has been growing interest in analyzing and extracting sentiments on various aspects from large financial texts for economic projection. However, Aspect-based Sentiment Analysis is not widely used on financial data due to the lack of large labeled dataset. In this paper, I propose a model to train ABSA on financial documents under weak supervision and analyze its predictive power on various macroeconomic indicators.
Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness
Authors: Yuwei Sun, Hideya Ochiai, Hiroshi Esaki
Date: 2021-07-30 00:00:00
Tasks: Distributed Computing, Edge-computing, Federated Learning, Privacy Preserving

Wider coverage and a better solution to a latency reduction in 5G necessitate its combination with multi-access edge computing (MEC) technology. Decentralized deep learning (DDL) such as federated learning and swarm learning as a promising solution to privacy-preserving data processing for millions of smart edge devices, leverages distributed computing of multi-layer neural networks within the networking of local clients, whereas, without disclosing the original local training data. Notably, in industries such as finance and healthcare where sensitive data of transactions and personal medical records is cautiously maintained, DDL can facilitate the collaboration among these institutes to improve the performance of trained models while protecting the data privacy of participating clients. In this survey paper, we demonstrate the technical fundamentals of DDL that benefit many walks of society through decentralized learning. Furthermore, we offer a comprehensive overview of the current state-of-the-art in the field by outlining the challenges of DDL and the most relevant solutions from novel perspectives of communication efficiency and trustworthiness.
A new step for computing
Authors: Xavier Vasques
Date: 2021-07-22 00:00:00

The data center of tomorrow is a data center made up of heterogeneous systems, which will run heterogeneous workloads. The systems will be located as close as possible to the data. Heterogeneous systems will be equipped with binary, biological inspired and quantum accelerators. These architectures will be the foundations to address challenges. Like an orchestra conductor, the hybrid cloud will make it possible to set these systems to music thanks to a layer of security and intelligent automation.
Cayley Graphs of Semigroups Applied to Atom Tracking in Chemistry
Authors: Nikolai Nøjgaard, Walter Fontana, Marc Hellmuth, Daniel Merkle
Date: 2021-08-09 00:00:00

While atom tracking with isotope-labeled compounds is an essential and sophisticated wet-lab tool in order to, e.g., illuminate reaction mechanisms, there exists only a limited amount of formal methods to approach the problem. Specifically when large (bio-)chemical networks are considered where reactions are stereo-specific, rigorous techniques are inevitable. We present an approach using the right Cayley graph of a monoid in order to track atoms concurrently through sequences of reactions and predict their potential location in product molecules. This can not only be used to systematically build hypothesis or reject reaction mechanisms (we will use the ANRORC mechanism "Addition of the Nucleophile, Ring Opening, and Ring Closure" as an example), but also to infer naturally occurring subsystems of (bio-)chemical systems. Our results include the analysis of the carbon traces within the TCA cycle and infer subsystems based on projections of the right Cayley graph onto a set of relevant atoms.
Adding Visibility to Visibility Graphs: Weighting Visibility Analysis with Attenuation Coefficients
Authors: Mathew Schwartz, Margarita Vinnikov, John Federici
Date: 2021-07-28 00:00:00

Evaluating the built environment based on visibility has been long used as a tool for human-centric design. The origins of isovists and visibility graphs are within interior spaces, while more recently, these evaluation techniques have been applied in the urban context. One of the key differentiators of an outside environment is the weather, which has largely been ignored in the design computation and space-syntax research areas. While a visibility graph is a straightforward metric for determining connectivity between regions of space through a line of sight calculation, this approach largely ignores the actual visibility of one point to another. This paper introduces a new method for weighting a visibility graph based on weather conditions (i.e. rain, fog, snow). These new factors are integrated into visibility graphs and applied to sample environments to demonstrate the variance between assuming a straight line of sight and reduced visibility.
Autonomous Vehicle Convoy Control as a Differential Game
Authors: Hossein B. Jond, Jan Platoš
Date: 2021-01-21 00:00:00
Tasks: Autonomous Vehicles

Group control of connected and autonomous vehicles on automated highways is challenging for the advanced driver assistance systems (ADAS) and the automated driving systems (ADS). This paper investigates the differential game-based approach to autonomous convoy control with the aim of deployment on automated highways. Under the non-cooperative differential games, the coupled vehicles make their decisions independently while their states are interdependent. The receding horizon Nash equilibrium of the linear-quadratic differential game provides the convoy a distributed state-feedback control strategy. This approach suffers a fundamental issue that neither a Nash equilibrium's existence nor the uniqueness is guaranteed. We convert the individual dynamics-based differential game to a relative dynamics-based optimal control problem that carries all the features of the differential game. The existence of a unique Nash control under the differential game corresponds to a unique solution to the optimal control problem. The latter is shown, as well as the asymptotic stability of the closed-loop system. Simulations illustrate the effectiveness of the presented convey control scheme and how it well suits automated highway driving scenarios.
Multiscale Wavelet Transfer Entropy with Application to Corticomuscular Coupling Analysis
Authors: Zhenghao Guo, Verity M. McClelland, Osvaldo Simeone, Kerry R. Mills, Zoran Cvetkovic
Date: 2021-08-09 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Objective: Functional coupling between the motor cortex and muscle activity is commonly detected and quantified by cortico-muscular coherence (CMC) or Granger causality (GC) analysis, which are applicable only to linear couplings and are not sufficiently sensitive: some healthy subjects show no significant CMC and GC, and yet have good motor skills. The objective of this work is to develop measures of functional cortico-muscular coupling that have improved sensitivity and are capable of detecting both linear and non-linear interactions. Methods: A multiscale wavelet transfer entropy (TE) methodology is proposed. The methodology relies on a dyadic stationary wavelet transform to decompose electroencephalogram (EEG) and electromyogram (EMG) signals into functional bands of neural oscillations. Then, it applies TE analysis based on a range of embedding delay vectors to detect and quantify intra- and cross-frequency band cortico-muscular coupling at different time scales. Results: Our experiments with neurophysiological signals substantiate the potential of the developed methodologies for detecting and quantifying information flow between EEG and EMG signals for subjects with and without significant CMC or GC, including non-linear cross-frequency interactions, and interactions across different temporal scales. The obtained results are in agreement with the underlying sensorimotor neurophysiology. Conclusion: These findings suggest that the concept of multiscale wavelet TE provides a comprehensive framework for analysing cortex-muscle interactions. Significance: The proposed methodologies will enable developing novel insights into movement control and neurophysiological processes more generally.
Mean-square Analysis of the NLMS Algorithm
Authors: Tareq Y. Al-Naffouri, Muhammad Moinuddin, Anum Ali
Date: 2021-08-08 00:00:00
Tasks: Form

This work presents a novel approach to the mean-square analysis of the normalized least mean squares (NLMS) algorithm for circular complex colored Gaussian inputs. The analysis is based on the derivation of a closed-form expression for the Cumulative Distribution Function (CDF) of random variables of the form $(||{\bf u}||_{{\bf D}_1}^2)(||{\bf u}||_{{\bf D}_2}^2)^{-1}$ where ${\bf u}$ is an isotropic vector and ${\bf D}_1$ and ${\bf D}_2$ are diagonal matrices and using that to derive some moments of these variables. These moments in turn completely characterize the mean-square behavior of the NLMS algorithm in explicit closed-form expressions. Specifically, the transient, steady-state, and tracking mean-square behavior of the NLMS algorithm is studied.
Event-Triggered Distributed Estimation With Decaying Communication Rate
Authors: Xingkang He, Yu Xing, Junfeng Wu, Karl H. Johansson
Date: 2021-03-10 00:00:00

We study distributed estimation of a high-dimensional static parameter vector through a group of sensors whose communication network is modeled by a fixed directed graph. Different from existing time-triggered communication schemes, an event-triggered asynchronous scheme is investigated in order to reduce communication while preserving estimation convergence. A distributed estimation algorithm with a single step size is first proposed based on an event-triggered communication scheme with a time-dependent decaying threshold. With the event-triggered scheme, each sensor sends its estimate to neighbor sensors only when the difference between the current estimate and the last sent-out estimate is larger than the triggering threshold. We prove that the proposed algorithm has mean-square and almost-sure convergence respectively, under an integrated condition of sensor network topology and sensor measurement matrices. The condition is satisfied if the topology is a balanced digraph containing a spanning tree and the system is collectively observable. Moreover, we provide estimates for the convergence rates, which are related to the step size as well as the triggering threshold. Furthermore, as an essential metric of sensor communication intensity in the event-triggered distributed algorithms, the communication rate is proved to decay to zero with a certain speed almost surely as time goes to infinity. We show that given the step size, adjusting the decay speed of the triggering threshold can lead to a tradeoff between the convergence rate of the estimation error and the decay speed of the communication rate. Specifically, increasing the decay speed of the threshold would make the communication rate decay faster, but reduce the convergence rate of the estimation error. Numerical simulations are provided to illustrate the developed results.
Impact of Acceleration/deceleration Limits on the String Stability of Adaptive Cruise Control
Authors: Hao Zhou, Anye Zhou, Tienan Li, Danjue Chen, Srinivas Peeta, Jorge Laval
Date: 2021-08-09 00:00:00

This paper demonstrates that the acceleration/deceleration limits in ACC systems can make a string stable ACC amplify the speed perturbation in natural driving. It is shown that the constrained acceleration/deceleration of the following ACCs are likely to cause speed overshoot to compensate for an extra large/small spacing. Additionally, we find that the constrained deceleration limits can also jeopardize safety, as the limited braking power produces extra small spacing or even crashes. The findings are validated through experiments on real cars. The paper suggests that the ACC parameter space should be extended to include the acceleration/deceleration limits considering their significant role exposed here. Through numerical simulations of ACC platoons, we show i) a marginal string stable ACC is preferable due to the smaller total queue length and the shorter duration in congestion; ii) congestion waves in a mixed ACC platoon largely depend on the sequence of vehicles provided different acceleration/deceleration limits, and iii) the safety hazard caused by the constrained deceleration limits is more significant in mixed ACC platoons when string unstable ACCs exist.