pwc-archive/papers-with-abstracts huggingface dataset:
Lane Attention: Predicting Vehicles' Moving Trajectories by Learning Their Attention over Lanes
Authors: Jiacheng Pan, Hongyi Sun, Kecheng Xu, Yifei Jiang, Xiangquan Xiao, Jiangtao Hu, Jinghao Miao
Date: 2019-09-29 00:00:00
Tasks: Autonomous Driving

Accurately forecasting the future movements of surrounding vehicles is essential for safe and efficient operations of autonomous driving cars. This task is difficult because a vehicle's moving trajectory is greatly determined by its driver's intention, which is often hard to estimate. By leveraging attention mechanisms along with long short-term memory (LSTM) networks, this work learns the relation between a driver's intention and the vehicle's changing positions relative to road infrastructures, and uses it to guide the prediction. Different from other state-of-the-art solutions, our work treats the on-road lanes as non-Euclidean structures, unfolds the vehicle's moving history to form a spatio-temporal graph, and uses methods from Graph Neural Networks to solve the problem. Not only is our approach a pioneering attempt in using non-Euclidean methods to process static environmental features around a predicted object, our model also outperforms other state-of-the-art models in several metrics. The practicability and interpretability analysis of the model shows great potential for large-scale deployment in various autonomous driving systems in addition to our own.
A Simple and Effective Model for Answering Multi-span Questions
Authors: Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, Jonathan Berant
Date: 2019-09-29 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Question Answering, Reading Comprehension

Models for reading comprehension (RC) commonly restrict their output space to the set of all single contiguous spans from the input, in order to alleviate the learning problem and avoid the need for a model that generates text explicitly. However, forcing an answer to be a single span can be restrictive, and some recent datasets also include multi-span questions, i.e., questions whose answer is a set of non-contiguous spans in the text. Naturally, models that return single spans cannot answer these questions. In this work, we propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem, namely, predicting for each input token whether it should be part of the output or not. Our model substantially improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively.
Deep k-NN Defense against Clean-label Data Poisoning Attacks
Authors: Neehar Peri, Neal Gupta, W. Ronny Huang, Liam Fowl, Chen Zhu, Soheil Feizi, Tom Goldstein, John P. Dickerson
Date: 2019-09-29 00:00:00
Tasks: Adversarial Attack, Data Poisoning

Targeted clean-label data poisoning is a type of adversarial attack on machine learning systems in which an adversary injects a few correctly-labeled, minimally-perturbed samples into the training data, causing a model to misclassify a particular test sample during inference. Although defenses have been proposed for general poisoning attacks, no reliable defense for clean-label attacks has been demonstrated, despite the attacks' effectiveness and realistic applications. In this work, we propose a simple, yet highly-effective Deep k-NN defense against both feature collision and convex polytope clean-label attacks on the CIFAR-10 dataset. We demonstrate that our proposed strategy is able to detect over 99% of poisoned examples in both attacks and remove them without compromising model performance. Additionally, through ablation studies, we discover simple guidelines for selecting the value of k as well as for implementing the Deep k-NN defense on real-world datasets with class imbalance. Our proposed defense shows that current clean-label poisoning attack strategies can be annulled, and serves as a strong yet simple-to-implement baseline defense to test future clean-label poisoning attacks. Our code is available at https://github.com/neeharperi/DeepKNNDefense
Gradient Descent: The Ultimate Optimizer
Authors: Kartik Chandra, Audrey Xie, Jonathan Ragan-Kelley, Erik Meijer
Date: 2019-09-29 00:00:00
Tasks: BIG-bench Machine Learning, Hyperparameter Optimization

Working with any gradient-based machine learning algorithm involves the tedious task of tuning the optimizer's hyperparameters, such as its step size. Recent work has shown how the step size can itself be optimized alongside the model parameters by manually deriving expressions for "hypergradients" ahead of time. We show how to automatically compute hypergradients with a simple and elegant modification to backpropagation. This allows us to easily apply the method to other optimizers and hyperparameters (e.g. momentum coefficients). We can even recursively apply the method to its own hyper-hyperparameters, and so on ad infinitum. As these towers of optimizers grow taller, they become less sensitive to the initial choice of hyperparameters. We present experiments validating this for MLPs, CNNs, and RNNs. Finally, we provide a simple PyTorch implementation of this algorithm (see people.csail.mit.edu/kach/gradient-descent-the-ultimate-optimizer).
Language-Agnostic Syllabification with Neural Sequence Labeling
Authors: Jacob Krantz, Maxwell Dulin, Paul De Palma
Date: 2019-09-29 00:00:00
Tasks: Chunking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, NER, Part-Of-Speech Tagging, POS, POS Tagging, speech-recognition, Speech Recognition

The identification of syllables within phonetic sequences is known as syllabification. This task is thought to play an important role in natural language understanding, speech production, and the development of speech recognition systems. The concept of the syllable is cross-linguistic, though formal definitions are rarely agreed upon, even within a language. In response, data-driven syllabification methods have been developed to learn from syllabified examples. These methods often employ classical machine learning sequence labeling models. In recent years, recurrence-based neural networks have been shown to perform increasingly well for sequence labeling tasks such as named entity recognition (NER), part of speech (POS) tagging, and chunking. We present a novel approach to the syllabification problem which leverages modern neural network techniques. Our network is constructed with long short-term memory (LSTM) cells, a convolutional component, and a conditional random field (CRF) output layer. Existing syllabification approaches are rarely evaluated across multiple language families. To demonstrate cross-linguistic generalizability, we show that the network is competitive with state of the art systems in syllabifying English, Dutch, Italian, French, Manipuri, and Basque datasets.
A Longitudinal Framework for Predicting Nonresponse in Panel Surveys
Authors: Christoph Kern, Bernd Weiss, Jan-Philipp Kolb
Date: 2019-09-29 00:00:00
Tasks: BIG-bench Machine Learning

Nonresponse in panel studies can lead to a substantial loss in data quality due to its potential to introduce bias and distort survey estimates. Recent work investigates the usage of machine learning to predict nonresponse in advance, such that predicted nonresponse propensities can be used to inform the data collection process. However, predicting nonresponse in panel studies requires accounting for the longitudinal data structure in terms of model building, tuning, and evaluation. This study proposes a longitudinal framework for predicting nonresponse with machine learning and multiple panel waves and illustrates its application. With respect to model building, this approach utilizes information from multiple waves by introducing features that aggregate previous (non)response patterns. Concerning model tuning and evaluation, temporal cross-validation is employed by iterating through pairs of panel waves such that the training and test sets move in time. Implementing this approach with data from a German probability-based mixed-mode panel shows that aggregating information over multiple panel waves can be used to build prediction models with competitive and robust performance over all test waves.
Library network, a possible path to explainable neural networks
Authors: Jung Hoon Lee
Date: 2019-09-29 00:00:00
Tasks: Decision Making

Deep neural networks (DNNs) may outperform human brains in complex tasks, but the lack of transparency in their decision-making processes makes us question whether we could fully trust DNNs with high stakes problems. As DNNs' operations rely on a massive number of both parallel and sequential linear/nonlinear computations, predicting their mistakes is nearly impossible. Also, a line of studies suggests that DNNs can be easily deceived by adversarial attacks, indicating that their decisions can easily be corrupted by unexpected factors. Such vulnerability must be overcome if we intend to take advantage of DNNs' efficiency in high stakes problems. Here, we propose an algorithm that can help us better understand DNNs' decision-making processes. Our empirical evaluations suggest that this algorithm can effectively trace DNNs' decision processes from one layer to another and detect adversarial attacks.
End-to-End Deep Convolutional Active Contours for Image Segmentation
Authors: Ali Hatamizadeh, Debleena Sengupta, Demetri Terzopoulos
Date: 2019-09-29 00:00:00
Tasks: Image Segmentation, Instance Segmentation, Segmentation, Semantic Segmentation

The Active Contour Model (ACM) is a standard image analysis technique whose numerous variants have attracted an enormous amount of research attention across multiple fields. Incorrectly, however, the ACM's differential-equation-based formulation and prototypical dependence on user initialization have been regarded as being largely incompatible with the recently popular deep learning approaches to image segmentation. This paper introduces the first tight unification of these two paradigms. In particular, we devise Deep Convolutional Active Contours (DCAC), a truly end-to-end trainable image segmentation framework comprising a Convolutional Neural Network (CNN) and an ACM with learnable parameters. The ACM's Eulerian energy functional includes per-pixel parameter maps predicted by the backbone CNN, which also initializes the ACM. Importantly, both the CNN and ACM components are fully implemented in TensorFlow, and the entire DCAC architecture is end-to-end automatically differentiable and backpropagation trainable without user intervention. As a challenging test case, we tackle the problem of building instance segmentation in aerial images and evaluate DCAC on two publicly available datasets, Vaihingen and Bing Huts. Our reseults demonstrate that, for building segmentation, the DCAC establishes a new state-of-the-art performance by a wide margin.
Siamese Neural Networks for Wireless Positioning and Channel Charting
Authors: Eric Lei, Oscar Castañeda, Olav Tirkkonen, Tom Goldstein, Christoph Studer
Date: 2019-09-29 00:00:00
Tasks: Dimensionality Reduction

Neural networks have been proposed recently for positioning and channel charting of user equipments (UEs) in wireless systems. Both of these approaches process channel state information (CSI) that is acquired at a multi-antenna base-station in order to learn a function that maps CSI to location information. CSI-based positioning using deep neural networks requires a dataset that contains both CSI and associated location information. Channel charting (CC) only requires CSI information to extract relative position information. Since CC builds on dimensionality reduction, it can be implemented using autoencoders. In this paper, we propose a unified architecture based on Siamese networks that can be used for supervised UE positioning and unsupervised channel charting. In addition, our framework enables semisupervised positioning, where only a small set of location information is available during training. We use simulations to demonstrate that Siamese networks achieve similar or better performance than existing positioning and CC approaches with a single, unified neural network architecture.
GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm
Authors: Parsa Esfahanian, Mohammad Akhavan
Date: 2019-09-29 00:00:00

Convolutional Neural Networks (CNNs) have gained a significant attraction in the recent years due to their increasing real-world applications. Their performance is highly dependent to the network structure and the selected optimization method for tuning the network parameters. In this paper, we propose novel yet efficient methods for training convolutional neural networks. The most of current state of the art learning method for CNNs are based on Gradient decent. In contrary to the traditional CNN training methods, we propose to optimize the CNNs using methods based on Genetic Algorithms (GAs). These methods are carried out using three individual GA schemes, Steady-State, Generational, and Elitism. We present new genetic operators for crossover, mutation and also an innovative encoding paradigm of CNNs to chromosomes aiming to reduce the resulting chromosome's size by a large factor. We compare the effectiveness and scalability of our encoding with the traditional encoding. Furthermore, the performance of individual GA schemes used for training the networks were compared with each other in means of convergence rate and overall accuracy. Finally, our new encoding alongside the superior GA-based training scheme is compared to Backpropagation training with Adam optimization.
ISTHMUS: Secure, Scalable, Real-time and Robust Machine Learning Platform for Healthcare
Authors: Akshay Arora, Arun Nethi, Priyanka Kharat, Vency Verghese, Grant Jenkins, Steve Miff, Vikas Chowdhry, Xiao Wang
Date: 2019-09-29 00:00:00
Tasks: BIG-bench Machine Learning

In recent times, machine learning (ML) and artificial intelligence (AI) based systems have evolved and scaled across different industries such as finance, retail, insurance, energy utilities, etc. Among other things, they have been used to predict patterns of customer behavior, to generate pricing models, and to predict the return on investments. But the successes in deploying machine learning models at scale in those industries have not translated into the healthcare setting. There are multiple reasons why integrating ML models into healthcare has not been widely successful, but from a technical perspective, general-purpose commercial machine learning platforms are not a good fit for healthcare due to complexities in handling data quality issues, mandates to demonstrate clinical relevance, and a lack of ability to monitor performance in a highly regulated environment with stringent security and privacy needs. In this paper, we describe Isthmus, a turnkey, cloud-based platform which addresses the challenges above and reduces time to market for operationalizing ML/AI in healthcare. Towards the end, we describe three case studies which shed light on Isthmus capabilities. These include (1) supporting an end-to-end lifecycle of a model which predicts trauma survivability at hospital trauma centers, (2) bringing in and harmonizing data from disparate sources to create a community data platform for inferring population as well as patient level insights for Social Determinants of Health (SDoH), and (3) ingesting live-streaming data from various IoT sensors to build models, which can leverage real-time and longitudinal information to make advanced time-sensitive predictions.
Learning Everywhere: A Taxonomy for the Integration of Machine Learning and Simulations
Authors: Geoffrey Fox, Shantenu Jha
Date: 2019-09-29 00:00:00
Tasks: BIG-bench Machine Learning

We present a taxonomy of research on Machine Learning (ML) applied to enhance simulations together with a catalog of some activities. We cover eight patterns for the link of ML to the simulations or systems plus three algorithmic areas: particle dynamics, agent-based models and partial differential equations. The patterns are further divided into three action areas: Improving simulation with Configurations and Integration of Data, Learn Structure, Theory and Model for Simulation, and Learn to make Surrogates.
MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
Authors: Badr-Eddine Chérief-Abdellatif, Pierre Alquier
Date: 2019-09-29 00:00:00
Proceeding: pproximateinference AABI Symposium 2019 12

In some misspecified settings, the posterior distribution in Bayesian statistics may lead to inconsistent estimates. To fix this issue, it has been suggested to replace the likelihood by a pseudo-likelihood, that is the exponential of a loss function enjoying suitable robustness properties. In this paper, we build a pseudo-likelihood based on the Maximum Mean Discrepancy, defined via an embedding of probability distributions into a reproducing kernel Hilbert space. We show that this MMD-Bayes posterior is consistent and robust to model misspecification. As the posterior obtained in this way might be intractable, we also prove that reasonable variational approximations of this posterior enjoy the same properties. We provide details on a stochastic gradient algorithm to compute these variational approximations. Numerical simulations indeed suggest that our estimator is more robust to misspecification than the ones based on the likelihood.
Symplectic Recurrent Neural Networks
Authors: Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, Léon Bottou
Date: 2019-09-29 00:00:00
Proceeding: ICLR 2020 1

We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories. An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiple-step training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems. We show that SRNNs succeed reliably on complex and noisy Hamiltonian systems. We also show how to augment the SRNN integration scheme in order to handle stiff dynamical systems such as bouncing billiards.
Recent Advances in End-to-End Spoken Language Understanding
Authors: Natalia Tomashenko, Antoine Caubriere, Yannick Esteve, Antoine Laurent, Emmanuel Morin
Date: 2019-09-29 00:00:00
Tasks: General Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, slot-filling, Slot Filling, Spoken Language Understanding

This work investigates spoken language understanding (SLU) systems in the scenario when the semantic information is extracted directly from the speech signal by means of a single end-to-end neural network model. Two SLU tasks are considered: named entity recognition (NER) and semantic slot filling (SF). For these tasks, in order to improve the model performance, we explore various techniques including speaker adaptation, a modification of the connectionist temporal classification (CTC) training criterion, and sequential pretraining.
Neural Hybrid Recommender: Recommendation needs collaboration
Authors: Ezgi Yildirim, Payam Azad, Şule Gündüz Öğüdücü
Date: 2019-09-29 00:00:00
Tasks: Recommendation Systems, speech-recognition, Speech Recognition

In recent years, deep learning has gained an indisputable success in computer vision, speech recognition, and natural language processing. After its rising success on these challenging areas, it has been studied on recommender systems as well, but mostly to include content features into traditional methods. In this paper, we introduce a generalized neural network-based recommender framework that is easily extendable by additional networks. This framework named NHR, short for Neural Hybrid Recommender allows us to include more elaborate information from the same and different data sources. We have worked on item prediction problems, but the framework can be used for rating prediction problems as well with a single change on the loss function. To evaluate the effect of such a framework, we have tested our approach on benchmark and not yet experimented datasets. The results in these real-world datasets show the superior performance of our approach in comparison with the state-of-the-art methods.
Natural representation of composite data with replicated autoencoders
Authors: Matteo Negri, Davide Bergamini, Carlo Baldassi, Riccardo Zecchina, Christoph Feinauer
Date: 2019-09-29 00:00:00

Generative processes in biology and other fields often produce data that can be regarded as resulting from a composition of basic features. Here we present an unsupervised method based on autoencoders for inferring these basic features of data. The main novelty in our approach is that the training is based on the optimization of the `local entropy' rather than the standard loss, resulting in a more robust inference, and enhancing the performance on this type of data considerably. Algorithmically, this is realized by training an interacting system of replicated autoencoders. We apply this method to synthetic and protein sequence data, and show that it is able to infer a hidden representation that correlates well with the underlying generative process, without requiring any prior knowledge.
Capacity Preserving Mapping for High-dimensional Data Visualization
Authors: Rongrong Wang, Xiaopeng Zhang
Date: 2019-09-29 00:00:00
Tasks: Data Visualization, Dimensionality Reduction, Vocal Bursts Intensity Prediction

We provide a rigorous mathematical treatment to the crowding issue in data visualization when high dimensional data sets are projected down to low dimensions for visualization. By properly adjusting the capacity of high dimensional balls, our method makes right enough room to prepare for the embedding. A key component of the proposed method is an estimation of the correlation dimension at various scales which reflects the data density variation. The proposed adjustment to the capacity applies to any distance (Euclidean, geodesic, diffusion) and can potentially be used in many existing methods to mitigate the crowding during the dimension reduction. We demonstrate the effectiveness of the new method using synthetic and real datasets.
Machine Learning vs Statistical Methods for Time Series Forecasting: Size Matters
Authors: Vitor Cerqueira, Luis Torgo, Carlos Soares
Date: 2019-09-29 00:00:00
Tasks: BIG-bench Machine Learning, Time Series, Time Series Analysis, Time Series Forecasting, valid

Time series forecasting is one of the most active research topics. Machine learning methods have been increasingly adopted to solve these predictive tasks. However, in a recent work, these were shown to systematically present a lower predictive performance relative to simple statistical methods. In this work, we counter these results. We show that these are only valid under an extremely low sample size. Using a learning curve method, our results suggest that machine learning methods improve their relative predictive performance as the sample size grows. The code to reproduce the experiments is available at https://github.com/vcerqueira/MLforForecasting.