pwc-archive/papers-with-abstracts huggingface dataset:
A Tensor Compiler for Unified Machine Learning Prediction Serving
Authors: Supun Nakandala, Karla Saur, Gyeong-In Yu, Konstantinos Karanasos, Carlo Curino, Markus Weimer, Matteo Interlandi
Date: 2020-10-09 00:00:00
Tasks: BIG-bench Machine Learning, CPU, GPU

Machine Learning (ML) adoption in the enterprise requires simpler and more efficient software infrastructure---the bespoke solutions typical in large web companies are simply untenable. Model scoring, the process of obtaining predictions from a trained model over new data, is a primary contributor to infrastructure complexity and cost as models are trained once but used many times. In this paper we propose HUMMINGBIRD, a novel approach to model scoring, which compiles featurization operators and traditional ML models (e.g., decision trees) into a small set of tensor operations. This approach inherently reduces infrastructure complexity and directly leverages existing investments in Neural Network compilers and runtimes to generate efficient computations for both CPU and hardware accelerators. Our performance results are intriguing: despite replacing imperative computations (e.g., tree traversals) with tensor computation abstractions, HUMMINGBIRD is competitive and often outperforms hand-crafted kernels on micro-benchmarks on both CPU and GPU, while enabling seamless end-to-end acceleration of ML pipelines. We have released HUMMINGBIRD as open source.
Cluster Activation Mapping with Applications to Medical Imaging
Authors: Sarah Ryan, Nichole Carlson, Harris Butler, Tasha Fingerlin, Lisa Maier, Fuyong Xing
Date: 2020-10-09 00:00:00
Tasks: Clustering, Decision Making, Deep Clustering, Open-Ended Question Answering

An open question in deep clustering is how to understand what in the image is creating the cluster assignments. This visual understanding is essential to be able to trust the results of an inherently complex algorithm like deep learning, especially when the derived cluster assignments may be used to inform decision-making or create new disease sub-types. In this work, we developed novel methodology to generate CLuster Activation Mapping (CLAM) which combines an unsupervised deep clustering framework with a modification of Score-CAM, an approach for discriminative localization in the supervised setting. We evaluated our approach using a simulation study based on computed tomography scans of the lung, and applied it to 3D CT scans from a sarcoidosis population to identify new clusters of sarcoidosis based purely on CT scan presentation.
ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization
Authors: Shiyue Zhang, Benjamin Frey, Mohit Bansal
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Cultural Vocal Bursts Intensity Prediction, Language Modelling, Machine Translation, NMT, Sentence, Transfer Learning, Translation

Cherokee is a highly endangered Native American language spoken by the Cherokee people. The Cherokee culture is deeply embedded in its language. However, there are approximately only 2,000 fluent first language Cherokee speakers remaining in the world, and the number is declining every year. To help save this endangered language, we introduce ChrEn, a Cherokee-English parallel dataset, to facilitate machine translation research between Cherokee and English. Compared to some popular machine translation language pairs, ChrEn is extremely low-resource, only containing 14k sentence pairs in total. We split our parallel data in ways that facilitate both in-domain and out-of-domain evaluation. We also collect 5k Cherokee monolingual data to enable semi-supervised learning. Besides these datasets, we propose several Cherokee-English and English-Cherokee machine translation systems. We compare SMT (phrase-based) versus NMT (RNN-based and Transformer-based) systems; supervised versus semi-supervised (via language model, back-translation, and BERT/Multilingual-BERT) methods; as well as transfer learning versus multilingual joint training with 4 other languages. Our best results are 15.8/12.7 BLEU for in-domain and 6.5/5.0 BLEU for out-of-domain Chr-En/EnChr translations, respectively, and we hope that our dataset and systems will encourage future work by the community for Cherokee language revitalization. Our data, code, and demo will be publicly available at https://github.com/ZhangShiyue/ChrEn
Reparametrizing gradient descent
Authors: David Sprunger
Date: 2020-10-09 00:00:00

In this work, we propose an optimization algorithm which we call norm-adapted gradient descent. This algorithm is similar to other gradient-based optimization algorithms like Adam or Adagrad in that it adapts the learning rate of stochastic gradient descent at each iteration. However, rather than using statistical properties of observed gradients, norm-adapted gradient descent relies on a first-order estimate of the effect of a standard gradient descent update step, much like the Newton-Raphson method in many dimensions. Our algorithm can also be compared to quasi-Newton methods, but we seek roots rather than stationary points. Seeking roots can be justified by the fact that for models with sufficient capacity measured by nonnegative loss functions, roots coincide with global optima. This work presents several experiments where we have used our algorithm; in these results, it appears norm-adapted descent is particularly strong in regression settings but is also capable of training classifiers.
Learning Languages in the Limit from Positive Information with Finitely Many Memory Changes
Authors: Timo Kötzing, Karen Seidel
Date: 2020-10-09 00:00:00
Tasks: LEMMA

We investigate learning collections of languages from texts by an inductive inference machine with access to the current datum and a bounded memory in form of states. Such a bounded memory states (BMS) learner is considered successful in case it eventually settles on a correct hypothesis while exploiting only finitely many different states. We give the complete map of all pairwise relations for an established collection of criteria of successfull learning. Most prominently, we show that non-U-shapedness is not restrictive, while conservativeness and (strong) monotonicity are. Some results carry over from iterative learning by a general lemma showing that, for a wealth of restrictions (the semantic restrictions), iterative and bounded memory states learning are equivalent. We also give an example of a non-semantic restriction (strongly non-U-shapedness) where the two settings differ.
A Graph Neural Network Approach for Scalable and Dynamic IP Similarity in Enterprise Networks
Authors: Hazem M. Soliman, Geoff Salmon, Dusan Sovilij, Mohan Rao
Date: 2020-10-09 00:00:00
Tasks: Graph Neural Network

Measuring similarity between IP addresses is an important task in the daily operations of any enterprise network. Applications that depend on an IP similarity measure include measuring correlation between security alerts, building baselines for behavioral modelling, debugging network failures and tracking persistent attacks. However, IPs do not have a natural similarity measure by definition. Deep Learning architectures are a promising solution here since they are able to learn numerical representations for IPs directly from data, allowing various distance measures to be applied on the calculated representations. Current works have utilized Natural Language Processing (NLP) techniques for learning IP embeddings. However, these approaches have no proper way to handle out-of-vocabulary (OOV) IPs not seen during training. In this paper, we propose a novel approach for IP embedding using an adapted graph neural network (GNN) architecture. This approach has the advantages of working on the raw data, scalability and, most importantly, induction, i.e. the ability to measure similarity between previously unseen IPs. Using data from an enterprise network, our approach is able to identify similarities between local DNS servers and root DNS servers even though some of these machines are never encountered during the training phase.
Thermal-Aware Compilation of Spiking Neural Networks to Neuromorphic Hardware
Authors: Twisha Titirsha, Anup Das
Date: 2020-10-09 00:00:00
Tasks: BIG-bench Machine Learning

Hardware implementation of neuromorphic computing can significantly improve performance and energy efficiency of machine learning tasks implemented with spiking neural networks (SNNs), making these hardware platforms particularly suitable for embedded systems and other energy-constrained environments. We observe that the long bitlines and wordlines in a crossbar of the hardware create significant current variations when propagating spikes through its synaptic elements, which are typically designed with non-volatile memory (NVM). Such current variations create a thermal gradient within each crossbar of the hardware, depending on the machine learning workload and the mapping of neurons and synapses of the workload to these crossbars. \mr{This thermal gradient becomes significant at scaled technology nodes and it increases the leakage power in the hardware leading to an increase in the energy consumption.} We propose a novel technique to map neurons and synapses of SNN-based machine learning workloads to neuromorphic hardware. We make two novel contributions. First, we formulate a detailed thermal model for a crossbar in a neuromorphic hardware incorporating workload dependency, where the temperature of each NVM-based synaptic cell is computed considering the thermal contributions from its neighboring cells. Second, we incorporate this thermal model in the mapping of neurons and synapses of SNN-based workloads using a hill-climbing heuristic. The objective is to reduce the thermal gradient in crossbars. We evaluate our neuron and synapse mapping technique using 10 machine learning workloads for a state-of-the-art neuromorphic hardware. We demonstrate an average 11.4K reduction in the average temperature of each crossbar in the hardware, leading to a 52% reduction in the leakage power consumption (11% lower total energy consumption) compared to a performance-oriented SNN mapping technique.
Robust Behavioral Cloning for Autonomous Vehicles using End-to-End Imitation Learning
Authors: Tanmay Vilas Samak, Chinmay Vilas Samak, Sivanathan Kandhasamy
Date: 2020-10-09 00:00:00
Tasks: Autonomous Vehicles, Computational Efficiency, Imitation Learning

In this work, we present a lightweight pipeline for robust behavioral cloning of a human driver using end-to-end imitation learning. The proposed pipeline was employed to train and deploy three distinct driving behavior models onto a simulated vehicle. The training phase comprised of data collection, balancing, augmentation, preprocessing and training a neural network, following which, the trained model was deployed onto the ego vehicle to predict steering commands based on the feed from an onboard camera. A novel coupled control law was formulated to generate longitudinal control commands on-the-go based on the predicted steering angle and other parameters such as actual speed of the ego vehicle and the prescribed constraints for speed and steering. We analyzed computational efficiency of the pipeline and evaluated robustness of the trained models through exhaustive experimentation during the deployment phase. We also compared our approach against state-of-the-art implementation in order to comment on its validity.
Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data
Authors: William Huang, Haokun Liu, Samuel R. Bowman
Date: 2020-10-09 00:00:00
Proceeding: EMNLP (insights) 2020 11
Tasks: counterfactual, Natural Language Inference, Natural Language Understanding, Reading Comprehension

A growing body of work shows that models exploit annotation artifacts to achieve state-of-the-art performance on standard crowdsourced benchmarks---datasets collected from crowdworkers to create an evaluation task---while still failing on out-of-domain examples for the same task. Recent work has explored the use of counterfactually-augmented data---data built by minimally editing a set of seed examples to yield counterfactual labels---to augment training data associated with these benchmarks and build more robust classifiers that generalize better. However, Khashabi et al. (2020) find that this type of augmentation yields little benefit on reading comprehension tasks when controlling for dataset size and cost of collection. We build upon this work by using English natural language inference data to test model generalization and robustness and find that models trained on a counterfactually-augmented SNLI dataset do not generalize better than unaugmented datasets of similar size and that counterfactual augmentation can hurt performance, yielding models that are less robust to challenge examples. Counterfactual augmentation of natural language understanding data through standard crowdsourcing techniques does not appear to be an effective way of collecting training data and further innovation is required to make this general line of work viable.
Predictive Modeling of Anatomy with Genetic and Clinical Data
Authors: Adrian V. Dalca, Ramesh Sridharan, Mert R. Sabuncu, Polina Golland
Date: 2020-10-09 00:00:00
Tasks: Anatomy, regression

We present a semi-parametric generative model for predicting anatomy of a patient in subsequent scans following a single baseline image. Such predictive modeling promises to facilitate novel analyses in both voxel-level studies and longitudinal biomarker evaluation. We capture anatomical change through a combination of population-wide regression and a non-parametric model of the subject's health based on individual genetic and clinical indicators. In contrast to classical correlation and longitudinal analysis, we focus on predicting new observations from a single subject observation. We demonstrate prediction of follow-up anatomical scans in the ADNI cohort, and illustrate a novel analysis approach that compares a patient's scans to the predicted subject-specific healthy anatomical trajectory. The code is available at https://github.com/adalca/voxelorb.
Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model
Authors: Jun Yen Leung, Guy Emerson, Ryan Cotterell
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11

Across languages, multiple consecutive adjectives modifying a noun (e.g. "the big red dog") follow certain unmarked ordering rules. While explanatory accounts have been put forward, much of the work done in this area has relied primarily on the intuitive judgment of native speakers, rather than on corpus data. We present the first purely corpus-driven model of multi-lingual adjective ordering in the form of a latent-variable model that can accurately order adjectives across 24 different languages, even when the training and testing languages are different. We utilize this novel statistical model to provide strong converging evidence for the existence of universal, cross-linguistic, hierarchical adjective ordering tendencies.
MEEP: An Open-Source Platform for Human-Human Dialog Collection and End-to-End Agent Training
Authors: Arkady Arkhangorodsky, Amittai Axelrod, Christopher Chu, Scot Fang, Yiqi Huang, Ajay Nagesh, Xing Shi, Boliang Zhang, Kevin Knight
Date: 2020-10-09 00:00:00

We create a new task-oriented dialog platform (MEEP) where agents are given considerable freedom in terms of utterances and API calls, but are constrained to work within a push-button environment. We include facilities for collecting human-human dialog corpora, and for training automatic agents in an end-to-end fashion. We demonstrate MEEP with a dialog assistant that lets users specify trip destinations.
Solving Historical Dictionary Codes with a Neural Language Model
Authors: Christopher Chu, Raphael Valenti, Kevin Knight
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modeling, Language Modelling

We solve difficult word-based substitution codes by constructing a decoding lattice and searching that lattice with a neural language model. We apply our method to a set of enciphered letters exchanged between US Army General James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s, obtained from the US Library of Congress. We are able to decipher 75.1% of the cipher-word tokens correctly.
Learning to Pronounce Chinese Without a Pronunciation Dictionary
Authors: Christopher Chu, Scot Fang, Kevin Knight
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11

We demonstrate a program that learns to pronounce Chinese text in Mandarin, without a pronunciation dictionary. From non-parallel streams of Chinese characters and Chinese pinyin syllables, it establishes a many-to-many mapping between characters and pronunciations. Using unsupervised methods, the program effectively deciphers writing into speech. Its token-level character-to-syllable accuracy is 89%, which significantly exceeds the 22% accuracy of prior work.
Machine Learning approach to muon spectroscopy analysis
Authors: T. Tula, G. Möller, J. Quintanilla, S. R. Giblin, A. D. Hillier, E. E. McCabe, S. Ramos, D. S. Barker, S. Gibson
Date: 2020-10-09 00:00:00
Tasks: BIG-bench Machine Learning

In recent years, Artificial Intelligence techniques have proved to be very successful when applied to problems in physical sciences. Here we apply an unsupervised Machine Learning (ML) algorithm called Principal Component Analysis (PCA) as a tool to analyse the data from muon spectroscopy experiments. Specifically, we apply the ML technique to detect phase transitions in various materials. The measured quantity in muon spectroscopy is an asymmetry function, which may hold information about the distribution of the intrinsic magnetic field in combination with the dynamics of the sample. Sharp changes of shape of asymmetry functions - measured at different temperatures - might indicate a phase transition. Existing methods of processing the muon spectroscopy data are based on regression analysis, but choosing the right fitting function requires knowledge about the underlying physics of the probed material. Conversely, Principal Component Analysis focuses on small differences in the asymmetry curves and works without any prior assumptions about the studied samples. We discovered that the PCA method works well in detecting phase transitions in muon spectroscopy experiments and can serve as an alternative to current analysis, especially if the physics of the studied material are not entirely known. Additionally, we found out that our ML technique seems to work best with large numbers of measurements, regardless of whether the algorithm takes data only for a single material or whether the analysis is performed simultaneously for many materials with different physical properties.
Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learning
Authors: Navid Naderializadeh, Fan H. Hung, Sean Soleyman, Deepak Khosla
Date: 2020-10-09 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), SMAC, SMAC+, Starcraft, Starcraft II

We propose a novel framework for value function factorization in multi-agent deep reinforcement learning (MARL) using graph neural networks (GNNs). In particular, we consider the team of agents as the set of nodes of a complete directed graph, whose edge weights are governed by an attention mechanism. Building upon this underlying graph, we introduce a mixing GNN module, which is responsible for i) factorizing the team state-action value function into individual per-agent observation-action value functions, and ii) explicit credit assignment to each agent in terms of fractions of the global team reward. Our approach, which we call GraphMIX, follows the centralized training and decentralized execution paradigm, enabling the agents to make their decisions independently once training is completed. We show the superiority of GraphMIX as compared to the state-of-the-art on several scenarios in the StarCraft II multi-agent challenge (SMAC) benchmark. We further demonstrate how GraphMIX can be used in conjunction with a recent hierarchical MARL architecture to both improve the agents' performance and enable fine-tuning them on mismatched test scenarios with higher numbers of agents and/or actions.
Evaluating and Characterizing Human Rationales
Authors: Samuel Carton, Anirudh Rathore, Chenhao Tan
Date: 2020-10-09 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Open-Ended Question Answering

Two main approaches for evaluating the quality of machine-generated rationales are: 1) using human rationales as a gold standard; and 2) automated metrics based on how rationales affect model behavior. An open question, however, is how human rationales fare with these automatic metrics. Analyzing a variety of datasets and models, we find that human rationales do not necessarily perform well on these metrics. To unpack this finding, we propose improved metrics to account for model-dependent baseline performance. We then propose two methods to further characterize rationale quality, one based on model retraining and one on using "fidelity curves" to reveal properties such as irrelevance and redundancy. Our work leads to actionable suggestions for evaluating and characterizing rationales.
Video Quality Enhancement Using Deep Learning-Based Prediction Models for Quantized DCT Coefficients in MPEG I-frames
Authors: Antonio J G Busson, Paulo R C Mendes, Daniel de S Moraes, Álvaro M da Veiga, Álan L V Guedes, Sérgio Colcher
Date: 2020-10-09 00:00:00
Tasks: Decoder

Recent works have successfully applied some types of Convolutional Neural Networks (CNNs) to reduce the noticeable distortion resulting from the lossy JPEG/MPEG compression technique. Most of them are built upon the processing made on the spatial domain. In this work, we propose a MPEG video decoder that is purely based on the frequency-to-frequency domain: it reads the quantized DCT coefficients received from a low-quality I-frames bitstream and, using a deep learning-based model, predicts the missing coefficients in order to recompose the same frames with enhanced quality. In experiments with a video dataset, our best model was able to improve from frames with quantized DCT coefficients corresponding to a Quality Factor (QF) of 10 to enhanced quality frames with QF slightly near to 20.