pwc-archive/papers-with-abstracts huggingface dataset:
Preparation of ordered states in ultra-cold gases using Bayesian optimization
Authors: Rick Mukherjee, Frederic Sauvage, Harry Xie, Robert Löw, Florian Mintert
Date: 2020-01-10 00:00:00
Tasks: Bayesian Optimization

Ultra-cold atomic gases are unique in terms of the degree of controllability, both for internal and external degrees of freedom. This makes it possible to use them for the study of complex quantum many-body phenomena. However in many scenarios, the prerequisite condition of faithfully preparing a desired quantum state despite decoherence and system imperfections is not always adequately met. To path the way to a specific target state, we explore quantum optimal control framework based on Bayesian optimization. The probabilistic modeling and broad exploration aspects of Bayesian optimization is particularly suitable for quantum experiments where data acquisition can be expensive. Using numerical simulations for the superfluid to Mott-insulator transition for bosons in a lattice as well for the formation of Rydberg crystals as explicit examples, we demonstrate that Bayesian optimization is capable of finding better control solutions with regards to finite and noisy data compared to existing methods of optimal control.
Convolutional-Recurrent Neural Networks on Low-Power Wearable Platforms for Cardiac Arrhythmia Detection
Authors: Antonino Faraone, Ricard Delgado-Gonzalo
Date: 2020-01-08 00:00:00
Tasks: Arrhythmia Detection

Low-power sensing technologies, such as wearables, have emerged in the healthcare domain since they enable continuous and non-invasive monitoring of physiological signals. In order to endow such devices with clinical value, classical signal processing has encountered numerous challenges. However, data-driven methods, such as machine learning, offer attractive accuracies at the expense of being resource and memory demanding. In this paper, we focus on the inference of neural networks running in microcontrollers and low-power processors which wearable sensors and devices are generally equipped with. In particular, we adapted an existing convolutional-recurrent neural network, designed to detect and classify cardiac arrhythmias from a single-lead electrocardiogram, to the low-power embedded System-on-Chip nRF52 from Nordic Semiconductor with an ARM's Cortex-M4 processing core. We show our implementation in fixed-point precision, using the CMSIS-NN libraries, yields a drop of $F_1$ score from 0.8 to 0.784, from the original implementation, with a memory footprint of 195.6KB, and a throughput of 33.98MOps/s.
Biophysical models of cis-regulation as interpretable neural networks
Authors: Ammar Tareen, Justin B. Kinney
Date: 2019-12-30 00:00:00
Tasks: Deep Learning

The adoption of deep learning techniques in genomics has been hindered by the difficulty of mechanistically interpreting the models that these techniques produce. In recent years, a variety of post-hoc attribution methods have been proposed for addressing this neural network interpretability problem in the context of gene regulation. Here we describe a complementary way of approaching this problem. Our strategy is based on the observation that two large classes of biophysical models of cis-regulatory mechanisms can be expressed as deep neural networks in which nodes and weights have explicit physiochemical interpretations. We also demonstrate how such biophysical networks can be rapidly inferred, using modern deep learning frameworks, from the data produced by certain types of massively parallel reporter assays (MPRAs). These results suggest a scalable strategy for using MPRAs to systematically characterize the biophysical basis of gene regulation in a wide range of biological contexts. They also highlight gene regulation as a promising venue for the development of scientifically interpretable approaches to deep learning.
Simulating Lexical Semantic Change from Sense-Annotated Data
Authors: Dominik Schlechtweg, Sabine Schulte im Walde
Date: 2020-01-09 00:00:00
Tasks: Change Detection

We present a novel procedure to simulate lexical semantic change from synchronic sense-annotated data, and demonstrate its usefulness for assessing lexical semantic change detection models. The induced dataset represents a stronger correspondence to empirically observed lexical semantic change than previous synthetic datasets, because it exploits the intimate relationship between synchronic polysemy and diachronic change. We publish the data and provide the first large-scale evaluation gold standard for LSC detection models.
A Scalable Chatbot Platform Leveraging Online Community Posts: A Proof-of-Concept Study
Authors: Sihyeon Jo, Sangwon Im, SangWook Han, Seung Hee Yang, Hee-Eun Kim, Seong-Woo Kim
Date: 2020-01-10 00:00:00
Tasks: Chatbot

The development of natural language processing algorithms and the explosive growth of conversational data are encouraging researches on the human-computer conversation. Still, getting qualified conversational data on a large scale is difficult and expensive. In this paper, we verify the feasibility of constructing a data-driven chatbot with processed online community posts by using them as pseudo-conversational data. We argue that chatbots for various purposes can be built extensively through the pipeline exploiting the common structure of community posts. Our experiment demonstrates that chatbots created along the pipeline can yield the proper responses.
Learning to Multi-Task Learn for Better Neural Machine Translation
Authors: Poorya Zaremoodi, Gholamreza Haffari
Date: 2020-01-10 00:00:00
Tasks: Imitation Learning, Low Resource NMT, Machine Translation, Multi-Task Learning, NMT, Scheduling, Sentence, Translation

Scarcity of parallel sentence pairs is a major challenge for training high quality neural machine translation (NMT) models in bilingually low-resource scenarios, as NMT is data-hungry. Multi-task learning is an elegant approach to inject linguistic-related inductive biases into NMT, using auxiliary syntactic and semantic tasks, to improve generalisation. The challenge, however, is to devise effective training schedules, prescribing when to make use of the auxiliary tasks during the training process to fill the knowledge gaps of the main translation task, a setting referred to as biased-MTL. Current approaches for the training schedule are based on hand-engineering heuristics, whose effectiveness vary in different MTL settings. We propose a novel framework for learning the training schedule, ie learning to multi-task learn, for the MTL setting of interest. We formulate the training schedule as a Markov decision process which paves the way to employ policy learning methods to learn the scheduling policy. We effectively and efficiently learn the training schedule policy within the imitation learning framework using an oracle policy algorithm that dynamically sets the importance weights of auxiliary tasks based on their contributions to the generalisability of the main NMT task. Experiments on low-resource NMT settings show the resulting automatically learned training schedulers are competitive with the best heuristics, and lead to up to +1.1 BLEU score improvements.
Machine Learning Approaches for Amharic Parts-of-speech Tagging
Authors: Ibrahim Gashaw, H. L. Shashirekha
Date: 2020-01-10 00:00:00
Tasks: BIG-bench Machine Learning, Information Retrieval, Machine Translation, Part-Of-Speech Tagging, POS, POS Tagging, Question Answering, Retrieval, Translation, Word Sense Disambiguation

Part-of-speech (POS) tagging is considered as one of the basic but necessary tools which are required for many Natural Language Processing (NLP) applications such as word sense disambiguation, information retrieval, information processing, parsing, question answering, and machine translation. Performance of the current POS taggers in Amharic is not as good as that of the contemporary POS taggers available for English and other European languages. The aim of this work is to improve POS tagging performance for the Amharic language, which was never above 91%. Usage of morphological knowledge, an extension of the existing annotated data, feature extraction, parameter tuning by applying grid search and the tagging algorithms have been examined and obtained significant performance difference from the previous works. We have used three different datasets for POS experiments.
Co-evolution of language and agents in referential games
Authors: Gautier Dagan, Dieuwke Hupkes, Elia Bruni
Date: 2020-01-10 00:00:00
Proceeding: EACL 2021 2

Referential games offer a grounded learning environment for neural agents which accounts for the fact that language is functionally used to communicate. However, they do not take into account a second constraint considered to be fundamental for the shape of human language: that it must be learnable by new language learners. Cogswell et al. (2019) introduced cultural transmission within referential games through a changing population of agents to constrain the emerging language to be learnable. However, the resulting languages remain inherently biased by the agents' underlying capabilities. In this work, we introduce Language Transmission Engine to model both cultural and architectural evolution in a population of agents. As our core contribution, we empirically show that the optimal situation is to take into account also the learning biases of the language learners and thus let language and agents co-evolve. When we allow the agent population to evolve through architectural evolution, we achieve across the board improvements on all considered metrics and surpass the gains made with cultural transmission. These results stress the importance of studying the underlying agent architecture and pave the way to investigate the co-evolution of language and agent in language emergence studies.
Towards Minimal Supervision BERT-based Grammar Error Correction
Authors: Yiyuan Li, Antonios Anastasopoulos, Alan W. black
Date: 2020-01-10 00:00:00
Tasks: Grammatical Error Correction, Language Modeling, Language Modelling

Current grammatical error correction (GEC) models typically consider the task as sequence generation, which requires large amounts of annotated data and limit the applications in data-limited settings. We try to incorporate contextual information from pre-trained language model to leverage annotation and benefit multilingual scenarios. Results show strong potential of Bidirectional Encoder Representations from Transformers (BERT) in grammatical error correction task.
Linking Social Media Posts to News with Siamese Transformers
Authors: Jacob Danovitch
Date: 2020-01-10 00:00:00
Tasks: Articles, Retrieval

Many computational social science projects examine online discourse surrounding a specific trending topic. These works often involve the acquisition of large-scale corpora relevant to the event in question to analyze aspects of the response to the event. Keyword searches present a precision-recall trade-off and crowd-sourced annotations, while effective, are costly. This work aims to enable automatic and accurate ad-hoc retrieval of comments discussing a trending topic from a large corpus, using only a handful of seed news articles.
Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Models
Authors: Shuo Wang, Surya Nepal, Carsten Rudolph, Marthie Grobler, Shangyu Chen, Tianle Chen
Date: 2020-01-10 00:00:00
Tasks: Deep Learning, Electrocardiography (ECG), Electroencephalogram (EEG), Time Series, Time Series Analysis, Transfer Learning

Transfer learning provides an effective solution for feasibly and fast customize accurate \textit{Student} models, by transferring the learned knowledge of pre-trained \textit{Teacher} models over large datasets via fine-tuning. Many pre-trained Teacher models used in transfer learning are publicly available and maintained by public platforms, increasing their vulnerability to backdoor attacks. In this paper, we demonstrate a backdoor threat to transfer learning tasks on both image and time-series data leveraging the knowledge of publicly accessible Teacher models, aimed at defeating three commonly-adopted defenses: \textit{pruning-based}, \textit{retraining-based} and \textit{input pre-processing-based defenses}. Specifically, (A) ranking-based selection mechanism to speed up the backdoor trigger generation and perturbation process while defeating \textit{pruning-based} and/or \textit{retraining-based defenses}. (B) autoencoder-powered trigger generation is proposed to produce a robust trigger that can defeat the \textit{input pre-processing-based defense}, while guaranteeing that selected neuron(s) can be significantly activated. (C) defense-aware retraining to generate the manipulated model using reverse-engineered model inputs. We launch effective misclassification attacks on Student models over real-world images, brain Magnetic Resonance Imaging (MRI) data and Electrocardiography (ECG) learning systems. The experiments reveal that our enhanced attack can maintain the $98.4\%$ and $97.2\%$ classification accuracy as the genuine model on clean image and time series inputs respectively while improving $27.9\%-100\%$ and $27.1\%-56.1\%$ attack success rate on trojaned image and time series inputs respectively in the presence of pruning-based and/or retraining-based defenses.
Efficient Memory Management for Deep Neural Net Inference
Authors: Yury Pisarchyk, Juhyun Lee
Date: 2020-01-10 00:00:00
Tasks: Management

While deep neural net inference was considered a task for servers only, latest advances in technology allow the task of inference to be moved to mobile and embedded devices, desired for various reasons ranging from latency to privacy. These devices are not only limited by their compute power and battery, but also by their inferior physical memory and cache, and thus, an efficient memory manager becomes a crucial component for deep neural net inference at the edge. We explore various strategies to smartly share memory buffers among intermediate tensors in deep neural nets. Employing these can result in up to 11% smaller memory footprint than the state of the art.
A Deep Learning Approach Towards Prediction of Faults in Wind Turbines
Authors: Joyjit Chatterjee, Nina Dethlefs
Date: 2019-12-12 00:00:00

With the rising costs of conventional sources of energy, the world is moving towards sustainable energy sources including wind energy. Wind turbines consist of several electrical and mechanical components and experience an enormous amount of irregular loads, making their operational behaviour at times inconsistent. Operations and Maintenance (O&M) is a key factor in monitoring such inconsistent behaviour of the turbines in order to predict and prevent any incipient faults which may occur in the near future. Machine learning has been applied to the domain of wind energy over the last decade for analysing, diagnosing and predicting wind turbine faults. In particular, we follow the idea of modelling a turbine's performance as a power curve where any power outputs that fall off the curve can be seen as performance errors. Existing work using this idea has used data from a turbine's Supervisory Control & Acquisition (SCADA) system to filter and analyse fault & alarm data using regression techniques. In contrast to previous work, we explore how deep learning can be applied to fault prediction from open access meteorological data only.
Secure multiparty computations in floating-point arithmetic
Authors: Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, Ruiyu Zhu
Date: 2020-01-09 00:00:00
Tasks: All, Mathematical Proofs, Privacy Preserving, regression

Secure multiparty computations enable the distribution of so-called shares of sensitive data to multiple parties such that the multiple parties can effectively process the data while being unable to glean much information about the data (at least not without collusion among all parties to put back together all the shares). Thus, the parties may conspire to send all their processed results to a trusted third party (perhaps the data provider) at the conclusion of the computations, with only the trusted third party being able to view the final results. Secure multiparty computations for privacy-preserving machine-learning turn out to be possible using solely standard floating-point arithmetic, at least with a carefully controlled leakage of information less than the loss of accuracy due to roundoff, all backed by rigorous mathematical proofs of worst-case bounds on information loss and numerical stability in finite-precision arithmetic. Numerical examples illustrate the high performance attained on commodity off-the-shelf hardware for generalized linear models, including ordinary linear least-squares regression, binary and multinomial logistic regression, probit regression, and Poisson regression.
Theory In, Theory Out: The uses of social theory in machine learning for social science
Authors: Jason Radford, Kenneth Joseph
Date: 2020-01-09 00:00:00
Tasks: BIG-bench Machine Learning, Fairness, Navigate

Research at the intersection of machine learning and the social sciences has provided critical new insights into social behavior. At the same time, a variety of critiques have been raised ranging from technical issues with the data used and features constructed, problematic assumptions built into models, their limited interpretability, and their contribution to bias and inequality. We argue such issues arise primarily because of the lack of social theory at various stages of the model building and analysis. In the first half of this paper, we walk through how social theory can be used to answer the basic methodological and interpretive questions that arise at each stage of the machine learning pipeline. In the second half, we show how theory can be used to assess and compare the quality of different social learning models, including interpreting, generalizing, and assessing the fairness of models. We believe this paper can act as a guide for computer and social scientists alike to navigate the substantive questions involved in applying the tools of machine learning to social data.
Camera-Based Adaptive Trajectory Guidance via Neural Networks
Authors: Aditya Rajguru, Christopher Collander, William J. Beksi
Date: 2020-01-09 00:00:00

In this paper, we introduce a novel method to capture visual trajectories for navigating an indoor robot in dynamic settings using streaming image data. First, an image processing pipeline is proposed to accurately segment trajectories from noisy backgrounds. Next, the captured trajectories are used to design, train, and compare two neural network architectures for predicting acceleration and steering commands for a line following robot over a continuous space in real time. Lastly, experimental results demonstrate the performance of the neural networks versus human teleoperation of the robot and the viability of the system in environments with occlusions and/or low-light conditions.
Open Domain Question Answering Using Web Tables
Authors: Kaushik Chakrabarti, Zhimin Chen, Siamak Shakeri, Guihong Cao
Date: 2020-01-10 00:00:00
Tasks: Open-Domain Question Answering, Question Answering, Semantic Similarity, Semantic Textual Similarity

Tables extracted from web documents can be used to directly answer many web search queries. Previous works on question answering (QA) using web tables have focused on factoid queries, i.e., those answerable with a short string like person name or a number. However, many queries answerable using tables are non-factoid in nature. In this paper, we develop an open-domain QA approach using web tables that works for both factoid and non-factoid queries. Our key insight is to combine deep neural network-based semantic similarity between the query and the table with features that quantify the dominance of the table in the document as well as the quality of the information in the table. Our experiments on real-life web search queries show that our approach significantly outperforms state-of-the-art baseline approaches. Our solution is used in production in a major commercial web search engine and serves direct answers for tens of millions of real user queries per month.
A new approach for trading based on Long Short Term Memory technique
Authors: Zineb Lanbouri, Saaid Achchab
Date: 2020-01-10 00:00:00
Tasks: Prediction, Stock Market Prediction

The stock market prediction has always been crucial for stakeholders, traders and investors. We developed an ensemble Long Short Term Memory (LSTM) model that includes two-time frequencies (annual and daily parameters) in order to predict the next-day Closing price (one step ahead). Based on a four-step approach, this methodology is a serial combination of two LSTM algorithms. The empirical experiment is applied to 417 NY stock exchange companies. Based on Open High Low Close metrics and other financial ratios, this approach proves that the stock market prediction can be improved.
Time-Varying Graph Learning with Constraints on Graph Temporal Variation
Authors: Koki Yamada, Yuichi Tanaka, Antonio Ortega
Date: 2020-01-10 00:00:00
Tasks: Graph Learning

We propose a novel framework for learning time-varying graphs from spatiotemporal measurements. Given an appropriate prior on the temporal behavior of signals, our proposed method can estimate time-varying graphs from a small number of available measurements. To achieve this, we introduce two regularization terms in convex optimization problems that constrain sparseness of temporal variations of the time-varying networks. Moreover, a computationally-scalable algorithm is introduced to efficiently solve the optimization problem. The experimental results with synthetic and real datasets (point cloud and temperature data) demonstrate our proposed method outperforms the existing state-of-the-art methods.
Two Applications of Deep Learning in the Physical Layer of Communication Systems
Authors: Emil Björnson, Pontus Giselsson
Date: 2020-01-10 00:00:00
Tasks: Decision Making, Deep Learning

Deep learning has proved itself to be a powerful tool to develop data-driven signal processing algorithms for challenging engineering problems. By learning the key features and characteristics of the input signals, instead of requiring a human to first identify and model them, learned algorithms can beat many man-made algorithms. In particular, deep neural networks are capable of learning the complicated features in nature-made signals, such as photos and audio recordings, and use them for classification and decision making. The situation is rather different in communication systems, where the information signals are man-made, the propagation channels are relatively easy to model, and we know how to operate close to the Shannon capacity limits. Does this mean that there is no role for deep learning in the development of future communication systems?