pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Deceive with Attention-Based Explanations
Authors: Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, Zachary C. Lipton
Date: 2019-09-17 00:00:00
Proceeding: ACL 2020 6
Tasks: Fairness

Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing. In addition to yielding gains in predictive accuracy, attention weights are often claimed to confer interpretability, purportedly useful both for providing insights to practitioners and for explaining why a model makes its decisions to stakeholders. We call the latter use of attention mechanisms into question by demonstrating a simple method for training models to produce deceptive attention masks. Our method diminishes the total weight assigned to designated impermissible tokens, even when the models can be shown to nevertheless rely on these features to drive predictions. Across multiple models and tasks, our approach manipulates attention weights while paying surprisingly little cost in accuracy. Through a human study, we show that our manipulated attention-based explanations deceive people into thinking that predictions from a model biased against gender minorities do not rely on the gender. Consequently, our results cast doubt on attention's reliability as a tool for auditing algorithms in the context of fairness and accountability.
Say Anything: Automatic Semantic Infelicity Detection in L2 English Indefinite Pronouns
Authors: Ella Rabinovich, Julia Watson, Barend Beekhuizen, Suzanne Stevenson
Date: 2019-09-17 00:00:00
Proceeding: CONLL 2019 11

Computational research on error detection in second language speakers has mainly addressed clear grammatical anomalies typical to learners at the beginner-to-intermediate level. We focus instead on acquisition of subtle semantic nuances of English indefinite pronouns by non-native speakers at varying levels of proficiency. We first lay out theoretical, linguistically motivated hypotheses, and supporting empirical evidence on the nature of the challenges posed by indefinite pronouns to English learners. We then suggest and evaluate an automatic approach for detection of atypical usage patterns, demonstrating that deep learning architectures are promising for this task involving nuanced semantic anomalies.
Minimax Confidence Intervals for the Sliced Wasserstein Distance
Authors: Tudor Manole, Sivaraman Balakrishnan, Larry Wasserman
Date: 2019-09-17 00:00:00
Tasks: Uncertainty Quantification

Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance--an easily computable variant of the Wasserstein distance. Specifically, we construct confidence intervals for the Sliced Wasserstein distance which have finite-sample validity under no assumptions or under mild moment assumptions. These intervals are adaptive in length to the regularity of the underlying distributions. We also bound the minimax risk of estimating the Sliced Wasserstein distance, and as a consequence establish that the lengths of our proposed confidence intervals are minimax optimal over appropriate distribution classes. To motivate the choice of these classes, we also study minimax rates of estimating a distribution under the Sliced Wasserstein distance. These theoretical findings are complemented with a simulation study demonstrating the deficiencies of the classical bootstrap, and the advantages of our proposed methods. We also show strong correspondences between our theoretical predictions and the adaptivity of our confidence interval lengths in simulations. We conclude by demonstrating the use of our confidence intervals in the setting of simulator-based likelihood-free inference. In this setting, contrasting popular approximate Bayesian computation methods, we develop uncertainty quantification methods with rigorous frequentist coverage guarantees.
Reservoirs learn to learn
Authors: Anand Subramoney, Franz Scherr, Wolfgang Maass
Date: 2019-09-16 00:00:00

We consider reservoirs in the form of liquid state machines, i.e., recurrently connected networks of spiking neurons with randomly chosen weights. So far only the weights of a linear readout were adapted for a specific task. We wondered whether the performance of liquid state machines can be improved if the recurrent weights are chosen with a purpose, rather than randomly. After all, weights of recurrent connections in the brain are also not assumed to be randomly chosen. Rather, these weights were probably optimized during evolution, development, and prior learning experiences for specific task domains. In order to examine the benefits of choosing recurrent weights within a liquid with a purpose, we applied the Learning-to-Learn (L2L) paradigm to our model: We optimized the weights of the recurrent connections -- and hence the dynamics of the liquid state machine -- for a large family of potential learning tasks, which the network might have to learn later through modification of the weights of readout neurons. We found that this two-tiered process substantially improves the learning speed of liquid state machines for specific tasks. In fact, this learning speed increases further if one does not train the weights of linear readouts at all, and relies instead on the internal dynamics and fading memory of the network for remembering salient information that it could extract from preceding examples for the current learning task. This second type of learning has recently been proposed to underlie fast learning in the prefrontal cortex and motor cortex, and hence it is of interest to explore its performance also in models. Since liquid state machines share many properties with other types of reservoirs, our results raise the question whether L2L conveys similar benefits also to these other reservoirs.
High-Throughput In-Memory Computing for Binary Deep Neural Networks with Monolithically Integrated RRAM and 90nm CMOS
Authors: Shihui Yin, Xiaoyu Sun, Shimeng Yu, Jae-sun Seo
Date: 2019-09-16 00:00:00
Tasks: Edge-computing

Deep learning hardware designs have been bottlenecked by conventional memories such as SRAM due to density, leakage and parallel computing challenges. Resistive devices can address the density and volatility issues, but have been limited by peripheral circuit integration. In this work, we demonstrate a scalable RRAM based in-memory computing design, termed XNOR-RRAM, which is fabricated in a 90nm CMOS technology with monolithic integration of RRAM devices between metal 1 and 2. We integrated a 128x64 RRAM array with CMOS peripheral circuits including row/column decoders and flash analog-to-digital converters (ADCs), which collectively become a core component for scalable RRAM-based in-memory computing towards large deep neural networks (DNNs). To maximize the parallelism of in-memory computing, we assert all 128 wordlines of the RRAM array simultaneously, perform analog computing along the bitlines, and digitize the bitline voltages using ADCs. The resistance distribution of low resistance states is tightened by write-verify scheme, and the ADC offset is calibrated. Prototype chip measurements show that the proposed design achieves high binary DNN accuracy of 98.5% for MNIST and 83.5% for CIFAR-10 datasets, respectively, with energy efficiency of 24 TOPS/W and 158 GOPS throughput. This represents 5.6X, 3.2X, 14.1X improvements in throughput, energy-delay product (EDP), and energy-delay-squared product (ED2P), respectively, compared to the state-of-the-art literature. The proposed XNOR-RRAM can enable intelligent functionalities for area-/energy-constrained edge computing devices.
Barnes-Hut-SNE
Authors: Laurens van der Maaten
Date: 2013-01-15 00:00:00

The paper presents an O(N log N)-implementation of t-SNE -- an embedding technique that is commonly used for the visualization of high-dimensional data in scatter plots and that normally runs in O(N^2). The new implementation uses vantage-point trees to compute sparse pairwise similarities between the input data objects, and it uses a variant of the Barnes-Hut algorithm - an algorithm used by astronomers to perform N-body simulations - to approximate the forces between the corresponding points in the embedding. Our experiments show that the new algorithm, called Barnes-Hut-SNE, leads to substantial computational advantages over standard t-SNE, and that it makes it possible to learn embeddings of data sets with millions of objects.
Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts
Authors: Rohan Bavishi, Michael Pradel, Koushik Sen
Date: 2018-08-31 00:00:00

Most of the JavaScript code deployed in the wild has been minified, a process in which identifier names are replaced with short, arbitrary and meaningless names. Minified code occupies less space, but also makes the code extremely difficult to manually inspect and understand. This paper presents Context2Name, a deep learningbased technique that partially reverses the effect of minification by predicting natural identifier names for minified names. The core idea is to predict from the usage context of a variable a name that captures the meaning of the variable. The approach combines a lightweight, token-based static analysis with an auto-encoder neural network that summarizes usage contexts and a recurrent neural network that predict natural names for a given usage context. We evaluate Context2Name with a large corpus of real-world JavaScript code and show that it successfully predicts 47.5% of all minified identifiers while taking only 2.9 milliseconds on average to predict a name. A comparison with the state-of-the-art tools JSNice and JSNaughty shows that our approach performs comparably in terms of accuracy while improving in terms of efficiency. Moreover, Context2Name complements the state-of-the-art by predicting 5.3% additional identifiers that are missed by both existing tools.
Structure Label Prediction Using Similarity-Based Retrieval and Weakly-Supervised Label Mapping
Authors: Yazeed Alaudah, Motaz Alfarraj, Ghassan AlRegib
Date: 2019-05-16 00:00:00
Tasks: BIG-bench Machine Learning, Retrieval, Seismic Interpretation

Recently, there has been significant interest in various supervised machine learning techniques that can help reduce the time and effort consumed by manual interpretation workflows. However, most successful supervised machine learning algorithms require huge amounts of annotated training data. Obtaining these labels for large seismic volumes is a very time-consuming and laborious task. We address this problem by presenting a weakly-supervised approach for predicting the labels of various seismic structures. By having an interpreter select a very small number of exemplar images for every class of subsurface structures, we use a novel similarity-based retrieval technique to extract thousands of images that contain similar subsurface structures from the seismic volume. By assuming that similar images belong to the same class, we obtain thousands of image-level labels for these images; we validate this assumption in our results section. We then introduce a novel weakly-supervised algorithm for mapping these rough image-level labels into more accurate pixel-level labels that localize the different subsurface structures within the image. This approach dramatically simplifies the process of obtaining labeled data for training supervised machine learning algorithms on seismic interpretation tasks. Using our method we generate thousands of automatically-labeled images from the Netherlands Offshore F3 block with reasonably accurate pixel-level labels. We believe this work will allow for more advances in machine learning-enabled seismic interpretation.
Fast Large-Scale Discrete Optimization Based on Principal Coordinate Descent
Authors: Huan Xiong, Mengyang Yu, Li Liu, Fan Zhu, Fumin Shen, Ling Shao
Date: 2019-09-16 00:00:00
Tasks: Quantization

Binary optimization, a representative subclass of discrete optimization, plays an important role in mathematical optimization and has various applications in computer vision and machine learning. Usually, binary optimization problems are NP-hard and difficult to solve due to the binary constraints, especially when the number of variables is very large. Existing methods often suffer from high computational costs or large accumulated quantization errors, or are only designed for specific tasks. In this paper, we propose a fast algorithm to find effective approximate solutions for general binary optimization problems. The proposed algorithm iteratively solves minimization problems related to the linear surrogates of loss functions, which leads to the updating of some binary variables most impacting the value of loss functions in each step. Our method supports a wide class of empirical objective functions with/without restrictions on the numbers of $1$s and $-1$s in the binary variables. Furthermore, the theoretical convergence of our algorithm is proven, and the explicit convergence rates are derived, for objective functions with Lipschitz continuous gradients, which are commonly adopted in practice. Extensive experiments on several binary optimization tasks and large-scale datasets demonstrate the superiority of the proposed algorithm over several state-of-the-art methods in terms of both effectiveness and efficiency.
Learning Representations from Healthcare Time Series Data for Unsupervised Anomaly Detection
Authors: João Pereira, Margarida Silveira
Date: 2019-04-04 00:00:00
Proceeding: 2019 IEEE International Conference on Big Data and Smart Computing (BigComp) 2019 4
Tasks: Anomaly Detection, Clustering, Outlier Detection, Representation Learning, Time Series, Time Series Analysis, Unsupervised Anomaly Detection

The amount of time series data generated in Healthcare is growing very fast and so is the need for methods that can analyse these data, detect anomalies and provide meaningful insights. However, most of the data available is unlabelled and, therefore, anomaly detection in this scenario has been a great challenge for researchers and practitioners. Recently, unsupervised representation learning with deep generative models has been applied to find representations of data, without the need for big labelled datasets. Motivated by their success, we propose an unsupervised framework for anomaly detection in time series data. In our method, both representation learning and anomaly detection are fully unsupervised. In addition, the training data may contain anomalous data. We first learn representations of time series using a Variational Recurrent Autoencoder. Afterwards, based on those representations, we detect anomalous time series using Clustering and the Wasserstein distance. Our results on the publicly available ECG5000 electrocardiogram dataset show the ability of the proposed approach to detect anomalous heartbeats in a fully unsupervised fashion, while providing structured and expressive data representations. Furthermore, our approach outperforms previous supervised and unsupervised methods on this dataset.
Meet Spinky: An Open-Source Spindle and K-Complex Detection Toolbox Validated on the Open-Access Montreal Archive of Sleep Studies (MASS).
Authors: Tarek Lajnef, Christian O'Reilly, Etienne Combrisson, Sahbi Ch1aibi, Jb Eichenlaub, Perrine Ruby, Pierre-Emanuel Aguera, Mounir Samet, Abdennaceur Kachouri, Sonia Frenette, Julie Carrier, Karim Jerbi
Date: 2017-03-02 00:00:00
Proceeding: Frontiers in Neuroinformatics 2017 3
Tasks: Benchmarking, EEG, Electroencephalogram (EEG), K-complex detection, Spindle Detection

Sleep spindles and K-complexes are among the most prominent micro-events observed in electroencephalographic (EEG) recordings during sleep. These EEG microstructures are thought to be hallmarks of sleep-related cognitive processes. Although tedious and time-consuming, their identification and quantification is important for sleep studies in both healthy subjects and patients with sleep disorders. Therefore, procedures for automatic detection of spindles and K-complexes could provide valuable assistance to researchers and clinicians in the field. Recently, we proposed a framework for joint spindle and K-complex detection (Lajnef et al., 2015a) based on a Tunable Q-factor Wavelet Transform (TQWT; Selesnick, 2011a) and morphological component analysis (MCA). Using a wide range of performance metrics, the present article provides critical validation and benchmarking of the proposed approach by applying it to open-access EEG data from the Montreal Archive of Sleep Studies (MASS; O'Reilly et al., 2014). Importantly, the obtained scores were compared to alternative methods that were previously tested on the same database. With respect to spindle detection, our method achieved higher performance than most of the alternative methods. This was corroborated with statistic tests that took into account both sensitivity and precision (i.e., Matthew's coefficient of correlation (MCC), F1, Cohen κ). Our proposed method has been made available to the community via an open-source tool named Spinky (for spindle and K-complex detection). Thanks to a GUI implementation and access to Matlab and Python resources, Spinky is expected to contribute to an open-science approach that will enhance replicability and reliable comparisons of classifier performances for the detection of sleep EEG microstructure in both healthy and patient populations.
A single channel sleep-spindle detector based on multivariate classification of EEG epochs: MUSSDET.
Authors: DanielLachner-Piza, Nino Epitashvili, Andreas Schulze-Bonhage, Thomas Stieglitz, Julia Jacobs, Matthias Dümpelmann
Date: 2018-03-01 00:00:00
Proceeding: Journal of Neuroscience Methods Volume 297 2018 3
Tasks: EEG, Electroencephalogram (EEG), General Classification, Sensitivity, Specificity, Spindle Detection

BACKGROUND:
Studies on sleep-spindles are typically based on visual-marks performed by experts, however this process is time consuming and presents a low inter-expert agreement, causing the data to be limited in quantity and prone to bias. An automatic detector would tackle these issues by generating large amounts of objectively marked data.

NEW METHOD:
Our goal was to develop a sensitive, precise and robust sleep-spindle detection method. Emphasis has been placed on achieving a consistent performance across heterogeneous recordings and without the need for further parameter fine tuning. The developed detector runs on a single channel and is based on multivariate classification using a support vector machine. Scalp-electroencephalogram recordings were segmented into epochs which were then characterized by a selection of relevant and non-redundant features. The training and validation data came from the Medical Center-University of Freiburg, the test data consisted of 27 records coming from 2 public databases.

RESULTS:
Using a sample based assessment, 53% sensitivity, 37% precision and 96% specificity was achieved on the DREAMS database. On the MASS database, 77% sensitivity, 46% precision and 96% specificity was achieved. The developed detector performed favorably when compared to previous detectors. The classification of normalized EEG epochs in a multidimensional space, as well as the use of a validation set, allowed to objectively define a single detection threshold for all databases and participants.

CONCLUSIONS:
The use of the developed tool will allow increasing the data-size and statistical significance of research studies on the role of sleep-spindles.
Multichannel sleep spindle detection using sparse low-rank optimization
Authors: Ankit Parekha, Ivan W. Selesnick, Ricardo S.Osorio, Andrew W. Vargad, David M. Rapoport, Indu Ayappa
Date: 2017-08-15 00:00:00
Proceeding: Journal of Neuroscience Methods Volume 288 2017 8
Tasks: EEG, Electroencephalogram (EEG), Spindle Detection

BACKGROUND:
Automated single-channel spindle detectors, for human sleep EEG, are blind to the presence of spindles in other recorded channels unlike visual annotation by a human expert.

NEW METHOD:
We propose a multichannel spindle detection method that aims to detect global and local spindle activity in human sleep EEG. Using a non-linear signal model, which assumes the input EEG to be the sum of a transient and an oscillatory component, we propose a multichannel transient separation algorithm. Consecutive overlapping blocks of the multichannel oscillatory component are assumed to be low-rank whereas the transient component is assumed to be piecewise constant with a zero baseline. The estimated oscillatory component is used in conjunction with a bandpass filter and the Teager operator for detecting sleep spindles.

RESULTS AND COMPARISON WITH OTHER METHODS:
The proposed method is applied to two publicly available databases and compared with 7 existing single-channel automated detectors. F1 scores for the proposed spindle detection method averaged 0.66 (0.02) and 0.62 (0.06) for the two databases, respectively. For an overnight 6 channel EEG signal, the proposed algorithm takes about 4min to detect sleep spindles simultaneously across all channels with a single setting of corresponding algorithmic parameters.

CONCLUSIONS:
The proposed method attempts to mimic and utilize, for better spindle detection, a particular human expert behavior where the decision to mark a spindle event may be subconsciously influenced by the presence of a spindle in EEG channels other than the central channel visible on a digital screen.
The Dreem Headband as an Alternative to Polysomnography for EEG Signal Acquisition and Sleep Staging
Authors: Pierrick J. Arnal, Valentin Thorey, Michael E. Ballard, Albert Bou Hernandez, Antoine Guillot, Hugo Jourde, Mason Harris, Mathias Guillard, Pascal Van Beers, Mounir Chennaoui, Fabien Sauvet
Date: 2019-06-10 00:00:00
Proceeding: bioRxiv 2019 6
Tasks: EEG, Electroencephalogram (EEG), Sleep Quality, Sleep Stage Detection, Sleep Staging

Despite the central role of sleep in our lives and the high prevalence of sleep disorders, sleep is still poorly understood. The development of ambulatory technologies capable of monitoring brain activity during sleep longitudinally is critical to advancing sleep science and facilitating the diagnosis of sleep disorders. We introduced the Dreem headband (DH) as an affordable, comfortable, and user-friendly alternative to polysomnography (PSG). The purpose of this study was to assess the signal acquisition of the DH and the performance of its embedded automatic sleep staging algorithms compared to the gold-standard clinical PSG scored by 5 sleep experts. Thirty-one subjects completed an over-night sleep study at a sleep center while wearing both a PSG and the DH simultaneously. We assessed 1) the EEG signal quality between the DH and the PSG, 2) the heart rate, breathing frequency, and respiration rate variability (RRV) agreement between the DH and the PSG, and 3) the performance of the DH’s automatic sleep staging according to AASM guidelines vs. PSG sleep experts manual scoring. Results demonstrate a strong correlation between the EEG signals acquired by the DH and those from the PSG, and the signals acquired by the DH enable monitoring of alpha (r= 0.71 ± 0.13), beta (r= 0.71 ± 0.18), delta (r = 0.76 ± 0.14), and theta (r = 0.61 ± 0.12) frequencies during sleep. The mean absolute error for heart rate, breathing frequency and RRV was 1.2 ± 0.5 bpm, 0.3 ± 0.2 cpm and 3.2 ± 0.6 %, respectively. Automatic Sleep Staging reached an overall accuracy of 83.5 ± 6.4% (F1 score : 83.8 ± 6.3) for the DH to be compared with an average of 86.4 ± 8.0% (F1 score: 86.3 ± 7.4) for the five sleep experts. These results demonstrate the capacity of the DH to both precisely monitor sleep-related physiological signals and process them accurately into sleep stages. This device paves the way for high-quality, large-scale, longitudinal sleep studies.