pwc-archive/papers-with-abstracts huggingface dataset:
Learning Distributed Representations of Symbolic Structure Using Binding and Unbinding Operations
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Natural Language Inference

Widely used recurrent units, including Long-short Term Memory (LSTM) and Gated Recurrent Unit (GRU), perform well on natural language tasks, but their ability to learn structured representations is still questionable. Exploiting Tensor Product Representations (TPRs) --- distributed representations of symbolic structure in which vector-embedded symbols are bound to vector-embedded structural positions --- we propose the TPRU, a recurrent unit that, at each time step, explicitly executes structural-role binding and unbinding operations to incorporate structural information into learning. Experiments are conducted on both the Logical Entailment task and the Multi-genre Natural Language Inference (MNLI) task, and our TPR-derived recurrent unit provides strong performance with significantly fewer parameters than LSTM and GRU baselines. Furthermore, our learnt TPRU trained on MNLI demonstrates solid generalisation ability on downstream tasks.
DONUT: CTC-based Query-by-Example Keyword Spotting
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Keyword Spotting

Keyword spotting—or wakeword detection—is an essential feature for hands-free operation of modern voice-controlled devices. With such devices becoming ubiquitous, users might want to choose a personalized custom wakeword. In this work, we present DONUT, a CTC-based algorithm for online query-by-example keyword spotting that enables custom wakeword detection. The algorithm works by recording a small number of training examples from the user, generating a set of label sequence hypotheses from these training examples, and detecting the wakeword by aggregating the scores of all the hypotheses given a new audio recording. Our method combines the generalization and interpretability of CTC-based keyword spotting with the user-adaptation and convenience of a conventional query-by-example system. DONUT has low computational requirements and is well-suited for both learning and inference on embedded systems without requiring private user data to be uploaded to the cloud.
A Comparable Phone Set for the TIMIT Dataset Discovered in Clustering of Listen, Attend and Spell
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018

Listen, Attend and Spell(LAS) maps a sequence of acoustic spectra directly to a sequence of graphemes, with no explicit internal representation of phones. This paper asks whether LAS can be used as a scientific tool, to discover the phone set of a language whose phone set may be controversial or unknown.  Phonemes have a precise linguistic definition, but phones may be defined in any manner that is convenient for speech technology: we propose that a practical phone set is one that can be inferred from speech following certain procedures, but that is also highly predictive of the word sequence. We demonstrate that such a phone set can be inferred by clustering the hidden nodes activation vectors of an LAS model during training, thus encouraging the model to learn a hidden representation characterized by acoustically compact clusters that are nevertheless predictive of the word sequence. We further define a metric for the quality of a phone set (the sum of conditional entropy of the graphemes given the phone set and the phones given the acoustics), and demonstrate that according to this metric, the clustered-LAS phone set is comparable to the original TIMIT phone set. Specifically, the clustered-LAS phone set is closer to the acoustics; the original TIMIT phone set is closer to the text.
How transferable are features in convolutional neural network acoustic models across languages?
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Specificity, speech-recognition, Speech Recognition, Transfer Learning

Characterization of the representations learned in intermediate layers of deep networks can provide valuable insight into the nature of a task and can guide the development of well-tailored learning strategies. Here we study convolutional neural network-based acoustic models in the context of automatic speech recognition. Adapting a method proposed by Yosinski et al. [2014], we measure the transferability of each layer between German and English to assess the their language-specifity. We observe three distinct regions of transferability: (1) the first two layers are entirely transferable between languages, (2) layers 2–8 are also highly transferable but we find evidence of some language specificity, (3) the subsequent fully connected layers are more language specific but can be successfully finetuned to the target language. To further probe the effect of weight freezing, we performed follow-up experiments using freeze-training [Raghu et al., 2017]. Our results are consistent with the observation that CCNs converge 'bottom up' during training and demonstrate the benefit of freeze training, especially for transfer learning.
Local and global model interpretability via backward selection and clustering
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Clustering, Decision Making

Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model. Existing techniques are often restricted to a specific type of predictor or based on input saliency, which may be undesirably sensitive to factors unrelated to the model's decision making process. We instead propose sufficient input subsets that identify minimal subsets of features whose observed values alone suffice for the same decision to be reached, even if all other input feature values are missing. General principles that globally govern a model's decision-making can also be revealed by searching for clusters of such input patterns across many data points. Our approach is conceptually straightforward, entirely model-agnostic, simply implemented using instance-wise backward selection, and able to produce more concise rationales than existing techniques. We demonstrate the utility of our interpretation method on neural network models trained on text and image data.
Exponential Family Word Embeddings: An Iterative Approach for Learning Word Vectors
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Word Embeddings

GloVe and Skip-gram word embedding methods learn word vectors by decomposing a denoised matrix of word co-occurrences into a product of low-rank matrices. In this work, we propose an iterative algorithm for computing word vectors based on modeling word co-occurrence matrices with Generalized Low Rank Models. Our algorithm generalizes both Skip-gram and GloVe as well as giving rise to other embedding methods based on the specified co-occurrence matrix, distribution of co-occurences, and the number of iterations in the iterative algorithm. For example, using a Tweedie distribution with one iteration results in GloVe and using a Multinomial distribution with full-convergence mode results in Skip-gram. Experimental results demonstrate that multiple iterations of our algorithm improves results over the GloVe method on the Google word analogy similarity task.
SpeakerGAN: Recognizing Speakers in New Languages with Generative Adversarial Networks
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Speaker Verification

  Verifying a person's identity based on their voice is a challenging, real-world problem in biometric security. A crucial requirement of such speaker verification systems is to be domain robust. Performance should not degrade even if speakers are talking in languages not seen during training. To this end, we present a flexible and interpretable framework for learning domain invariant speaker embeddings using Generative Adversarial Networks. We combine adversarial training with an angular margin loss function, which encourages the speaker embedding model to be discriminative by directly optimizing for cosine similarity between classes. We are able to beat a strong baseline system using a cosine distance classifier and a simple score-averaging strategy. Our results also show that models with adversarial adaptation perform significantly better than unadapted models. In an attempt to better understand this behavior, we quantitatively measure the degree of invariance induced by our proposed methods using Maximum Mean Discrepancy and Frechet distances. Our analysis shows that our proposed adversarial speaker embedding models significantly reduce the distance between source and target data distributions, while performing similarly on the former and better on the latter.
Are you tough enough? Framework for Robustness Validation of Machine Comprehension Systems
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Question Answering, Reading Comprehension

Deep Learning NLP domain lacks procedures for the analysis of model robustness. In this paper we propose a framework which validates robustness of any Question Answering model through model explainers. We propose that output of a robust model should be invariant to alterations that do not change its semantics. We test this property by manipulating question in two ways: swapping important question word for 1) its semantically correct synonym and 2) for word vector that is close in embedding space. We estimate importance of words in asked questions with Locally Interpretable Model Agnostic Explanations method (LIME). With these two steps we compare state-of-the-art Q&A models. We show that although accuracy of state-of-the-art models is high, they are very fragile to changes in the input. We can choose architecture that is more immune to attacks and thus more robust and stable in production environment. Morevoer, we propose 2 adversarial training scenarios which raise model sensitivity to true synonyms by up to 7% accuracy measure. Our findings help to understand which models are more stable and how they can be improved. In addition, we have created and published a new dataset that may be used for validation of robustness of a Q&A model.
Identifying and Controlling Important Neurons in Neural Machine Translation
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) models learn representations containing substantial linguistic information. However, it is not clear if such information is fully distributed or if some of it can be attributed to individual neurons. We develop unsupervised methods for discovering important neurons in NMT models. Our methods rely on the intuition that different models learn similar properties, and do not require any costly external supervision. We show experimentally that translation quality depends on the discovered neurons, and find that many of them capture common linguistic phenomena. Finally, we show how to control NMT translations in predictable ways, by modifying activations of individual neurons.
Interpreting Word Embeddings with Eigenvector Analysis
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Word Embeddings

Dense word vectors have proven their values in many downstream NLP tasks over the past few years. However, the dimensions of such embeddings are not easily interpretable. Out of the d-dimensions in a word vector, we would not be able to understand what high or low values mean. Previous approaches addressing this issue have mainly focused on either training sparse/non-negative constrained word embeddings, or post-processing standard pre-trained word embeddings. On the other hand, we analyze conventional word embeddings trained with Singular Value Decomposition, and reveal similar interpretability. We use a novel eigenvector analysis method inspired from Random Matrix Theory and show that semantically coherent groups not only form in the row space, but also the column space. This allows us to view individual word vector dimensions as human-interpretable semantic features.
Robust Domain Adaptation By Augmented Cyclic Adversarial Learning
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Domain Adaptation, speech-recognition, Speech Recognition, Unsupervised Domain Adaptation

Training a model to perform a task typically requires a large amount of data from the domains in which the task will be applied. However, it is often the case that data are abundant in some domains but scarce in others. Domain adaptation deals with the challenge of adapting a model trained from a data-rich source domain to perform well in a data-poor target domain. In general, this requires learning plausible mappings between domains. CycleGAN is a powerful framework that efficiently learns to map inputs from one domain to another using adversarial training and a cycle-consistency constraint. However, the conventional approach of enforcing cycle-consistency via reconstruction may be overly restrictive in cases where one or more domains have limited training data. In this paper, we propose an augmented cyclic adversarial learning model that enforces the cycle-consistency constraint via an external task specific model, which encourages the preservation of task-relevant content as opposed to exact reconstruction. This task specific model both relaxes the cycle-consistency constraint and complements the role of the discriminator during training, serving as an augmented information source for learning the mapping. We explore adaptation in speech and visual domains in low resource in supervised setting. In speech domains, we adopt a speech recognition model from each domain as the task specific model. Our approach improves absolute performance of speech recognition by 2% for female speakers in the TIMIT dataset, where the majority of training samples are from male voices. In low-resource visual domain adaptation, the results show that our approach improves absolute performance by 14% and 4% when adapting SVHN to MNIST and vice versa, respectively, which outperforms unsupervised domain adaptation methods that require high-resource unlabeled target domain.
Improving Sentence Representations with Multi-view Frameworks
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: MULTI-VIEW LEARNING, Sentence

Multi-view learning can provide self-supervision when different views are available of the same data. Distributional hypothesis provides another form of useful self-supervision from adjacent sentences which are plentiful in large unlabelled corpora. Motivated by the asymmetry in the two hemispheres of the human brain as well as the observation that different learning architectures tend to emphasise different aspects of sentence meaning, we present two multi-view frameworks for learning sentence representations in an unsupervised fashion. One framework uses a generative objective and the other a discriminative one. In both frameworks, the final representation is an ensemble of two views, in which, one view encodes the input sentence with a Recurrent Neural Network (RNN), and the other view encodes it with a simple linear model. We show that, after learning, the vectors produced by our multi-view frameworks provide improved representations over their single-view learnt counterparts, and the combination of different views gives representational improvement over each view and demonstrates solid transferability on standard downstream tasks.
On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Inductive Bias, Multi-Task Learning, speech-recognition, Speech Recognition

End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER). This suggests that predicting sequences of words directly may be helpful instead. However, training with word-level supervision can be more difficult due to the sparsity of examples per label class. In this paper we analyze an end-to-end ASR model that combines a word-and-character representation in a multi-task learning (MTL) framework. We show that it improves on the WER and study how the word-level model can benefit from character-level supervision by analyzing the learned inductive preference bias of each model component empirically. We find that by adding character-level supervision, the MTL model interpolates between recognizing more frequent words (preferred by the word-level model) and shorter words (preferred by the character-level model).
A comprehensive analysis on attention models
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: speech-recognition, Speech Recognition

Sequence-to-sequence attention-based models are a promising approach for end-to-end speech recognition. The increased model power makes the training procedure more difficult, and analyzing failure modes of these models becomes harder because of the end-to-end nature. In this work, we present various analyses to better understand training and model properties. We investigate on pretraining variants such as growing in depth and width, and their impact on the final performance, which leads to over 8% relative improvement in word error rate. For a better understanding of how the attention process works, we study the encoder output and the attention energies and weights. Our experiments were performed on Switchboard, LibriSpeech and Wall Street Journal.
Robust Spoken Term Detection Automatically Adjusted for a Given Threshold
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Keyword Spotting, Structured Prediction

Spoken term detection (STD) is the task of determining whether and where a given word or phrase appears in a given segment of speech. Algorithms for STD are often aimed at maximizing the gap between the scores of positive and negative examples. As such they are focused on ensuring that utterances where the term appears are ranked higher than utterances where the term does not appear. However, they do not determine a detection threshold between the two. In this paper, we propose a new approach for setting an absolute detection threshold for all terms by introducing a new calibrated loss function. The advantage of minimizing this loss function during training is that it aims at maximizing not only the relative ranking scores, but also adjusts the system to use a fixed threshold and thus enhances system robustness and maximizes the detection accuracy rates. We use the new loss function in the structured prediction setting and extend the discriminative keyword spotting algorithm for learning the spoken term detector with a single threshold for all terms. We further demonstrate the effectiveness of the new loss function by applying it on a deep neural Siamese network in a weakly supervised setting for template-based spoken term detection, again with a single fixed threshold. Experiments with the TIMIT, WSJ and Switchboard corpora showed that our approach not only improved the accuracy rates when a fixed threshold was used but also obtained higher Area Under Curve (AUC).
Investigating the effect of residual and highway connections in speech enhancement models
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Denoising, Speech Denoising, Speech Enhancement

  Residual and skip connections play an important role in many current
  generative models. Although their theoretical and numerical advantages
  are understood, their role in speech enhancement systems has not been
  investigated so far. When performing spectral speech enhancement,
  residual connections are very similar in nature to spectral subtraction,
  which is the one of the most commonly employed speech enhancement approaches.
  Highway networks, on the other hand, can be seen as a combination of spectral
  masking and spectral subtraction. However, when using deep neural networks, such operations would
  normally happen in a transformed spectral domain, as opposed to traditional speech
  enhancement where all operations are often done directly on the spectrum.
  In this paper, we aim to investigate the role of residual and highway
  connections in deep neural networks for speech enhancement, and verify whether
  or not they operate similarly to their traditional, digital signal processing
  counterparts. We visualize the outputs of such connections, projected back to
  the spectral domain, in models trained for speech denoising, and show that while
  skip connections do not necessarily improve performance with regards to the
  number of parameters, they make speech enhancement models more interpretable.
Audio Adversarial example detection using Generative Adversarial Networks
Authors: Anonymous
Date: 2018-10-22 00:00:00

In this paper, we exploit the discriminator's knowledge of the data manifold to effectively detect audio adversarial examples which are generated for both white box and black box ASR systems. We then investigate ways to mitigate them.
Cycle-Consistent GAN Front-End to Improve ASR Robustness to Perturbed Speech
Authors: Anonymous
Date: 2018-10-22 00:00:00
Proceeding: NIPS Workshop IRASL 2018
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Generative Adversarial Network, speech-recognition, Speech Recognition

Automatic Speech Recognition (ASR) systems, which perform well on regular speech, are found to be vulnerable to adversarial examples generated by  small perturbations in the audio signal. Even naturally introduced perturbations in audio signal, caused by emotional and physical states of the speaker, can significantly degrade ASR performance. In this paper, we propose a front-end based on Cycle-Consistent Generative Adversarial Network (CycleGAN) to reduce the perturbations, and hence add robustness to ASR performance. CycleGAN is trained using non-parallel examples of perturbed and normal speech. Experiments on spontaneously generated laughter-speech and creaky voice datasets tested with Google cloud ASR show absolute improvements in WER of 14.9% and 11%, respectively, on speech converted using the CycleGAN based front-end as compared to the original perturbed speech.