pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Source Domain Adaptation with Mixture of Experts
Authors: Jiang Guo, Darsh J Shah, Regina Barzilay
Date: 2018-09-07 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Domain Adaptation, Mixture-of-Experts, Part-Of-Speech Tagging, Sentiment Analysis, Unsupervised Domain Adaptation

We propose a mixture-of-experts approach for unsupervised domain adaptation
from multiple sources. The key idea is to explicitly capture the relationship
between a target example and different source domains. This relationship,
expressed by a point-to-set metric, determines how to combine predictors
trained on various domains. The metric is learned in an unsupervised fashion
using meta-training. Experimental results on sentiment analysis and
part-of-speech tagging demonstrate that our approach consistently outperforms
multiple baselines and can robustly handle negative transfer.
Adversarial Domain Adaptation for Duplicate Question Detection
Authors: Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, Preslav Nakov
Date: 2018-09-07 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Domain Adaptation, Question Similarity

We address the problem of detecting duplicate questions in forums, which is
an important step towards automating the process of answering new questions. As
finding and annotating such potential duplicates manually is very tedious and
costly, automatic methods based on machine learning are a viable alternative.
However, many forums do not have annotated data, i.e., questions labeled by
experts as duplicates, and thus a promising solution is to use domain
adaptation from another forum that has such annotations. Here we focus on
adversarial domain adaptation, deriving important findings about when it
performs well and what properties of the domains are important in this regard.
Our experiments with StackExchange data show an average improvement of 5.6%
over the best baseline across multiple pairs of domains.
Cycle-Consistent Speech Enhancement
Authors: Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang, Juang
Date: 2018-09-06 00:00:00
Tasks: Multi-Task Learning, Speech Enhancement

Feature mapping using deep neural networks is an effective approach for
single-channel speech enhancement. Noisy features are transformed to the
enhanced ones through a mapping network and the mean square errors between the
enhanced and clean features are minimized. In this paper, we propose a
cycle-consistent speech enhancement (CSE) in which an additional inverse
mapping network is introduced to reconstruct the noisy features from the
enhanced ones. A cycle-consistent constraint is enforced to minimize the
reconstruction loss. Similarly, a backward cycle of mappings is performed in
the opposite direction with the same networks and losses. With
cycle-consistency, the speech structure is well preserved in the enhanced
features while noise is effectively reduced such that the feature-mapping
network generalizes better to unseen data. In cases where only unparalleled
noisy and clean data is available for training, two discriminator networks are
used to distinguish the enhanced and noised features from the clean and noisy
ones. The discrimination losses are jointly optimized with reconstruction
losses through adversarial multi-task learning. Evaluated on the CHiME-3
dataset, the proposed CSE achieves 19.60% and 6.69% relative word error rate
improvements respectively when using or without using parallel clean and noisy
speech data.
Adversarial Feature-Mapping for Speech Enhancement
Authors: Zhong Meng, Jinyu Li, Yifan Gong, Biing-Hwang, Juang
Date: 2018-09-06 00:00:00
Tasks: Speech Enhancement

Feature-mapping with deep neural networks is commonly used for single-channel
speech enhancement, in which a feature-mapping network directly transforms the
noisy features to the corresponding enhanced ones and is trained to minimize
the mean square errors between the enhanced and clean features. In this paper,
we propose an adversarial feature-mapping (AFM) method for speech enhancement
which advances the feature-mapping approach with adversarial learning. An
additional discriminator network is introduced to distinguish the enhanced
features from the real clean ones. The two networks are jointly optimized to
minimize the feature-mapping loss and simultaneously mini-maximize the
discrimination loss. The distribution of the enhanced features is further
pushed towards that of the clean features through this adversarial multi-task
training. To achieve better performance on ASR task, senone-aware (SA) AFM is
further proposed in which an acoustic model network is jointly trained with the
feature-mapping and discriminator networks to optimize the senone
classification loss in addition to the AFM losses. Evaluated on the CHiME-3
dataset, the proposed AFM achieves 16.95% and 5.27% relative word error rate
(WER) improvements over the real noisy data and the feature-mapping baseline
respectively and the SA-AFM achieves 9.85% relative WER improvement over the
multi-conditional acoustic model.
Learning Optimal Fair Policies
Authors: Razieh Nabi, Daniel Malinsky, Ilya Shpitser
Date: 2018-09-06 00:00:00
Tasks: Causal Inference, Decision Making, Fairness

Systematic discriminatory biases present in our society influence the way data is collected and stored, the way variables are defined, and the way scientific findings are put into practice as policy. Automated decision procedures and learning algorithms applied to such data may serve to perpetuate existing injustice or unfairness in our society. In this paper, we consider how to make optimal but fair decisions, which "break the cycle of injustice" by correcting for the unfair dependence of both decisions and outcomes on sensitive features (e.g., variables that correspond to gender, race, disability, or other protected attributes). We use methods from causal inference and constrained optimization to learn optimal policies in a way that addresses multiple potential biases which afflict data analysis in sensitive contexts, extending the approach of (Nabi and Shpitser 2018). Our proposal comes equipped with the theoretical guarantee that the chosen fair policy will induce a joint distribution for new instances that satisfies given fairness constraints. We illustrate our approach with both synthetic data and real criminal justice data.
82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Models
Authors: Aaron Smith, Bernd Bohnet, Miryam de Lhoneux, Joakim Nivre, Yan Shao, Sara Stymne
Date: 2018-09-06 00:00:00
Proceeding: CONLL 2018 10
Tasks: Dependency Parsing, POS, POS Tagging, Segmentation, Sentence, Sentence segmentation

We present the Uppsala system for the CoNLL 2018 Shared Task on universal
dependency parsing. Our system is a pipeline consisting of three components:
the first performs joint word and sentence segmentation; the second predicts
part-of- speech tags and morphological features; the third predicts dependency
trees from words and tags. Instead of training a single parsing model for each
treebank, we trained models with multiple treebanks for one language or closely
related languages, greatly reducing the number of models. On the official test
run, we ranked 7th of 27 teams for the LAS and MLAS metrics. Our system
obtained the best scores overall for word segmentation, universal POS tagging,
and morphological features.
A Bandit Approach to Multiple Testing with False Discovery Control
Authors: Kevin Jamieson, Lalit Jain
Date: 2018-09-06 00:00:00
Tasks: Drug Discovery

We propose an adaptive sampling approach for multiple testing which aims to maximize statistical power while ensuring anytime false discovery control. We consider $n$ distributions whose means are partitioned by whether they are below or equal to a baseline (nulls), versus above the baseline (actual positives). In addition, each distribution can be sequentially and repeatedly sampled. Inspired by the multi-armed bandit literature, we provide an algorithm that takes as few samples as possible to exceed a target true positive proportion (i.e. proportion of actual positives discovered) while giving anytime control of the false discovery proportion (nulls predicted as actual positives). Our sample complexity results match known information theoretic lower bounds and through simulations we show a substantial performance improvement over uniform sampling and an adaptive elimination style algorithm. Given the simplicity of the approach, and its sample efficiency, the method has promise for wide adoption in the biological sciences, clinical testing for drug discovery, and online A/B/n testing problems.
Deeply Learning Derivatives
Authors: Ryan Ferguson, Andrew Green
Date: 2018-09-06 00:00:00
Tasks: Deep Learning

This paper uses deep learning to value derivatives. The approach is broadly
applicable, and we use a call option on a basket of stocks as an example. We
show that the deep learning model is accurate and very fast, capable of
producing valuations a million times faster than traditional models. We develop
a methodology to randomly generate appropriate training data and explore the
impact of several parameters including layer width and depth, training data
quality and quantity on model speed and accuracy.
Automated Game Design via Conceptual Expansion
Authors: Matthew Guzdial, Mark Riedl
Date: 2018-09-06 00:00:00
Tasks: BIG-bench Machine Learning, Game Design

Automated game design has remained a key challenge within the field of Game
AI. In this paper, we introduce a method for recombining existing games to
create new games through a process called conceptual expansion. Prior automated
game design approaches have relied on hand-authored or crowd-sourced knowledge,
which limits the scope and applications of such systems. Our approach instead
relies on machine learning to learn approximate representations of games. Our
approach recombines knowledge from these learned representations to create new
games via conceptual expansion. We evaluate this approach by demonstrating the
ability for the system to recreate existing games. To the best of our
knowledge, this represents the first machine learning-based automated game
design system.
Deep Neural Net with Attention for Multi-channel Multi-touch Attribution
Authors: Ning li, Sai Kumar Arava, Chen Dong, Zhenyu Yan, Abhishek Pani
Date: 2018-09-06 00:00:00
Tasks: Marketing

Customers are usually exposed to online digital advertisement channels, such
as email marketing, display advertising, paid search engine marketing, along
their way to purchase or subscribe products( aka. conversion). The marketers
track all the customer journey data and try to measure the effectiveness of
each advertising channel. The inference about the influence of each channel
plays an important role in budget allocation and inventory pricing decisions.
Several simplistic rule-based strategies and data-driven algorithmic strategies
have been widely used in marketing field, but they do not address the issues,
such as channel interaction, time dependency, user characteristics. In this
paper, we propose a novel attribution algorithm based on deep learning to
assess the impact of each advertising channel. We present Deep Neural Net With
Attention multi-touch attribution model (DNAMTA) model in a supervised learning
fashion of predicting if a series of events leads to conversion, and it leads
us to have a deep understanding of the dynamic interaction effects between
media channels. DNAMTA also incorporates user-context information, such as user
demographics and behavior, as control variables to reduce the estimation biases
of media effects. We used computational experiment of large real world
marketing dataset to demonstrate that our proposed model is superior to
existing methods in both conversion prediction and media channel influence
evaluation.
Obstacle Detection Quality as a Problem-Oriented Approach to Stereo Vision Algorithms Estimation in Road Situation Analysis
Authors: A. A. Smagina, D. A. Shepelev, E. I. Ershov, A. S. Grigoryev
Date: 2018-09-06 00:00:00
Tasks: Self-Driving Cars

In this work we present a method for performance evaluation of stereo vision
based obstacle detection techniques that takes into account the specifics of
road situation analysis to minimize the effort required to prepare a test
dataset. This approach has been designed to be implemented in systems such as
self-driving cars or driver assistance and can also be used as problem-oriented
quality criterion for evaluation of stereo vision algorithms.
Content-based Propagation of User Markings for Interactive Segmentation of Patterned Images
Authors: Vedrana Andersen Dahl, Monica Jane Emerson, Camilla Himmelstrup Trinderup, Anders Bjorholm Dahl
Date: 2018-09-06 00:00:00
Tasks: Classification, Clustering, General Classification, Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

Efficient and easy segmentation of images and volumes is of great practical importance. Segmentation problems that motivate our approach originate from microscopy imaging commonly used in materials science, medicine, and biology. We formulate image segmentation as a probabilistic pixel classification problem, and we apply segmentation as a step towards characterising image content. Our method allows the user to define structures of interest by interactively marking a subset of pixels. Thanks to the real-time feedback, the user can place new markings strategically, depending on the current outcome. The final pixel classification may be obtained from a very modest user input. An important ingredient of our method is a graph that encodes image content. This graph is built in an unsupervised manner during initialisation and is based on clustering of image features. Since we combine a limited amount of user-labelled data with the clustering information obtained from the unlabelled parts of the image, our method fits in the general framework of semi-supervised learning. We demonstrate how this can be a very efficient approach to segmentation through pixel classification.
Character-Aware Decoder for Translation into Morphologically Rich Languages
Authors: Adithya Renduchintala, Pamela Shapiro, Kevin Duh, Philipp Koehn
Date: 2018-09-06 00:00:00
Proceeding: WS 2019 8
Tasks: Decoder, Machine Translation, NMT, Translation

Neural machine translation (NMT) systems operate primarily on words (or sub-words), ignoring lower-level patterns of morphology. We present a character-aware decoder designed to capture such patterns when translating into morphologically rich languages. We achieve character-awareness by augmenting both the softmax and embedding layers of an attention-based encoder-decoder model with convolutional neural networks that operate on the spelling of a word. To investigate performance on a wide variety of morphological phenomena, we translate English into 14 typologically diverse target languages using the TED multi-target dataset. In this low-resource setting, the character-aware decoder provides consistent improvements with BLEU score gains of up to $+3.05$. In addition, we analyze the relationship between the gains obtained and properties of the target language and find evidence that our model does indeed exploit morphological patterns.
2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy
Authors: Chuhan Min, Aosen Wang, Yiran Chen, Wenyao Xu, Xin Chen
Date: 2018-09-06 00:00:00
Tasks: Edge-computing, Neural Network Compression, Vocal Bursts Valence Prediction

Deep Convolutional Neural Networks~(CNNs) offer remarkable performance of
classifications and regressions in many high-dimensional problems and have been
widely utilized in real-word cognitive applications. However, high
computational cost of CNNs greatly hinder their deployment in
resource-constrained applications, real-time systems and edge computing
platforms. To overcome this challenge, we propose a novel filter-pruning
framework, two-phase filter pruning based on conditional entropy, namely
\textit{2PFPCE}, to compress the CNN models and reduce the inference time with
marginal performance degradation. In our proposed method, we formulate filter
pruning process as an optimization problem and propose a novel filter selection
criteria measured by conditional entropy. Based on the assumption that the
representation of neurons shall be evenly distributed, we also develop a
maximum-entropy filter freeze technique that can reduce over fitting. Two
filter pruning strategies -- global and layer-wise strategies, are compared.
Our experiment result shows that combining these two strategies can achieve a
higher neural network compression ratio than applying only one of them under
the same accuracy drop threshold. Two-phase pruning, that is, combining both
global and layer-wise strategies, achieves 10 X FLOPs reduction and 46%
inference time reduction on VGG-16, with 2% accuracy drop.
Dynamic Hierarchical Empirical Bayes: A Predictive Model Applied to Online Advertising
Authors: Yuan Yuan, Xiaojing Dong, Chen Dong, Yiwen Sun, Zhenyu Yan, Abhishek Pani
Date: 2018-09-06 00:00:00

Predicting keywords performance, such as number of impressions, click-through
rate (CTR), conversion rate (CVR), revenue per click (RPC), and cost per click
(CPC), is critical for sponsored search in the online advertising industry. An
interesting phenomenon is that, despite the size of the overall data, the data
are very sparse at the individual unit level. To overcome the sparsity and
leverage hierarchical information across the data structure, we propose a
Dynamic Hierarchical Empirical Bayesian (DHEB) model that dynamically
determines the hierarchy through a data-driven process and provides
shrinkage-based estimations. Our method is also equipped with an efficient
empirical approach to derive inferences through the hierarchy. We evaluate the
proposed method in both simulated and real-world datasets and compare to
several competitive models. The results favor the proposed method among all
comparisons in terms of both accuracy and efficiency. In the end, we design a
two-phase system to serve prediction in real time.
ProdSumNet: reducing model parameters in deep neural networks via product-of-sums matrix decompositions
Authors: Chai Wah Wu
Date: 2018-09-06 00:00:00

We consider a general framework for reducing the number of trainable model parameters in deep learning networks by decomposing linear operators as a product of sums of simpler linear operators. Recently proposed deep learning architectures such as CNN, KFC, Dilated CNN, etc. are all subsumed in this framework and we illustrate other types of neural network architectures within this framework. We show that good accuracy on MNIST and Fashion MNIST can be obtained using a relatively small number of trainable parameters. In addition, since implementation of the convolutional layer is resource-heavy, we consider an approach in the transform domain that obviates the need for convolutional layers. One of the advantages of this general framework over prior approaches is that the number of trainable parameters is not fixed and can be varied arbitrarily. In particular, we illustrate the tradeoff of varying the number of trainable variables and the corresponding error rate. As an example, by using this decomposition on a reference CNN architecture for MNIST with over 3x10^6 trainable parameters, we are able to obtain an accuracy of 98.44% using only 3554 trainable parameters.
Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate
Authors: Marcelo O. R. Prates, Pedro H. C. Avelar, Luis Lamb
Date: 2018-09-06 00:00:00
Tasks: Machine Translation, Translation

Recently there has been a growing concern about machine bias, where trained
statistical models grow to reflect controversial societal asymmetries, such as
gender or racial bias. A significant number of AI tools have recently been
suggested to be harmfully biased towards some minority, with reports of racist
criminal behavior predictors, Iphone X failing to differentiate between two
Asian people and Google photos' mistakenly classifying black people as
gorillas. Although a systematic study of such biases can be difficult, we
believe that automated translation tools can be exploited through gender
neutral languages to yield a window into the phenomenon of gender bias in AI.
  In this paper, we start with a comprehensive list of job positions from the
U.S. Bureau of Labor Statistics (BLS) and used it to build sentences in
constructions like "He/She is an Engineer" in 12 different gender neutral
languages such as Hungarian, Chinese, Yoruba, and several others. We translate
these sentences into English using the Google Translate API, and collect
statistics about the frequency of female, male and gender-neutral pronouns in
the translated output. We show that GT exhibits a strong tendency towards male
defaults, in particular for fields linked to unbalanced gender distribution
such as STEM jobs. We ran these statistics against BLS' data for the frequency
of female participation in each job position, showing that GT fails to
reproduce a real-world distribution of female workers. We provide experimental
evidence that even if one does not expect in principle a 50:50 pronominal
gender distribution, GT yields male defaults much more frequently than what
would be expected from demographic data alone.
  We are hopeful that this work will ignite a debate about the need to augment
current statistical translation tools with debiasing techniques which can
already be found in the scientific literature.
Bayesian Nonparametric Spectral Estimation
Authors: Felipe Tobar
Date: 2018-09-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Time Series, Time Series Analysis

Spectral estimation (SE) aims to identify how the energy of a signal (e.g., a
time series) is distributed across different frequencies. This can become
particularly challenging when only partial and noisy observations of the signal
are available, where current methods fail to handle uncertainty appropriately.
In this context, we propose a joint probabilistic model for signals,
observations and spectra, where SE is addressed as an exact inference problem.
Assuming a Gaussian process prior over the signal, we apply Bayes' rule to find
the analytic posterior distribution of the spectrum given a set of
observations. Besides its expressiveness and natural account of spectral
uncertainty, the proposed model also provides a functional-form representation
of the power spectral density, which can be optimised efficiently. Comparison
with previous approaches, in particular against Lomb-Scargle, is addressed
theoretically and also experimentally in three different scenarios. Code and
demo available at https://github.com/GAMES-UChile/BayesianSpectralEstimation.
Logical Rule Induction and Theory Learning Using Neural Theorem Proving
Authors: Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, Sebastian Riedel
Date: 2018-09-06 00:00:00
Tasks: Automated Theorem Proving

A hallmark of human cognition is the ability to continually acquire and
distill observations of the world into meaningful, predictive theories. In this
paper we present a new mechanism for logical theory acquisition which takes a
set of observed facts and learns to extract from them a set of logical rules
and a small set of core facts which together entail the observations. Our
approach is neuro-symbolic in the sense that the rule pred- icates and core
facts are given dense vector representations. The rules are applied to the core
facts using a soft unification procedure to infer additional facts. After k
steps of forward inference, the consequences are compared to the initial
observations and the rules and core facts are then encouraged towards
representations that more faithfully generate the observations through
inference. Our approach is based on a novel neural forward-chaining
differentiable rule induction network. The rules are interpretable and learned
compositionally from their predicates, which may be invented. We demonstrate
the efficacy of our approach on a variety of ILP rule induction and domain
theory learning datasets.