pwc-archive/papers-with-abstracts huggingface dataset:
A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
Authors: Yu Wu, Wei Wu, Chen Xing, Can Xu, Zhoujun Li, Ming Zhou
Date: 2017-10-31 00:00:00
Proceeding: CL 2019 3
Tasks: Retrieval

We study the problem of response selection for multi-turn conversation in
retrieval-based chatbots. The task requires matching a response candidate with
a conversation context, whose challenges include how to recognize important
parts of the context, and how to model the relationships among utterances in
the context. Existing matching methods may lose important information in
contexts as we can interpret them with a unified framework in which contexts
are transformed to fixed-length vectors without any interaction with responses
before matching. The analysis motivates us to propose a new matching framework
that can sufficiently carry the important information in contexts to matching
and model the relationships among utterances at the same time. The new
framework, which we call a sequential matching framework (SMF), lets each
utterance in a context interacts with a response candidate at the first step
and transforms the pair to a matching vector. The matching vectors are then
accumulated following the order of the utterances in the context with a
recurrent neural network (RNN) which models the relationships among the
utterances. The context-response matching is finally calculated with the hidden
states of the RNN. Under SMF, we propose a sequential convolutional network and
sequential attention network and conduct experiments on two public data sets to
test their performance. Experimental results show that both models can
significantly outperform the state-of-the-art matching methods. We also show
that the models are interpretable with visualizations that provide us insights
on how they capture and leverage the important information in contexts for
matching.
Shallow Discourse Parsing with Maximum Entropy Model
Authors: Jingjing Xu
Date: 2017-10-31 00:00:00
Tasks: Discourse Parsing, model

In recent years, more research has been devoted to studying the subtask of
the complete shallow discourse parsing, such as indentifying discourse
connective and arguments of connective. There is a need to design a full
discourse parser to pull these subtasks together. So we develop a discourse
parser turning the free text into discourse relations. The parser includes
connective identifier, arguments identifier, sense classifier and non-explicit
identifier, which connects with each other in pipeline. Each component applies
the maximum entropy model with abundant lexical and syntax features extracted
from the Penn Discourse Tree-bank. The head-based representation of the PDTB is
adopted in the arguments identifier, which turns the problem of indentifying
the arguments of discourse connective into finding the head and end of the
arguments. In the non-explicit identifier, the contextual type features like
words which have high frequency and can reflect the discourse relation are
introduced to improve the performance of non-explicit identifier. Compared with
other methods, experimental results achieve the considerable performance.
Improving Social Media Text Summarization by Learning Sentence Weight Distribution
Authors: Jingjing Xu
Date: 2017-10-31 00:00:00
Tasks: Decoder, Sentence, Text Summarization

Recently, encoder-decoder models are widely used in social media text
summarization. However, these models sometimes select noise words in irrelevant
sentences as part of a summary by error, thus declining the performance. In
order to inhibit irrelevant sentences and focus on key information, we propose
an effective approach by learning sentence weight distribution. In our model,
we build a multi-layer perceptron to predict sentence weights. During training,
we use the ROUGE score as an alternative to the estimated sentence weight, and
try to minimize the gap between estimated weights and predicted weights. In
this way, we encourage our model to focus on the key sentences, which have high
relevance with the summary. Experimental results show that our approach
outperforms baselines on a large-scale social media corpus.
Ternary Residual Networks
Authors: Abhisek Kundu, Kunal Banerjee, Naveen Mellempudi, Dheevatsa Mudigere, Dipankar Das, Bharat Kaul, Pradeep Dubey
Date: 2017-07-15 00:00:00

Sub-8-bit representation of DNNs incur some discernible loss of accuracy
despite rigorous (re)training at low-precision. Such loss of accuracy
essentially makes them equivalent to a much shallower counterpart, diminishing
the power of being deep networks. To address this problem of accuracy drop we
introduce the notion of \textit{residual networks} where we add more
low-precision edges to sensitive branches of the sub-8-bit network to
compensate for the lost accuracy. Further, we present a perturbation theory to
identify such sensitive edges. Aided by such an elegant trade-off between
accuracy and compute, the 8-2 model (8-bit activations, ternary weights),
enhanced by ternary residual edges, turns out to be sophisticated enough to
achieve very high accuracy ($\sim 1\%$ drop from our FP-32 baseline), despite
$\sim 1.6\times$ reduction in model size, $\sim 26\times$ reduction in number
of multiplications, and potentially $\sim 2\times$ power-performance gain
comparing to 8-8 representation, on the state-of-the-art deep network
ResNet-101 pre-trained on ImageNet dataset. Moreover, depending on the varying
accuracy requirements in a dynamic environment, the deployed low-precision
model can be upgraded/downgraded on-the-fly by partially enabling/disabling
residual connections. For example, disabling the least important residual
connections in the above enhanced network, the accuracy drop is $\sim 2\%$
(from FP32), despite $\sim 1.9\times$ reduction in model size, $\sim 32\times$
reduction in number of multiplications, and potentially $\sim 2.3\times$
power-performance gain comparing to 8-8 representation. Finally, all the
ternary connections are sparse in nature, and the ternary residual conversion
can be done in a resource-constraint setting with no low-precision
(re)training.
Servant of Many Masters: Shifting priorities in Pareto-optimal sequential decision-making
Authors: Andrew Critch, Stuart Russell
Date: 2017-10-31 00:00:00
Tasks: Decision Making, Sequential Decision Making

It is often argued that an agent making decisions on behalf of two or more
principals who have different utility functions should adopt a {\em
Pareto-optimal} policy, i.e., a policy that cannot be improved upon for one
agent without making sacrifices for another. A famous theorem of Harsanyi shows
that, when the principals have a common prior on the outcome distributions of
all policies, a Pareto-optimal policy for the agent is one that maximizes a
fixed, weighted linear combination of the principals' utilities.
  In this paper, we show that Harsanyi's theorem does not hold for principals
with different priors, and derive a more precise generalization which does
hold, which constitutes our main result. In this more general case, the
relative weight given to each principal's utility should evolve over time
according to how well the agent's observations conform with that principal's
prior. The result has implications for the design of contracts, treaties, joint
ventures, and robots.
Differentially Private Testing of Identity and Closeness of Discrete Distributions
Authors: Jayadev Acharya, Ziteng Sun, Huanyu Zhang
Date: 2017-07-17 00:00:00
Proceeding: NeurIPS 2018 12

We study the fundamental problems of identity testing (goodness of fit), and
closeness testing (two sample test) of distributions over $k$ elements, under
differential privacy. While the problems have a long history in statistics,
finite sample bounds for these problems have only been established recently.
  In this work, we derive upper and lower bounds on the sample complexity of
both the problems under $(\varepsilon, \delta)$-differential privacy. We
provide optimal sample complexity algorithms for identity testing problem for
all parameter ranges, and the first results for closeness testing. Our
closeness testing bounds are optimal in the sparse regime where the number of
samples is at most $k$.
  Our upper bounds are obtained by privatizing non-private estimators for these
problems. The non-private estimators are chosen to have small sensitivity. We
propose a general framework to establish lower bounds on the sample complexity
of statistical tasks under differential privacy. We show a bound on
differentially private algorithms in terms of a coupling between the two
hypothesis classes we aim to test. By constructing carefully chosen priors over
the hypothesis classes, and using Le Cam's two point theorem we provide a
general mechanism for proving lower bounds. We believe that the framework can
be used to obtain strong lower bounds for other statistical tasks under
privacy.
Rate-optimal Meta Learning of Classification Error
Authors: Morteza Noshad Iranzad, Alfred O. Hero III
Date: 2017-10-31 00:00:00
Tasks: Classification, Density Estimation, General Classification, Meta-Learning

Meta learning of optimal classifier error rates allows an experimenter to
empirically estimate the intrinsic ability of any estimator to discriminate
between two populations, circumventing the difficult problem of estimating the
optimal Bayes classifier. To this end we propose a weighted nearest neighbor
(WNN) graph estimator for a tight bound on the Bayes classification error; the
Henze-Penrose (HP) divergence. Similar to recently proposed HP estimators
[berisha2016], the proposed estimator is non-parametric and does not require
density estimation. However, unlike previous approaches the proposed estimator
is rate-optimal, i.e., its mean squared estimation error (MSEE) decays to zero
at the fastest possible rate of $O(1/M+1/N)$ where $M,N$ are the sample sizes
of the respective populations. We illustrate the proposed WNN meta estimator
for several simulated and real data sets.
Tumor Classification and Segmentation of MR Brain Images
Authors: Tanvi Gupta, Pranay Manocha, Tapan K. Gandhi, RK Gupta, BK Panigrahi
Date: 2017-10-31 00:00:00
Tasks: Classification, Diagnostic, General Classification, Segmentation, Specificity

The diagnosis and segmentation of tumors using any medical diagnostic tool
can be challenging due to the varying nature of this pathology. Magnetic Reso-
nance Imaging (MRI) is an established diagnostic tool for various diseases and
disorders and plays a major role in clinical neuro-diagnosis. Supplementing
this technique with automated classification and segmentation tools is gaining
importance, to reduce errors and time needed to make a conclusive diagnosis. In
this paper a simple three-step algorithm is proposed; (1) identification of
patients that present with tumors, (2) automatic selection of abnormal slices
of the patients, and (3) segmentation and detection of the tumor. Features were
extracted by using discrete wavelet transform on the normalized images and
classified by support vector machine (for step (1)) and random forest (for step
(2)). The 400 subjects were divided in a 3:1 ratio between training and test
with no overlap. This study is novel in terms of use of data, as it employed
the entire T2 weighted slices as a single image for classification and a unique
combination of contralateral approach with patch thresholding for segmentation,
which does not require a training set or a template as is used by most
segmentation studies. Using the proposed method, the tumors were segmented
accurately with a classification accuracy of 95% with 100% specificity and 90%
sensitivity.
The Exact Solution to Rank-1 L1-norm TUCKER2 Decomposition
Authors: Panos P. Markopoulos, Dimitris G. Chachlakis, Evangelos E. Papalexakis
Date: 2017-10-31 00:00:00
Tasks: Combinatorial Optimization

We study rank-1 {L1-norm-based TUCKER2} (L1-TUCKER2) decomposition of 3-way
tensors, treated as a collection of $N$ $D \times M$ matrices that are to be
jointly decomposed. Our contributions are as follows. i) We prove that the
problem is equivalent to combinatorial optimization over $N$ antipodal-binary
variables. ii) We derive the first two algorithms in the literature for its
exact solution. The first algorithm has cost exponential in $N$; the second one
has cost polynomial in $N$ (under a mild assumption). Our algorithms are
accompanied by formal complexity analysis. iii) We conduct numerical studies to
compare the performance of exact L1-TUCKER2 (proposed) with standard HOSVD,
HOOI, GLRAM, PCA, L1-PCA, and TPCA-L1. Our studies show that L1-TUCKER2
outperforms (in tensor approximation) all the above counterparts when the
processed data are outlier corrupted.
Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes
Authors: Taylor Killian, Samuel Daulton, George Konidaris, Finale Doshi-Velez
Date: 2017-06-20 00:00:00
Tasks: Transfer Learning

We introduce a new formulation of the Hidden Parameter Markov Decision
Process (HiP-MDP), a framework for modeling families of related tasks using
low-dimensional latent embeddings. Our new framework correctly models the joint
uncertainty in the latent parameters and the state space. We also replace the
original Gaussian Process-based model with a Bayesian Neural Network, enabling
more scalable inference. Thus, we expand the scope of the HiP-MDP to
applications with higher dimensions and more complex dynamics.
Which Distribution Distances are Sublinearly Testable?
Authors: Constantinos Daskalakis, Gautam Kamath, John Wright
Date: 2017-07-31 00:00:00

Given samples from an unknown distribution $p$ and a description of a
distribution $q$, are $p$ and $q$ close or far? This question of "identity
testing" has received significant attention in the case of testing whether $p$
and $q$ are equal or far in total variation distance. However, in recent work,
the following questions have been been critical to solving problems at the
frontiers of distribution testing:
  -Alternative Distances: Can we test whether $p$ and $q$ are far in other
distances, say Hellinger?
  -Tolerance: Can we test when $p$ and $q$ are close, rather than equal? And if
so, close in which distances?
  Motivated by these questions, we characterize the complexity of distribution
testing under a variety of distances, including total variation, $\ell_2$,
Hellinger, Kullback-Leibler, and $\chi^2$. For each pair of distances $d_1$ and
$d_2$, we study the complexity of testing if $p$ and $q$ are close in $d_1$
versus far in $d_2$, with a focus on identifying which problems allow strongly
sublinear testers (i.e., those with complexity $O(n^{1 - \gamma})$ for some
$\gamma > 0$ where $n$ is the size of the support of the distributions $p$ and
$q$). We provide matching upper and lower bounds for each case. We also study
these questions in the case where we only have samples from $q$ (equivalence
testing), showing qualitative differences from identity testing in terms of
when tolerance can be achieved. Our algorithms fall into the classical paradigm
of $\chi^2$-statistics, but require crucial changes to handle the challenges
introduced by each distance we consider. Finally, we survey other recent
results in an attempt to serve as a reference for the complexity of various
distribution testing problems.
Characterizing the structural diversity of complex networks across domains
Authors: Kansuke Ikehara, Aaron Clauset
Date: 2017-10-31 00:00:00
Tasks: Diversity

The structure of complex networks has been of interest in many scientific and
engineering disciplines over the decades. A number of studies in the field have
been focused on finding the common properties among different kinds of networks
such as heavy-tail degree distribution, small-worldness and modular structure
and they have tried to establish a theory of structural universality in complex
networks. However, there is no comprehensive study of network structure across
a diverse set of domains in order to explain the structural diversity we
observe in the real-world networks. In this paper, we study 986 real-world
networks of diverse domains ranging from ecological food webs to online social
networks along with 575 networks generated from four popular network models.
Our study utilizes a number of machine learning techniques such as random
forest and confusion matrix in order to show the relationships among network
domains in terms of network structure. Our results indicate that there are some
partitions of network categories in which networks are hard to distinguish
based purely on network structure. We have found that these partitions of
network categories tend to have similar underlying functions, constraints
and/or generative mechanisms of networks even though networks in the same
partition have different origins, e.g., biological processes, results of
engineering by human being, etc. This suggests that the origin of a network,
whether it's biological, technological or social, may not necessarily be a
decisive factor of the formation of similar network structure. Our findings
shed light on the possible direction along which we could uncover the hidden
principles for the structural diversity of complex networks.
meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting
Authors: Xu Sun, Xuancheng Ren, Shuming Ma, Houfeng Wang
Date: 2017-06-19 00:00:00
Proceeding: ICML 2017 8
Tasks: Deep Learning

We propose a simple yet effective technique for neural network learning. The
forward propagation is computed as usual. In back propagation, only a small
subset of the full gradient is computed to update the model parameters. The
gradient vectors are sparsified in such a way that only the top-$k$ elements
(in terms of magnitude) are kept. As a result, only $k$ rows or columns
(depending on the layout) of the weight matrix are modified, leading to a
linear reduction ($k$ divided by the vector dimension) in the computational
cost. Surprisingly, experimental results demonstrate that we can update only
1-4% of the weights at each back propagation pass. This does not result in a
larger number of training iterations. More interestingly, the accuracy of the
resulting models is actually improved rather than degraded, and a detailed
analysis is given. The code is available at https://github.com/lancopku/meProp
Evolving Deep Convolutional Neural Networks for Image Classification
Authors: Yanan Sun, Bing Xue, Mengjie Zhang, Gary G. Yen
Date: 2017-10-30 00:00:00
Tasks: Classification, General Classification, Heuristic Search, image-classification, Image Classification

Evolutionary computation methods have been successfully applied to neural
networks since two decades ago, while those methods cannot scale well to the
modern deep neural networks due to the complicated architectures and large
quantities of connection weights. In this paper, we propose a new method using
genetic algorithms for evolving the architectures and connection weight
initialization values of a deep convolutional neural network to address image
classification problems. In the proposed algorithm, an efficient
variable-length gene encoding strategy is designed to represent the different
building blocks and the unpredictable optimal depth in convolutional neural
networks. In addition, a new representation scheme is developed for effectively
initializing connection weights of deep convolutional neural networks, which is
expected to avoid networks getting stuck into local minima which is typically a
major issue in the backward gradient-based optimization. Furthermore, a novel
fitness evaluation method is proposed to speed up the heuristic search with
substantially less computational resource. The proposed algorithm is examined
and compared with 22 existing algorithms on nine widely used image
classification tasks, including the state-of-the-art methods. The experimental
results demonstrate the remarkable superiority of the proposed algorithm over
the state-of-the-art algorithms in terms of classification error rate and the
number of parameters (weights).
Efficient Gender Classification Using a Deep LDA-Pruned Net
Authors: Qing Tian, Tal Arbel, James J. Clark
Date: 2017-04-20 00:00:00
Tasks: Classification, Gender Classification, General Classification, GPU

Many real-time tasks, such as human-computer interaction, require fast and
efficient facial gender classification. Although deep CNN nets have been very
effective for a multitude of classification tasks, their high space and time
demands make them impractical for personal computers and mobile devices without
a powerful GPU. In this paper, we develop a 16-layer, yet lightweight, neural
network which boosts efficiency while maintaining high accuracy. Our net is
pruned from the VGG-16 model starting from the last convolutional (conv) layer
where we find neuron activations are highly uncorrelated given the gender.
Through Fisher's Linear Discriminant Analysis (LDA), we show that this high
decorrelation makes it safe to discard directly last conv layer neurons with
high within-class variance and low between-class variance. Combined with either
Support Vector Machines (SVM) or Bayesian classification, the reduced CNNs are
capable of achieving comparable (or even higher) accuracies on the LFW and
CelebA datasets than the original net with fully connected layers. On LFW, only
four Conv5_3 neurons are able to maintain a comparably high recognition
accuracy, which results in a reduction of total network size by a factor of 70X
with a 11 fold speedup. Comparisons with a state-of-the-art pruning method as
well as two smaller nets in terms of accuracy loss and convolutional layers
pruning rate are also provided.
Empirical analysis of non-linear activation functions for Deep Neural Networks in classification tasks
Authors: Giovanni Alcantara
Date: 2017-10-30 00:00:00
Tasks: General Classification

We provide an overview of several non-linear activation functions in a neural
network architecture that have proven successful in many machine learning
applications. We conduct an empirical analysis on the effectiveness of using
these function on the MNIST classification task, with the aim of clarifying
which functions produce the best results overall. Based on this first set of
results, we examine the effects of building deeper architectures with an
increasing number of hidden layers. We also survey the impact of using, on the
same task, different initialisation schemes for the weights of our neural
network. Using these sets of experiments as a base, we conclude by providing a
optimal neural network architecture that yields impressive results in accuracy
on the MNIST classification task.
Implicit Manifold Learning on Generative Adversarial Networks
Authors: Kry Yik Chau Lui, Yanshuai Cao, Maxime Gazeau, Kelvin Shuangjian Zhang
Date: 2017-10-30 00:00:00

This paper raises an implicit manifold learning perspective in Generative
Adversarial Networks (GANs), by studying how the support of the learned
distribution, modelled as a submanifold $\mathcal{M}_{\theta}$, perfectly match
with $\mathcal{M}_{r}$, the support of the real data distribution. We show that
optimizing Jensen-Shannon divergence forces $\mathcal{M}_{\theta}$ to perfectly
match with $\mathcal{M}_{r}$, while optimizing Wasserstein distance does not.
On the other hand, by comparing the gradients of the Jensen-Shannon divergence
and the Wasserstein distances ($W_1$ and $W_2^2$) in their primal forms, we
conjecture that Wasserstein $W_2^2$ may enjoy desirable properties such as
reduced mode collapse. It is therefore interesting to design new distances that
inherit the best from both distances.