pwc-archive/papers-with-abstracts huggingface dataset:
Towards Automatic Generation of Entertaining Dialogues in Chinese Crosstalks
Authors: Shikang Du, Xiaojun Wan, Yajie Ye
Date: 2017-11-01 00:00:00
Tasks: Dialogue Generation, Translation

Crosstalk, also known by its Chinese name xiangsheng, is a traditional
Chinese comedic performing art featuring jokes and funny dialogues, and one of
China's most popular cultural elements. It is typically in the form of a
dialogue between two performers for the purpose of bringing laughter to the
audience, with one person acting as the leading comedian and the other as the
supporting role. Though general dialogue generation has been widely explored in
previous studies, it is unknown whether such entertaining dialogues can be
automatically generated or not. In this paper, we for the first time
investigate the possibility of automatic generation of entertaining dialogues
in Chinese crosstalks. Given the utterance of the leading comedian in each
dialogue, our task aims to generate the replying utterance of the supporting
role. We propose a humor-enhanced translation model to address this task and
human evaluation results demonstrate the efficacy of our proposed model. The
feasibility of automatic entertaining dialogue generation is also verified.
Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
Authors: Yiping Lu, Aoxiao Zhong, Quanzheng Li, Bin Dong
Date: 2017-10-27 00:00:00
Proceeding: ICML 2018 7

In our work, we bridge deep neural network design with numerical differential equations. We show that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations. This finding brings us a brand new perspective on the design of effective deep architectures. We can take advantage of the rich knowledge in numerical analysis to guide us in designing new and potentially more effective deep networks. As an example, we propose a linear multi-step architecture (LM-architecture) which is inspired by the linear multi-step method solving ordinary differential equations. The LM-architecture is an effective structure that can be used on any ResNet-like networks. In particular, we demonstrate that LM-ResNet and LM-ResNeXt (i.e. the networks obtained by applying the LM-architecture on ResNet and ResNeXt respectively) can achieve noticeably higher accuracy than ResNet and ResNeXt on both CIFAR and ImageNet with comparable numbers of trainable parameters. In particular, on both CIFAR and ImageNet, LM-ResNet/LM-ResNeXt can significantly compress ($>50$\%) the original networks while maintaining a similar performance. This can be explained mathematically using the concept of modified equation from numerical analysis. Last but not least, we also establish a connection between stochastic control and noise injection in the training process which helps to improve generalization of the networks. Furthermore, by relating stochastic training strategy with stochastic dynamic system, we can easily apply stochastic training to the networks with the LM-architecture. As an example, we introduced stochastic depth to LM-ResNet and achieve significant improvement over the original LM-ResNet on CIFAR10.
Openmv: A Python powered, extensible machine vision camera
Authors: Ibrahim Abdelkader, Yasser El-Sonbaty, Mohamed El-Habrouk
Date: 2017-11-01 00:00:00
Tasks: Edge-computing

Advances in semiconductor manufacturing processes and large scale integration
keep pushing demanding applications further away from centralized processing,
and closer to the edges of the network (i.e. Edge Computing). It has become
possible to perform complex in-network image processing using low-power
embedded smart cameras, enabling a multitude of new collaborative image
processing applications. This paper introduces OpenMV, a new low-power smart
camera that lends itself naturally to wireless sensor networks and machine
vision applications. The uniqueness of this platform lies in running an
embedded Python3 interpreter, allowing its peripherals and machine vision
library to be scripted in Python. In addition, its hardware is extensible via
modules that augment the platform with new capabilities, such as thermal
imaging and networking modules.
The Expressive Power of Neural Networks: A View from the Width
Authors: Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, Li-Wei Wang
Date: 2017-09-08 00:00:00
Proceeding: NeurIPS 2017 12

The expressive power of neural networks is important for understanding deep
learning. Most existing works consider this problem from the view of the depth
of a network. In this paper, we study how width affects the expressiveness of
neural networks. Classical results state that depth-bounded (e.g. depth-$2$)
networks with suitable activation functions are universal approximators. We
show a universal approximation theorem for width-bounded ReLU networks:
width-$(n+4)$ ReLU networks, where $n$ is the input dimension, are universal
approximators. Moreover, except for a measure zero set, all functions cannot be
approximated by width-$n$ ReLU networks, which exhibits a phase transition.
Several recent works demonstrate the benefits of depth by proving the
depth-efficiency of neural networks. That is, there are classes of deep
networks which cannot be realized by any shallow network whose size is no more
than an exponential bound. Here we pose the dual question on the
width-efficiency of ReLU networks: Are there wide networks that cannot be
realized by narrow networks whose size is not substantially larger? We show
that there exist classes of wide networks which cannot be realized by any
narrow network whose depth is no more than a polynomial bound. On the other
hand, we demonstrate by extensive experiments that narrow networks whose size
exceed the polynomial bound by a constant factor can approximate wide and
shallow network with high accuracy. Our results provide more comprehensive
evidence that depth is more effective than width for the expressiveness of ReLU
networks.
On- and Off-Policy Monotonic Policy Improvement
Authors: Ryo Iwaki, Minoru Asada
Date: 2017-10-10 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Monotonic policy improvement and off-policy learning are two main desirable
properties for reinforcement learning algorithms. In this paper, by lower
bounding the performance difference of two policies, we show that the monotonic
policy improvement is guaranteed from on- and off-policy mixture samples. An
optimization procedure which applies the proposed bound can be regarded as an
off-policy natural policy gradient method. In order to support the theoretical
result, we provide a trust region policy optimization method using experience
replay as a naive application of our bound, and evaluate its performance in two
classical benchmark problems.
Query-free Clothing Retrieval via Implicit Relevance Feedback
Authors: Zhuoxiang Chen, Zhe Xu, Ya zhang, Xiao Gu
Date: 2017-11-01 00:00:00
Tasks: Decision Making, Image Retrieval, Retrieval

Image-based clothing retrieval is receiving increasing interest with the
growth of online shopping. In practice, users may often have a desired piece of
clothing in mind (e.g., either having seen it before on the street or requiring
certain specific clothing attributes) but may be unable to supply an image as a
query. We model this problem as a new type of image retrieval task in which the
target image resides only in the user's mind (called "mental image retrieval"
hereafter). Because of the absence of an explicit query image, we propose to
solve this problem through relevance feedback. Specifically, a new Bayesian
formulation is proposed that simultaneously models the retrieval target and its
high-level representation in the mind of the user (called the "user metric"
hereafter) as posterior distributions of pre-fetched shop images and
heterogeneous features extracted from multiple clothing attributes,
respectively. Requiring only clicks as user feedback, the proposed algorithm is
able to account for the variability in human decision-making. Experiments with
real users demonstrate the effectiveness of the proposed algorithm.
Improved Text Language Identification for the South African Languages
Authors: Bernardt Duvenhage, Mfundo Ntini, Phala Ramonyai
Date: 2017-11-01 00:00:00
Tasks: Language Identification

Virtual assistants and text chatbots have recently been gaining popularity.
Given the short message nature of text-based chat interactions, the language
identification systems of these bots might only have 15 or 20 characters to
make a prediction. However, accurate text language identification is important,
especially in the early stages of many multilingual natural language processing
pipelines.
  This paper investigates the use of a naive Bayes classifier, to accurately
predict the language family that a piece of text belongs to, combined with a
lexicon based classifier to distinguish the specific South African language
that the text is written in. This approach leads to a 31% reduction in the
language detection error.
  In the spirit of reproducible research the training and testing datasets as
well as the code are published on github. Hopefully it will be useful to create
a text language identification shared task for South African languages.
Efficient Inferencing of Compressed Deep Neural Networks
Authors: Dharma Teja Vooturi, Saurabh Goyal, Anamitra R. Choudhury, Yogish Sabharwal, Ashish Verma
Date: 2017-11-01 00:00:00
Tasks: Quantization

Large number of weights in deep neural networks makes the models difficult to
be deployed in low memory environments such as, mobile phones, IOT edge devices
as well as "inferencing as a service" environments on cloud. Prior work has
considered reduction in the size of the models, through compression techniques
like pruning, quantization, Huffman encoding etc. However, efficient
inferencing using the compressed models has received little attention,
specially with the Huffman encoding in place. In this paper, we propose
efficient parallel algorithms for inferencing of single image and batches,
under various memory constraints. Our experimental results show that our
approach of using variable batch size for inferencing achieves 15-25\%
performance improvement in the inference throughput for AlexNet, while
maintaining memory and latency constraints.
Secure Classification With Augmented Features
Authors: Chenping Hou, Ling-Li Zeng, Dewen Hu
Date: 2017-11-01 00:00:00
Tasks: Classification, Diagnostic, General Classification

With the evolution of data collection ways, it is possible to produce
abundant data described by multiple feature sets. Previous studies show that
including more features does not necessarily bring positive effect. How to
prevent the augmented features worsening classification performance is crucial
but rarely studied. In this paper, we study this challenging problem by
proposing a secure classification approach, whose accuracy is never degenerated
when exploiting augmented features. We propose two ways to achieve the security
of our method named as SEcure Classification (SEC). Firstly, to leverage
augmented features, we learn various types of classifiers and adapt them by
employing a specially designed robust loss. It provides various candidate
classifiers to meet the following assumption of security operation. Secondly,
we integrate all candidate classifiers by approximately maximizing the
performance improvement. Under a mild assumption, the integrated classifier has
theoretical security guarantee. Several new optimization methods have been
developed to accommodate the problems with proved convergence. Besides
evaluating SEC on 16 data sets, we also apply SEC in the application of
diagnostic classification of schizophrenia since it has vast application
potentiality. Experimental results demonstrate the effectiveness of SEC in both
tackling security problem and discriminating schizophrenic patients from
healthy controls.
Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression
Authors: Haibin Yu, Trong Nghia Hoang, Kian Hsiang Low, Patrick Jaillet
Date: 2017-11-01 00:00:00
Tasks: GPR, regression, Stochastic Optimization, Variational Inference

This paper presents a novel variational inference framework for deriving a
family of Bayesian sparse Gaussian process regression (SGPR) models whose
approximations are variationally optimal with respect to the full-rank GPR
model enriched with various corresponding correlation structures of the
observation noises. Our variational Bayesian SGPR (VBSGPR) models jointly treat
both the distributions of the inducing variables and hyperparameters as
variational parameters, which enables the decomposability of the variational
lower bound that in turn can be exploited for stochastic optimization. Such a
stochastic optimization involves iteratively following the stochastic gradient
of the variational lower bound to improve its estimates of the optimal
variational distributions of the inducing variables and hyperparameters (and
hence the predictive distribution) of our VBSGPR models and is guaranteed to
achieve asymptotic convergence to them. We show that the stochastic gradient is
an unbiased estimator of the exact gradient and can be computed in constant
time per iteration, hence achieving scalability to big data. We empirically
evaluate the performance of our proposed framework on two real-world, massive
datasets.
Parallel Bayesian Global Optimization of Expensive Functions
Authors: Jialei Wang, Scott C. Clark, Eric Liu, Peter I. Frazier
Date: 2016-02-16 00:00:00
Tasks: Bayesian Optimization, global-optimization

We consider parallel global optimization of derivative-free expensive-to-evaluate functions, and propose an efficient method based on stochastic approximation for implementing a conceptual Bayesian optimization algorithm proposed by Ginsbourger et al. (2007). At the heart of this algorithm is maximizing the information criterion called the "multi-points expected improvement'', or the q-EI. To accomplish this, we use infinitessimal perturbation analysis (IPA) to construct a stochastic gradient estimator and show that this estimator is unbiased. We also show that the stochastic gradient ascent algorithm using the constructed gradient estimator converges to a stationary point of the q-EI surface, and therefore, as the number of multiple starts of the gradient ascent algorithm and the number of steps for each start grow large, the one-step Bayes optimal set of points is recovered. We show in numerical experiments that our method for maximizing the q-EI is faster than methods based on closed-form evaluation using high-dimensional integration, when considering many parallel function evaluations, and is comparable in speed when considering few. We also show that the resulting one-step Bayes optimal algorithm for parallel global optimization finds high-quality solutions with fewer evaluations than a heuristic based on approximately maximizing the q-EI. A high-quality open source implementation of this algorithm is available in the open source Metrics Optimization Engine (MOE).
Optimization for L1-Norm Error Fitting via Data Aggregation
Authors: Young Woong Park
Date: 2017-03-15 00:00:00
Tasks: regression

We propose a data aggregation-based algorithm with monotonic convergence to a global optimum for a generalized version of the L1-norm error fitting model with an assumption of the fitting function. The proposed algorithm generalizes the recent algorithm in the literature, aggregate and iterative disaggregate (AID), which selectively solves three specific L1-norm error fitting problems. With the proposed algorithm, any L1-norm error fitting model can be solved optimally if it follows the form of the L1-norm error fitting problem and if the fitting function satisfies the assumption. The proposed algorithm can also solve multi-dimensional fitting problems with arbitrary constraints on the fitting coefficients matrix. The generalized problem includes popular models such as regression and the orthogonal Procrustes problem. The results of the computational experiment show that the proposed algorithms are faster than the state-of-the-art benchmarks for L1-norm regression subset selection and L1-norm regression over a sphere. Further, the relative performance of the proposed algorithm improves as data size increases.
Stochastic Non-convex Optimization with Strong High Probability Second-order Convergence
Authors: Mingrui Liu, Tianbao Yang
Date: 2017-10-25 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In this paper, we study stochastic non-convex optimization with non-convex
random functions. Recent studies on non-convex optimization revolve around
establishing second-order convergence, i.e., converging to a nearly
second-order optimal stationary points. However, existing results on stochastic
non-convex optimization are limited, especially with a high probability
second-order convergence. We propose a novel updating step (named NCG-S) by
leveraging a stochastic gradient and a noisy negative curvature of a stochastic
Hessian, where the stochastic gradient and Hessian are based on a proper
mini-batch of random functions. Building on this step, we develop two
algorithms and establish their high probability second-order convergence. To
the best of our knowledge, the proposed stochastic algorithms are the first
with a second-order convergence in {\it high probability} and a time complexity
that is {\it almost linear} in the problem's dimensionality.
Learning deep features for source color laser printer identification based on cascaded learning
Authors: Do-Guk Kim, Jong-Uk Hou, Heung-Kyu Lee
Date: 2017-11-01 00:00:00

Color laser printers have fast printing speed and high resolution, and
forgeries using color laser printers can cause significant harm to society. A
source printer identification technique can be employed as a countermeasure to
those forgeries. This paper presents a color laser printer identification
method based on cascaded learning of deep neural networks. The refiner network
is trained by adversarial training to refine the synthetic dataset for halftone
color decomposition. The halftone color decomposing ConvNet is trained with the
refined dataset, and the trained knowledge is transferred to the printer
identifying ConvNet to enhance the accuracy. The robustness about rotation and
scaling is considered in training process, which is not considered in existing
methods. Experiments are performed on eight color laser printers, and the
performance is compared with several existing methods. The experimental results
clearly show that the proposed method outperforms existing source color laser
printer identification methods.
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Authors: Nicholas Carlini, David Wagner
Date: 2017-05-20 00:00:00

Neural networks are known to be vulnerable to adversarial examples: inputs
that are close to natural inputs but classified incorrectly. In order to better
understand the space of adversarial examples, we survey ten recent proposals
that are designed for detection and compare their efficacy. We show that all
can be defeated by constructing new loss functions. We conclude that
adversarial examples are significantly harder to detect than previously
appreciated, and the properties believed to be intrinsic to adversarial
examples are in fact not. Finally, we propose several simple guidelines for
evaluating future proposed defenses.
Convergence of Langevin MCMC in KL-divergence
Authors: Xiang Cheng, Peter Bartlett
Date: 2017-05-25 00:00:00

Langevin diffusion is a commonly used tool for sampling from a given
distribution. In this work, we establish that when the target density $p^*$ is
such that $\log p^*$ is $L$ smooth and $m$ strongly convex, discrete Langevin
diffusion produces a distribution $p$ with $KL(p||p^*)\leq \epsilon$ in
$\tilde{O}(\frac{d}{\epsilon})$ steps, where $d$ is the dimension of the sample
space. We also study the convergence rate when the strong-convexity assumption
is absent. By considering the Langevin diffusion as a gradient flow in the
space of probability distributions, we obtain an elegant analysis that applies
to the stronger property of convergence in KL-divergence and gives a
conceptually simpler proof of the best-known convergence results in weaker
metrics.
Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification
Authors: Jonggu Kim, Jong-Hyeok Lee
Date: 2017-07-05 00:00:00
Tasks: Classification, General Classification, Relation, Relation Classification, Sentence

Most of neural approaches to relation classification have focused on finding
short patterns that represent the semantic relation using Convolutional Neural
Networks (CNNs) and those approaches have generally achieved better
performances than using Recurrent Neural Networks (RNNs). In a similar
intuition to the CNN models, we propose a novel RNN-based model that strongly
focuses on only important parts of a sentence using multiple range-restricted
bidirectional layers and attention for relation classification. Experimental
results on the SemEval-2010 relation classification task show that our model is
comparable to the state-of-the-art CNN-based and RNN-based models that use
additional linguistic information.
Target Curricula via Selection of Minimum Feature Sets: a Case Study in Boolean Networks
Authors: Shannon Fenn, Pablo Moscato
Date: 2017-06-15 00:00:00
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We consider the effect of introducing a curriculum of targets when training
Boolean models on supervised Multi Label Classification (MLC) problems. In
particular, we consider how to order targets in the absence of prior knowledge,
and how such a curriculum may be enforced when using meta-heuristics to train
discrete non-linear models.
  We show that hierarchical dependencies between targets can be exploited by
enforcing an appropriate curriculum using hierarchical loss functions. On
several multi output circuit-inference problems with known target difficulties,
Feedforward Boolean Networks (FBNs) trained with such a loss function achieve
significantly lower out-of-sample error, up to $10\%$ in some cases. This
improvement increases as the loss places more emphasis on target order and is
strongly correlated with an easy-to-hard curricula. We also demonstrate the
same improvements on three real-world models and two Gene Regulatory Network
(GRN) inference problems.
  We posit a simple a-priori method for identifying an appropriate target order
and estimating the strength of target relationships in Boolean MLCs. These
methods use intrinsic dimension as a proxy for target difficulty, which is
estimated using optimal solutions to a combinatorial optimisation problem known
as the Minimum-Feature-Set (minFS) problem. We also demonstrate that the same
generalisation gains can be achieved without providing any knowledge of target
difficulty.