pwc-archive/papers-with-abstracts huggingface dataset:
Distilling Knowledge Using Parallel Data for Far-field Speech Recognition
Authors: Jiangyan Yi, Jian-Hua Tao, Zhengqi Wen, Bin Liu
Date: 2018-02-20 00:00:00
Tasks: speech-recognition, Speech Recognition

In order to improve the performance for far-field speech recognition, this
paper proposes to distill knowledge from the close-talking model to the
far-field model using parallel data. The close-talking model is called the
teacher model. The far-field model is called the student model. The student
model is trained to imitate the output distributions of the teacher model. This
constraint can be realized by minimizing the Kullback-Leibler (KL) divergence
between the output distribution of the student model and the teacher model.
Experimental results on AMI corpus show that the best student model achieves up
to 4.7% absolute word error rate (WER) reduction when compared with the
conventionally-trained baseline models.
Using Automatic Generation of Relaxation Constraints to Improve the Preimage Attack on 39-step MD4
Authors: Gribanova Irina, Semenov Alexander
Date: 2018-02-20 00:00:00

In this paper we construct preimage attack on the truncated variant of the
MD4 hash function. Specifically, we study the MD4-39 function defined by the
first 39 steps of the MD4 algorithm. We suggest a new attack on MD4-39, which
develops the ideas proposed by H. Dobbertin in 1998. Namely, the special
relaxation constraints are introduced in order to simplify the equations
corresponding to the problem of finding a preimage for an arbitrary MD4-39 hash
value. The equations supplemented with the relaxation constraints are then
reduced to the Boolean Satisfiability Problem (SAT) and solved using the
state-of-the-art SAT solvers. We show that the effectiveness of a set of
relaxation constraints can be evaluated using the black-box function of a
special kind. Thus, we suggest automatic method of relaxation constraints
generation by applying the black-box optimization to this function. The
proposed method made it possible to find new relaxation constraints that
contribute to a SAT-based preimage attack on MD4-39 which significantly
outperforms the competition.
Online Learning with an Unknown Fairness Metric
Authors: Stephen Gillen, Christopher Jung, Michael Kearns, Aaron Roth
Date: 2018-02-20 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Fairness, Multi-Armed Bandits

We consider the problem of online learning in the linear contextual bandits
setting, but in which there are also strong individual fairness constraints
governed by an unknown similarity metric. These constraints demand that we
select similar actions or individuals with approximately equal probability
(arXiv:1104.3913), which may be at odds with optimizing reward, thus modeling
settings where profit and social policy are in tension. We assume we learn
about an unknown Mahalanobis similarity metric from only weak feedback that
identifies fairness violations, but does not quantify their extent. This is
intended to represent the interventions of a regulator who "knows unfairness
when he sees it" but nevertheless cannot enunciate a quantitative fairness
metric over individuals. Our main result is an algorithm in the adversarial
context setting that has a number of fairness violations that depends only
logarithmically on $T$, while obtaining an optimal $O(\sqrt{T})$ regret bound
to the best fair policy.
Non-Local Graph-Based Prediction For Reversible Data Hiding In Images
Authors: Qi Chang, Gene Cheung, Yao Zhao, Xiaolong Li, Rongrong Ni
Date: 2018-02-20 00:00:00
Tasks: Prediction

Reversible data hiding (RDH) is desirable in applications where both the
hidden message and the cover medium need to be recovered without loss. Among
many RDH approaches is prediction-error expansion (PEE), containing two steps:
i) prediction of a target pixel value, and ii) embedding according to the value
of prediction-error. In general, higher prediction performance leads to larger
embedding capacity and/or lower signal distortion. Leveraging on recent
advances in graph signal processing (GSP), we pose pixel prediction as a
graph-signal restoration problem, where the appropriate edge weights of the
underlying graph are computed using a similar patch searched in a semi-local
neighborhood. Specifically, for each candidate patch, we first examine
eigenvalues of its structure tensor to estimate its local smoothness. If
sufficiently smooth, we pose a maximum a posteriori (MAP) problem using either
a quadratic Laplacian regularizer or a graph total variation (GTV) term as
signal prior. While the MAP problem using the first prior has a closed-form
solution, we design an efficient algorithm for the second prior using
alternating direction method of multipliers (ADMM) with nested proximal
gradient descent. Experimental results show that with better quality GSP-based
prediction, at low capacity the visual quality of the embedded image exceeds
state-of-the-art methods noticeably.
Unsupervised Document Embedding With CNNs
Authors: Chundi Liu, Shunan Zhao, Maksims Volkovs
Date: 2017-11-11 00:00:00
Tasks: Document Embedding

We propose a new model for unsupervised document embedding. Leading existing
approaches either require complex inference or use recurrent neural networks
(RNN) that are difficult to parallelize. We take a different route and develop
a convolutional neural network (CNN) embedding model. Our CNN architecture is
fully parallelizable resulting in over 10x speedup in inference time over RNN
models. Parallelizable architecture enables to train deeper models where each
successive layer has increasingly larger receptive field and models longer
range semantic structure within the document. We additionally propose a fully
unsupervised learning algorithm to train this model based on stochastic forward
prediction. Empirical results on two public benchmarks show that our approach
produces comparable to state-of-the-art accuracy at a fraction of computational
cost.
On Lyapunov exponents and adversarial perturbation
Authors: Vinay Uday Prabhu, Nishant Desai, John Whaley
Date: 2018-02-20 00:00:00
Tasks: Time Series, Time Series Analysis

In this paper, we would like to disseminate a serendipitous discovery
involving Lyapunov exponents of a 1-D time series and their use in serving as a
filtering defense tool against a specific kind of deep adversarial
perturbation. To this end, we use the state-of-the-art CleverHans library to
generate adversarial perturbations against a standard Convolutional Neural
Network (CNN) architecture trained on the MNIST as well as the Fashion-MNIST
datasets. We empirically demonstrate how the Lyapunov exponents computed on the
flattened 1-D vector representations of the images served as highly
discriminative features that could be to pre-classify images as adversarial or
legitimate before feeding the image into the CNN for classification. We also
explore the issue of possible false-alarms when the input images are noisy in a
non-adversarial sense.
Scale Optimization for Full-Image-CNN Vehicle Detection
Authors: Yang Gao, Shouyan Guo, Kaimin Huang, Jiaxin Chen, Qian Gong, Yang Zou, Tong Bai, Gary Overett
Date: 2018-02-20 00:00:00
Tasks: Object, object-detection, Object Detection, Region Proposal, vehicle detection

Many state-of-the-art general object detection methods make use of shared
full-image convolutional features (as in Faster R-CNN). This achieves a
reasonable test-phase computation time while enjoys the discriminative power
provided by large Convolutional Neural Network (CNN) models. Such designs excel
on benchmarks which contain natural images but which have very unnatural
distributions, i.e. they have an unnaturally high-frequency of the target
classes and a bias towards a "friendly" or "dominant" object scale. In this
paper we present further study of the use and adaptation of the Faster R-CNN
object detection method for datasets presenting natural scale distribution and
unbiased real-world object frequency. In particular, we show that better
alignment of the detector scale sensitivity to the extant distribution improves
vehicle detection performance. We do this by modifying both the selection of
Region Proposals, and through using more scale-appropriate full-image
convolution features within the CNN model. By selecting better scales in the
region proposal input and by combining feature maps through careful design of
the convolutional neural network, we improve performance on smaller objects. We
significantly increase detection AP for the KITTI dataset car class from 76.3%
on our baseline Faster R-CNN detector to 83.6% in our improved detector.
Teaching Categories to Human Learners with Visual Explanations
Authors: Oisin Mac Aodha, Shih-An Su, Yuxin Chen, Pietro Perona, Yisong Yue
Date: 2018-02-20 00:00:00
Proceeding: CVPR 2018 6

We study the problem of computer-assisted teaching with explanations.
Conventional approaches for machine teaching typically only provide feedback at
the instance level e.g., the category or label of the instance. However, it is
intuitive that clear explanations from a knowledgeable teacher can
significantly improve a student's ability to learn a new concept. To address
these existing limitations, we propose a teaching framework that provides
interpretable explanations as feedback and models how the learner incorporates
this additional information. In the case of images, we show that we can
automatically generate explanations that highlight the parts of the image that
are responsible for the class label. Experiments on human learners illustrate
that, on average, participants achieve better test set performance on
challenging categorization tasks when taught with our interpretable approach
compared to existing methods.
Mixed Membership Word Embeddings for Computational Social Science
Authors: James Foulds
Date: 2017-05-20 00:00:00
Tasks: Articles, Language Modeling, Language Modelling, Topic Models, Word Embeddings

Word embeddings improve the performance of NLP systems by revealing the
hidden structural relationships between words. Despite their success in many
applications, word embeddings have seen very little use in computational social
science NLP tasks, presumably due to their reliance on big data, and to a lack
of interpretability. I propose a probabilistic model-based word embedding
method which can recover interpretable embeddings, without big data. The key
insight is to leverage mixed membership modeling, in which global
representations are shared, but individual entities (i.e. dictionary words) are
free to use these representations to uniquely differing degrees. I show how to
train the model using a combination of state-of-the-art training techniques for
word embeddings and topic models. The experimental results show an improvement
in predictive language modeling of up to 63% in MRR over the skip-gram, and
demonstrate that the representations are beneficial for supervised learning. I
illustrate the interpretability of the models with computational social science
case studies on State of the Union addresses and NIPS articles.
Layer-wise synapse optimization for implementing neural networks on general neuromorphic architectures
Authors: John Mern, Jayesh K. Gupta, Mykel Kochenderfer
Date: 2018-02-20 00:00:00
Tasks: CPU

Deep artificial neural networks (ANNs) can represent a wide range of complex
functions. Implementing ANNs in Von Neumann computing systems, though, incurs a
high energy cost due to the bottleneck created between CPU and memory.
Implementation on neuromorphic systems may help to reduce energy demand.
Conventional ANNs must be converted into equivalent Spiking Neural Networks
(SNNs) in order to be deployed on neuromorphic chips. This paper presents a way
to perform this translation. We map the ANN weights to SNN synapses
layer-by-layer by forming a least-square-error approximation problem at each
layer.
  An optimal set of synapse weights may then be found for a given choice of ANN
activation function and SNN neuron. Using an appropriate constrained solver, we
can generate SNNs compatible with digital, analog, or hybrid chip
architectures. We present an optimal node pruning method to allow SNN layer
sizes to be set by the designer. To illustrate this process, we convert three
ANNs, including one convolutional network, to SNNs. In all three cases, a
simple linear program solver was used. The experiments show that the resulting
networks maintain agreement with the original ANN and excellent performance on
the evaluation tasks. The networks were also reduced in size with little loss
in task performance.
Simplicial Closure and higher-order link prediction
Authors: Austin R. Benson, Rediet Abebe, Michael T. Schaub, Ali Jadbabaie, Jon Kleinberg
Date: 2018-02-20 00:00:00
Tasks: Link Prediction, Prediction

Networks provide a powerful formalism for modeling complex systems by using a
model of pairwise interactions. But much of the structure within these systems
involves interactions that take place among more than two nodes at once; for
example, communication within a group rather than person-to person,
collaboration among a team rather than a pair of coauthors, or biological
interaction between a set of molecules rather than just two. Such higher-order
interactions are ubiquitous, but their empirical study has received limited
attention, and little is known about possible organizational principles of such
structures. Here we study the temporal evolution of 19 datasets with explicit
accounting for higher-order interactions. We show that there is a rich variety
of structure in our datasets but datasets from the same system types have
consistent patterns of higher-order structure. Furthermore, we find that tie
strength and edge density are competing positive indicators of higher-order
organization, and these trends are consistent across interactions involving
differing numbers of nodes. To systematically further the study of theories for
such higher-order structures, we propose higher-order link prediction as a
benchmark problem to assess models and algorithms that predict higher-order
structure. We find a fundamental differences from traditional pairwise link
prediction, with a greater role for local rather than long-range information in
predicting the appearance of new interactions.
Provably noise-robust, regularised $k$-means clustering
Authors: Shrinu Kushagra, Yao-Liang Yu, Shai Ben-David
Date: 2017-11-30 00:00:00
Tasks: Clustering

We consider the problem of clustering in the presence of noise. That is, when
on top of cluster structure, the data also contains a subset of
\emph{unstructured} points. Our goal is to detect the clusters despite the
presence of many unstructured points. Any algorithm that achieves this goal is
noise-robust. We consider a regularisation method which converts any
center-based clustering objective into a noise-robust one. We focus on the
$k$-means objective and we prove that the regularised version of $k$-means is
NP-Hard even for $k=1$. We consider two algorithms based on the convex (sdp and
lp) relaxation of the regularised objective and prove robustness guarantees for
both.
  The sdp and lp relaxation of the standard (non-regularised) $k$-means
objective has been previously studied by [ABC+15]. Under the stochastic ball
model of the data they show that the sdp-based algorithm recovers the
underlying structure as long as the balls are separated by $\delta > 2\sqrt{2}
+ \epsilon$. We improve upon this result in two ways. First, we show recovery
even for $\delta > 2 + \epsilon$. Second, our regularised algorithm recovers
the balls even in the presence of noise so long as the number of noisy points
is not too large. We complement our theoretical analysis with simulations and
analyse the effect of various parameters like regularization constant,
noise-level etc. on the performance of our algorithm. In the presence of noise,
our algorithm performs better than $k$-means++ on MNIST.
Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions
Authors: Damek Davis, Dmitriy Drusvyatskiy
Date: 2018-02-08 00:00:00
Tasks: Open-Ended Question Answering

We prove that the proximal stochastic subgradient method, applied to a weakly
convex problem, drives the gradient of the Moreau envelope to zero at the rate
$O(k^{-1/4})$. As a consequence, we resolve an open question on the convergence
rate of the proximal stochastic gradient method for minimizing the sum of a
smooth nonconvex function and a convex proximable function.
Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization
Authors: Yi Zhou, Yingbin Liang, Huishuai Zhang
Date: 2018-02-19 00:00:00

The success of deep learning has led to a rising interest in the
generalization property of the stochastic gradient descent (SGD) method, and
stability is one popular approach to study it. Existing works based on
stability have studied nonconvex loss functions, but only considered the
generalization error of the SGD in expectation. In this paper, we establish
various generalization error bounds with probabilistic guarantee for the SGD.
Specifically, for both general nonconvex loss functions and gradient dominant
loss functions, we characterize the on-average stability of the iterates
generated by SGD in terms of the on-average variance of the stochastic
gradients. Such characterization leads to improved bounds for the
generalization error for SGD. We then study the regularized risk minimization
problem with strongly convex regularizers, and obtain improved generalization
error bounds for proximal SGD. With strongly convex regularizers, we further
establish the generalization error bounds for nonconvex loss functions under
proximal SGD with high-probability guarantee, i.e., exponential concentration
in probability.
Machine Learning Methods for Data Association in Multi-Object Tracking
Authors: Patrick Emami, Panos M. Pardalos, Lily Elefteriadou, Sanjay Ranka
Date: 2018-02-19 00:00:00
Tasks: BIG-bench Machine Learning, Combinatorial Optimization, Multi-Object Tracking, Object Tracking, Representation Learning, Survey

Data association is a key step within the multi-object tracking pipeline that is notoriously challenging due to its combinatorial nature. A popular and general way to formulate data association is as the NP-hard multidimensional assignment problem (MDAP). Over the last few years, data-driven approaches to assignment have become increasingly prevalent as these techniques have started to mature. We focus this survey solely on learning algorithms for the assignment step of multi-object tracking, and we attempt to unify various methods by highlighting their connections to linear assignment as well as to the MDAP. First, we review probabilistic and end-to-end optimization approaches to data association, followed by methods that learn association affinities from data. We then compare the performance of the methods presented in this survey, and conclude by discussing future research directions.
Hierarchical Expertise-Level Modeling for User Specific Robot-Behavior Explanations
Authors: Sarath Sreedharan, Siddharth Srivastava, Subbarao Kambhampati
Date: 2018-02-19 00:00:00
Tasks: Explanation Generation

There is a growing interest within the AI research community to develop
autonomous systems capable of explaining their behavior to users. One aspect of
the explanation generation problem that has yet to receive much attention is
the task of explaining plans to users whose level of expertise differ from that
of the explainer. We propose an approach for addressing this problem by
representing the user's model as an abstraction of the domain model that the
planner uses. We present algorithms for generating minimal explanations in
cases where this abstract human model is not known. We reduce the problem of
generating explanation to a search over the space of abstract models and
investigate possible greedy approximations for minimal explanations. We also
empirically show that our approach can efficiently compute explanations for a
variety of problems.
Learning Hidden Markov Models from Pairwise Co-occurrences with Application to Topic Modeling
Authors: Kejun Huang, Xiao Fu, Nicholas D. Sidiropoulos
Date: 2018-02-19 00:00:00
Proceeding: ICML 2018 7

We present a new algorithm for identifying the transition and emission
probabilities of a hidden Markov model (HMM) from the emitted data.
Expectation-maximization becomes computationally prohibitive for long
observation records, which are often required for identification. The new
algorithm is particularly suitable for cases where the available sample size is
large enough to accurately estimate second-order output probabilities, but not
higher-order ones. We show that if one is only able to obtain a reliable
estimate of the pairwise co-occurrence probabilities of the emissions, it is
still possible to uniquely identify the HMM if the emission probability is
\emph{sufficiently scattered}. We apply our method to hidden topic Markov
modeling, and demonstrate that we can learn topics with higher quality if
documents are modeled as observations of HMMs sharing the same emission (topic)
probability, compared to the simple but widely used bag-of-words model.
Automated Playtesting with Procedural Personas through MCTS with Evolved Heuristics
Authors: Christoffer Holmgård, Michael Cerny Green, Antonios Liapis, Julian Togelius
Date: 2018-02-19 00:00:00

This paper describes a method for generative player modeling and its
application to the automatic testing of game content using archetypal player
models called procedural personas. Theoretically grounded in psychological
decision theory, procedural personas are implemented using a variation of Monte
Carlo Tree Search (MCTS) where the node selection criteria are developed using
evolutionary computation, replacing the standard UCB1 criterion of MCTS. Using
these personas we demonstrate how generative player models can be applied to a
varied corpus of game levels and demonstrate how different play styles can be
enacted in each level. In short, we use artificially intelligent personas to
construct synthetic playtesters. The proposed approach could be used as a tool
for automatic play testing when human feedback is not readily available or when
quick visualization of potential interactions is necessary. Possible
applications include interactive tools during game development or procedural
content generation systems where many evaluations must be conducted within a
short time span.
Adaptive and Resilient Soft Tensegrity Robots
Authors: John Rieffel, Jean-Baptiste Mouret
Date: 2017-02-10 00:00:00

Living organisms intertwine soft (e.g., muscle) and hard (e.g., bones)
materials, giving them an intrinsic flexibility and resiliency often lacking in
conventional rigid robots. The emerging field of soft robotics seeks to harness
these same properties in order to create resilient machines. The nature of soft
materials, however, presents considerable challenges to aspects of design,
construction, and control -- and up until now, the vast majority of gaits for
soft robots have been hand-designed through empirical trial-and-error. This
manuscript describes an easy-to-assemble tensegrity-based soft robot capable of
highly dynamic locomotive gaits and demonstrating structural and behavioral
resilience in the face of physical damage. Enabling this is the use of a
machine learning algorithm able to discover effective gaits with a minimal
number of physical trials. These results lend further credence to soft-robotic
approaches that seek to harness the interaction of complex material dynamics in
order to generate a wealth of dynamical behaviors.