pwc-archive/papers-with-abstracts huggingface dataset:
The IIT Bombay English-Hindi Parallel Corpus
Authors: Anoop Kunchukuttan, Pratik Mehta, Pushpak Bhattacharyya
Date: 2017-10-08 00:00:00
Proceeding: LREC 2018 5
Tasks: Machine Translation, NMT, Translation

We present the IIT Bombay English-Hindi Parallel Corpus. The corpus is a
compilation of parallel corpora previously available in the public domain as
well as new parallel corpora we collected. The corpus contains 1.49 million
parallel segments, of which 694k segments were not previously available in the
public domain. The corpus has been pre-processed for machine translation, and
we report baseline phrase-based SMT and NMT translation results on this corpus.
This corpus has been used in two editions of shared tasks at the Workshop on
Asian Language Translation (2016 and 2017). The corpus is freely available for
non-commercial research. To the best of our knowledge, this is the largest
publicly available English-Hindi parallel corpus.
Regularized Loss Minimizers with Local Data Perturbation: Consistency and Data Irrecoverability
Authors: Zitao Li, Jean Honorio
Date: 2018-05-19 00:00:00

We introduce a new concept, data irrecoverability, and show that the well-studied concept of data privacy is sufficient but not necessary for data irrecoverability. We show that there are several regularized loss minimization problems that can use perturbed data with theoretical guarantees of generalization, i.e., loss consistency. Our results quantitatively connect the convergence rates of the learning problems to the impossibility for any adversary for recovering the original data from perturbed observations. In addition, we show several examples where the convergence rates with perturbed data only increase the convergence rates with original data within a constant factor related to the amount of perturbation, i.e., noise.
Capturing human category representations by sampling in deep feature spaces
Authors: Joshua C. Peterson, Jordan W. Suchow, Krisha Aghi, Alexander Y. Ku, Thomas L. Griffiths
Date: 2018-05-19 00:00:00

Understanding how people represent categories is a core problem in cognitive
science. Decades of research have yielded a variety of formal theories of
categories, but validating them with naturalistic stimuli is difficult. The
challenge is that human category representations cannot be directly observed
and running informative experiments with naturalistic stimuli such as images
requires a workable representation of these stimuli. Deep neural networks have
recently been successful in solving a range of computer vision tasks and
provide a way to compactly represent image features. Here, we introduce a
method to estimate the structure of human categories that combines ideas from
cognitive science and machine learning, blending human-based algorithms with
state-of-the-art deep image generators. We provide qualitative and quantitative
results as a proof-of-concept for the method's feasibility. Samples drawn from
human distributions rival those from state-of-the-art generative models in
quality and outperform alternative methods for estimating the structure of
human categories.
NtMalDetect: A Machine Learning Approach to Malware Detection Using Native API System Calls
Authors: Chan Woo Kim
Date: 2018-02-15 00:00:00
Tasks: BIG-bench Machine Learning, Document Classification, General Classification, Malware Analysis, Malware Detection

As computing systems become increasingly advanced and as users increasingly
engage themselves in technology, security has never been a greater concern. In
malware detection, static analysis, the method of analyzing potentially
malicious files, has been the prominent approach. This approach, however,
quickly falls short as malicious programs become more advanced and adopt the
capabilities of obfuscating its binaries to execute the same malicious
functions, making static analysis extremely difficult for newer variants. The
approach assessed in this paper is a novel dynamic malware analysis method,
which may generalize better than static analysis to newer variants. Inspired by
recent successes in Natural Language Processing (NLP), widely used document
classification techniques were assessed in detecting malware by doing such
analysis on system calls, which contain useful information about the operation
of a program as requests that the program makes of the kernel. Features
considered are extracted from system call traces of benign and malicious
programs, and the task to classify these traces is treated as a binary document
classification task of system call traces. The system call traces were
processed to remove the parameters to only leave the system call function
names. The features were grouped into various n-grams and weighted with Term
Frequency-Inverse Document Frequency. This paper shows that Linear Support
Vector Machines (SVM) optimized by Stochastic Gradient Descent and the
traditional Coordinate Descent on the Wolfe Dual form of the SVM are effective
in this approach, achieving a highest of 96% accuracy with 95% recall score.
Additional contributions include the identification of significant system call
sequences that could be avenues for further research.
Learning Sampling Policies for Domain Adaptation
Authors: Yash Patel, Kashyap Chitta, Bhavan Jasani
Date: 2018-05-19 00:00:00
Tasks: Classification, Domain Adaptation, General Classification, Q-Learning, Semi-supervised Domain Adaptation

We address the problem of semi-supervised domain adaptation of classification
algorithms through deep Q-learning. The core idea is to consider the
predictions of a source domain network on target domain data as noisy labels,
and learn a policy to sample from this data so as to maximize classification
accuracy on a small annotated reward partition of the target domain. Our
experiments show that learned sampling policies construct labeled sets that
improve accuracies of visual classifiers over baselines.
Comments on "Momentum fractional LMS for power signal parameter estimation"
Authors: Shujaat Khan, Imran Naseem, Alishba Sadiq, Jawwad Ahmad, Muhammad Moinuddin
Date: 2018-05-19 00:00:00
Tasks: parameter estimation

The purpose of this paper is to indicate that the recently proposed Momentum
fractional least mean squares (mFLMS) algorithm has some serious flaws in its
design and analysis. Our apprehensions are based on the evidence we found in
the derivation and analysis in the paper titled: \textquotedblleft
\textit{Momentum fractional LMS for power signal parameter
estimation}\textquotedblright. In addition to the theoretical bases our claims
are also verified through extensive simulation results. The experiments clearly
show that the new method does not have any advantage over the classical least
mean square (LMS) method.
Estimation of Low-Rank Matrices via Approximate Message Passing
Authors: Andrea Montanari, Ramji Venkataramanan
Date: 2017-11-06 00:00:00
Tasks: Community Detection

Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially outperform simple spectral approaches. Recent work characterizes the asymptotic accuracy of Bayes-optimal estimators in the high-dimensional limit. In this paper we present a practical algorithm that can achieve Bayes-optimal accuracy above the spectral threshold. A bold conjecture from statistical physics posits that no polynomial-time algorithm achieves optimal error below the same threshold (unless the best estimator is trivial). Our approach uses Approximate Message Passing (AMP) in conjunction with a spectral initialization. AMP algorithms have proved successful in a variety of statistical estimation tasks, and are amenable to exact asymptotic analysis via state evolution. Unfortunately, state evolution is uninformative when the algorithm is initialized near an unstable fixed point, as often happens in low-rank matrix estimation. We develop a new analysis of AMP that allows for spectral initializations. Our main theorem is general and applies beyond matrix estimation. However, we use it to derive detailed predictions for the problem of estimating a rank-one matrix in noise. Special cases of this problem are closely related---via universality arguments---to the network community detection problem for two asymmetric communities. For general rank-one models, we show that AMP can be used to construct confidence intervals and control false discovery rate. We provide illustrations of the general methodology by considering the cases of sparse low-rank matrices and of block-constant low-rank matrices with symmetric blocks (we refer to the latter as to the `Gaussian Block Model').
Heterogeneous Multi-output Gaussian Process Prediction
Authors: Pablo Moreno-Muñoz, Antonio Artés-Rodríguez, Mauricio A. Álvarez
Date: 2018-05-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Gaussian Processes, Prediction, Variational Inference

We present a novel extension of multi-output Gaussian processes for handling
heterogeneous outputs. We assume that each output has its own likelihood
function and use a vector-valued Gaussian process prior to jointly model the
parameters in all likelihoods as latent functions. Our multi-output Gaussian
process uses a covariance function with a linear model of coregionalisation
form. Assuming conditional independence across the underlying latent functions
together with an inducing variable framework, we are able to obtain tractable
variational bounds amenable to stochastic variational inference. We illustrate
the performance of the model on synthetic data and two real datasets: a human
behavioral study and a demographic high-dimensional dataset.
Latent Space Non-Linear Statistics
Authors: Line Kuhnel, Tom Fletcher, Sarang Joshi, Stefan Sommer
Date: 2018-05-19 00:00:00

Given data, deep generative models, such as variational autoencoders (VAE)
and generative adversarial networks (GAN), train a lower dimensional latent
representation of the data space. The linear Euclidean geometry of data space
pulls back to a nonlinear Riemannian geometry on the latent space. The latent
space thus provides a low-dimensional nonlinear representation of data and
classical linear statistical techniques are no longer applicable. In this paper
we show how statistics of data in their latent space representation can be
performed using techniques from the field of nonlinear manifold statistics.
Nonlinear manifold statistics provide generalizations of Euclidean statistical
notions including means, principal component analysis, and maximum likelihood
fits of parametric probability distributions. We develop new techniques for
maximum likelihood inference in latent space, and adress the computational
complexity of using geometric algorithms with high-dimensional data by training
a separate neural network to approximate the Riemannian metric and cometric
tensor capturing the shape of the learned data manifold.
Learning to Detect
Authors: Neev Samuel, Tzvi Diskin, Ami Wiesel
Date: 2018-05-19 00:00:00

In this paper we consider Multiple-Input-Multiple-Output (MIMO) detection
using deep neural networks. We introduce two different deep architectures: a
standard fully connected multi-layer network, and a Detection Network (DetNet)
which is specifically designed for the task. The structure of DetNet is
obtained by unfolding the iterations of a projected gradient descent algorithm
into a network. We compare the accuracy and runtime complexity of the purposed
approaches and achieve state-of-the-art performance while maintaining low
computational requirements. Furthermore, we manage to train a single network to
detect over an entire distribution of channels. Finally, we consider detection
with soft outputs and show that the networks can easily be modified to produce
soft decisions.
Nonparametric Bayesian Deep Networks with Local Competition
Authors: Konstantinos P. Panousis, Sotirios Chatzis, Sergios Theodoridis
Date: 2018-05-19 00:00:00
Tasks: Bayesian Inference

The aim of this work is to enable inference of deep networks that retain high accuracy for the least possible model complexity, with the latter deduced from the data during inference. To this end, we revisit deep networks that comprise competing linear units, as opposed to nonlinear units that do not entail any form of (local) competition. In this context, our main technical innovation consists in an inferential setup that leverages solid arguments from Bayesian nonparametrics. We infer both the needed set of connections or locally competing sets of units, as well as the required floating-point precision for storing the network parameters. Specifically, we introduce auxiliary discrete latent variables representing which initial network components are actually needed for modeling the data at hand, and perform Bayesian inference over them by imposing appropriate stick-breaking priors. As we experimentally show using benchmark datasets, our approach yields networks with less computational footprint than the state-of-the-art, and with no compromises in predictive accuracy.
CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces
Authors: Liheng Zhang, Marzieh Edraki, Guo-Jun Qi
Date: 2018-05-19 00:00:00
Proceeding: NeurIPS 2018 12

In this paper, we formalize the idea behind capsule nets of using a capsule
vector rather than a neuron activation to predict the label of samples. To this
end, we propose to learn a group of capsule subspaces onto which an input
feature vector is projected. Then the lengths of resultant capsules are used to
score the probability of belonging to different classes. We train such a
Capsule Projection Network (CapProNet) by learning an orthogonal projection
matrix for each capsule subspace, and show that each capsule subspace is
updated until it contains input feature vectors corresponding to the associated
class. We will also show that the capsule projection can be viewed as
normalizing the multiple columns of the weight matrix simultaneously to form an
orthogonal basis, which makes it more effective in incorporating novel
components of input features to update capsule representations. In other words,
the capsule projection can be viewed as a multi-dimensional weight
normalization in capsule subspaces, where the conventional weight normalization
is simply a special case of the capsule projection onto 1D lines. Only a small
negligible computing overhead is incurred to train the network in
low-dimensional capsule subspaces or through an alternative hyper-power
iteration to estimate the normalization matrix. Experiment results on image
datasets show the presented model can greatly improve the performance of the
state-of-the-art ResNet backbones by $10-20\%$ and that of the Densenet by
$5-7\%$ respectively at the same level of computing and memory expenses. The
CapProNet establishes the competitive state-of-the-art performance for the
family of capsule nets by significantly reducing test errors on the benchmark
datasets.
Long short-term memory and learning-to-learn in networks of spiking neurons
Authors: Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, Wolfgang Maass
Date: 2018-03-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Reinforcement Learning, Sequential Image Classification, Speech Recognition

Recurrent networks of spiking neurons (RSNNs) underlie the astounding
computing and learning capabilities of the brain. But computing and learning
capabilities of RSNN models have remained poor, at least in comparison with
artificial neural networks (ANNs). We address two possible reasons for that.
One is that RSNNs in the brain are not randomly connected or designed according
to simple rules, and they do not start learning as a tabula rasa network.
Rather, RSNNs in the brain were optimized for their tasks through evolution,
development, and prior experience. Details of these optimization processes are
largely unknown. But their functional contribution can be approximated through
powerful optimization methods, such as backpropagation through time (BPTT).
  A second major mismatch between RSNNs in the brain and models is that the
latter only show a small fraction of the dynamics of neurons and synapses in
the brain. We include neurons in our RSNN model that reproduce one prominent
dynamical process of biological neurons that takes place at the behaviourally
relevant time scale of seconds: neuronal adaptation. We denote these networks
as LSNNs because of their Long short-term memory. The inclusion of adapting
neurons drastically increases the computing and learning capability of RSNNs if
they are trained and configured by deep learning (BPTT combined with a rewiring
algorithm that optimizes the network architecture). In fact, the computational
performance of these RSNNs approaches for the first time that of LSTM networks.
In addition RSNNs with adapting neurons can acquire abstract knowledge from
prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge
in order to learn new but related tasks from very few examples. We demonstrate
this for supervised learning and reinforcement learning.
Generative Creativity: Adversarial Learning for Bionic Design
Authors: Simiao Yu, Hao Dong, Pan Wang, Chao Wu, Yike Guo
Date: 2018-05-19 00:00:00

Bionic design refers to an approach of generative creativity in which a
target object (e.g. a floor lamp) is designed to contain features of biological
source objects (e.g. flowers), resulting in creative biologically-inspired
design. In this work, we attempt to model the process of shape-oriented bionic
design as follows: given an input image of a design target object, the model
generates images that 1) maintain shape features of the input design target
image, 2) contain shape features of images from the specified biological source
domain, 3) are plausible and diverse. We propose DesignGAN, a novel
unsupervised deep generative approach to realising bionic design. Specifically,
we employ a conditional Generative Adversarial Networks architecture with
several designated losses (an adversarial loss, a regression loss, a cycle loss
and a latent loss) that respectively constrict our model to meet the
corresponding aforementioned requirements of bionic design modelling. We
perform qualitative and quantitative experiments to evaluate our method, and
demonstrate that our proposed approach successfully generates creative images
of bionic design.
Fast Retinomorphic Event Stream for Video Recognition and Reinforcement Learning
Authors: Wanjia Liu, Huaijin Chen, Rishab Goel, Yuzhong Huang, Ashok Veeraraghavan, Ankit Patel
Date: 2018-05-16 00:00:00
Tasks: Action Recognition, Atari Games, Domain Adaptation, Event Detection, Optical Flow Estimation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Temporal Action Localization, Video Recognition, Video Understanding

Good temporal representations are crucial for video understanding, and the
state-of-the-art video recognition framework is based on two-stream networks.
In such framework, besides the regular ConvNets responsible for RGB frame
inputs, a second network is introduced to handle the temporal representation,
usually the optical flow (OF). However, OF or other task-oriented flow is
computationally costly, and is thus typically pre-computed. Critically, this
prevents the two-stream approach from being applied to reinforcement learning
(RL) applications such as video game playing, where the next state depends on
current state and action choices. Inspired by the early vision systems of
mammals and insects, we propose a fast event-driven representation (EDR) that
models several major properties of early retinal circuits: (1) logarithmic
input response, (2) multi-timescale temporal smoothing to filter noise, and (3)
bipolar (ON/OFF) pathways for primitive event detection[12]. Trading off the
directional information for fast speed (> 9000 fps), EDR en-ables fast
real-time inference/learning in video applications that require interaction
between an agent and the world such as game-playing, virtual robotics, and
domain adaptation. In this vein, we use EDR to demonstrate performance
improvements over state-of-the-art reinforcement learning algorithms for Atari
games, something that has not been possible with pre-computed OF. Moreover,
with UCF-101 video action recognition experiments, we show that EDR performs
near state-of-the-art in accuracy while achieving a 1,500x speedup in input
representation processing, as compared to optical flow.
Episodic Memory Deep Q-Networks
Authors: Zichuan Lin, Tianqi Zhao, Guangwen Yang, Lintao Zhang
Date: 2018-05-19 00:00:00
Tasks: Atari Games, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) algorithms have made huge progress in recent
years by leveraging the power of deep neural networks (DNN). Despite the
success, deep RL algorithms are known to be sample inefficient, often requiring
many rounds of interaction with the environments to obtain satisfactory
performance. Recently, episodic memory based RL has attracted attention due to
its ability to latch on good actions quickly. In this paper, we present a
simple yet effective biologically inspired RL algorithm called Episodic Memory
Deep Q-Networks (EMDQN), which leverages episodic memory to supervise an agent
during training. Experiments show that our proposed method can lead to better
sample efficiency and is more likely to find good policies. It only requires
1/5 of the interactions of DQN to achieve many state-of-the-art performances on
Atari games, significantly outperforming regular DQN and other episodic memory
based RL algorithms.
Deep Generative Markov State Models
Authors: Hao Wu, Andreas Mardt, Luca Pasquali, Frank Noe
Date: 2018-05-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Time Series, Time Series Analysis, valid

We propose a deep generative Markov State Model (DeepGenMSM) learning
framework for inference of metastable dynamical systems and prediction of
trajectories. After unsupervised training on time series data, the model
contains (i) a probabilistic encoder that maps from high-dimensional
configuration space to a small-sized vector indicating the membership to
metastable (long-lived) states, (ii) a Markov chain that governs the
transitions between metastable states and facilitates analysis of the long-time
dynamics, and (iii) a generative part that samples the conditional distribution
of configurations in the next time step. The model can be operated in a
recursive fashion to generate trajectories to predict the system evolution from
a defined starting state and propose new configurations. The DeepGenMSM is
demonstrated to provide accurate estimates of the long-time kinetics and
generate valid distributions for molecular dynamics (MD) benchmark systems.
Remarkably, we show that DeepGenMSMs are able to make long time-steps in
molecular configuration space and generate physically realistic structures in
regions that were not seen in training data.
Adaptively Pruning Features for Boosted Decision Trees
Authors: Maryam Aziz, Jesse Anderton, Javed Aslam
Date: 2018-05-19 00:00:00

Boosted decision trees enjoy popularity in a variety of applications;
however, for large-scale datasets, the cost of training a decision tree in each
round can be prohibitively expensive. Inspired by ideas from the multi-arm
bandit literature, we develop a highly efficient algorithm for computing exact
greedy-optimal decision trees, outperforming the state-of-the-art Quick Boost
method. We further develop a framework for deriving lower bounds on the problem
that applies to a wide family of conceivable algorithms for the task (including
our algorithm and Quick Boost), and we demonstrate empirically on a wide
variety of data sets that our algorithm is near-optimal within this family of
algorithms. We also derive a lower bound applicable to any algorithm solving
the task, and we demonstrate that our algorithm empirically achieves
performance close to this best-achievable lower bound.