pwc-archive/papers-with-abstracts huggingface dataset:
SIMILARnet: Simultaneous Intelligent Localization and Recognition Network
Authors: Arna Ghosh, Biswarup Bhattacharya, Somnath Basu Roy Chowdhury
Date: 2017-11-08 00:00:00
Tasks: General Classification, image-classification, Image Classification, Position

Global Average Pooling (GAP) [4] has been used previously to generate class
activation for image classification tasks. The motivation behind SIMILARnet
comes from the fact that the convolutional filters possess position information
of the essential features and hence, combination of the feature maps could help
us locate the class instances in an image. We propose a biologically inspired
model that is free of differential connections and doesn't require separate
training thereby reducing computation overhead. Our novel architecture
generates promising results and unlike existing methods, the model is not
sensitive to the input image size, thus promising wider application. Codes for
the experiment and illustrations can be found at:
https://github.com/brcsomnath/Advanced-GAP .
Inverse Reward Design
Authors: Dylan Hadfield-Menell, Smitha Milli, Pieter Abbeel, Stuart Russell, Anca Dragan
Date: 2017-11-08 00:00:00
Proceeding: NeurIPS 2017 12

Autonomous agents optimize the reward function we give them. What they don't know is how hard it is for us to design a reward function that actually captures what we want. When designing the reward, we might think of some specific training scenarios, and make sure that the reward will lead to the right behavior in those scenarios. Inevitably, agents encounter new scenarios (e.g., new types of terrain) where optimizing that same reward may lead to undesired behavior. Our insight is that reward functions are merely observations about what the designer actually wants, and that they should be interpreted in the context in which they were designed. We introduce inverse reward design (IRD) as the problem of inferring the true objective based on the designed reward and the training MDP. We introduce approximate methods for solving IRD problems, and use their solution to plan risk-averse behavior in test MDPs. Empirical results suggest that this approach can help alleviate negative side effects of misspecified reward functions and mitigate reward hacking.
Preference-based performance measures for Time-Domain Global Similarity method
Authors: Ting Lan, Jian Liu, Hong Qin
Date: 2017-06-30 00:00:00
Tasks: Binary Classification, General Classification

For Time-Domain Global Similarity (TDGS) method, which transforms the data
cleaning problem into a binary classification problem about the physical
similarity between channels, directly adopting common performance measures
could only guarantee the performance for physical similarity. Nevertheless,
practical data cleaning tasks have preferences for the correctness of original
data sequences. To obtain the general expressions of performance measures based
on the preferences of tasks, the mapping relations between performance of TDGS
method about physical similarity and correctness of data sequences are
investigated by probability theory in this paper. Performance measures for TDGS
method in several common data cleaning tasks are set. Cases when these
preference-based performance measures could be simplified are introduced.
Heuristic Search for Structural Constraints in Data Association
Authors: Xiao Zhou, Peilin Jiang, Fei Wang
Date: 2017-11-08 00:00:00
Tasks: Heuristic Search, Multi-Object Tracking, Object, Object Tracking, Online Multi-Object Tracking

The research on multi-object tracking (MOT) is essentially to solve for the
data association assignment, the core of which is to design the association
cost as discriminative as possible. Generally speaking, the match ambiguities
caused by similar appearances of objects and the moving cameras make the data
association perplexing and challenging. In this paper, we propose a new
heuristic method to search for structural constraints (HSSC) of multiple
targets when solving the problem of online multi-object tracking. We believe
that the internal structure among multiple targets in the adjacent frames could
remain constant and stable even though the video sequences are captured by a
moving camera. As a result, the structural constraints are able to cut down the
match ambiguities caused by the moving cameras as well as similar appearances
of the tracked objects. The proposed heuristic method aims to obtain a maximum
match set under the minimum structural cost for each available match pair,
which can be integrated with the raw association costs and make them more
elaborate and discriminative compared with other approaches. In addition, this
paper presents a new method to recover missing targets by minimizing the cost
function generated from both motion and structure cues. Our online multi-object
tracking (MOT) algorithm based on HSSC has achieved the multi-object tracking
accuracy (MOTA) of 25.0 on the public dataset 2DMOT2015[1].
Multi-label Image Recognition by Recurrently Discovering Attentional Regions
Authors: Zhouxia Wang, Tianshui Chen, Guanbin Li, Ruijia Xu, Liang Lin
Date: 2017-11-08 00:00:00
Proceeding: ICCV 2017 10
Tasks: General Classification, image-classification, Image Classification, Multi-Label Image Classification, Multi-Label Image Recognition, Region Proposal

This paper proposes a novel deep architecture to address multi-label image
recognition, a fundamental and practical task towards general visual
understanding. Current solutions for this task usually rely on an extra step of
extracting hypothesis regions (i.e., region proposals), resulting in redundant
computation and sub-optimal performance. In this work, we achieve the
interpretable and contextualized multi-label image classification by developing
a recurrent memorized-attention module. This module consists of two alternately
performed components: i) a spatial transformer layer to locate attentional
regions from the convolutional feature maps in a region-proposal-free way and
ii) an LSTM (Long-Short Term Memory) sub-network to sequentially predict
semantic labeling scores on the located regions while capturing the global
dependencies of these regions. The LSTM also output the parameters for
computing the spatial transformer. On large-scale benchmarks of multi-label
image classification (e.g., MS-COCO and PASCAL VOC 07), our approach
demonstrates superior performances over other existing state-of-the-arts in
both accuracy and efficiency.
Deep Fault Analysis and Subset Selection in Solar Power Grids
Authors: Biswarup Bhattacharya, Abhishek Sinha
Date: 2017-11-08 00:00:00

Non-availability of reliable and sustainable electric power is a major
problem in the developing world. Renewable energy sources like solar are not
very lucrative in the current stage due to various uncertainties like weather,
storage, land use among others. There also exists various other issues like
mis-commitment of power, absence of intelligent fault analysis, congestion,
etc. In this paper, we propose a novel deep learning-based system for
predicting faults and selecting power generators optimally so as to reduce
costs and ensure higher reliability in solar power systems. The results are
highly encouraging and they suggest that the approaches proposed in this paper
have the potential to be applied successfully in the developing world.
Machine Learning Based Fast Power Integrity Classifier
Authors: HuaChun Zhang, Lynden Kagan, Chen Zheng
Date: 2017-11-08 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we proposed a new machine learning based fast power integrity
classifier that quickly flags the EM/IR hotspots. We discussed the features to
extract to describe the power grid, cell power density, routing impact and
controlled collapse chip connection (C4) bumps, etc. The continuous and
discontinuous cases are identified and treated using different machine learning
models. Nearest neighbors, random forest and neural network models are compared
to select the best performance candidates. Experiments are run on open source
benchmark, and result is showing promising prediction accuracy.
A New Hybrid-parameter Recurrent Neural Networks for Online Handwritten Chinese Character Recognition
Authors: Haiqing Ren, Weiqiang Wang
Date: 2017-11-08 00:00:00
Tasks: Temporal Sequences

The recurrent neural network (RNN) is appropriate for dealing with temporal
sequences. In this paper, we present a deep RNN with new features and apply it
for online handwritten Chinese character recognition. Compared with the
existing RNN models, three innovations are involved in the proposed system.
First, a new hidden layer function for RNN is proposed for learning temporal
information better. we call it Memory Pool Unit (MPU). The proposed MPU has a
simple architecture. Second, a new RNN architecture with hybrid parameter is
presented, in order to increasing the expression capacity of RNN. The proposed
hybrid-parameter RNN has parameter changes when calculating the iteration at
temporal dimension. Third, we make a adaptation that all the outputs of each
layer are stacked as the output of network. Stacked hidden layer states combine
all the hidden layer states for increasing the expression capacity. Experiments
are carried out on the IAHCC-UCAS2016 dataset and the CASIA-OLHWDB1.1 dataset.
The experimental results show that the hybrid-parameter RNN obtain a better
recognition performance with higher efficiency (fewer parameters and faster
speed). And the proposed Memory Pool Unit is proved to be a simple hidden layer
function and obtains a competitive recognition results.
Faster Fuzzing: Reinitialization with Deep Neural Models
Authors: Nicole Nichols, Mark Raugas, Robert Jasper, Nathan Hilliard
Date: 2017-11-08 00:00:00
Tasks: CPU, Generative Adversarial Network

We improve the performance of the American Fuzzy Lop (AFL) fuzz testing
framework by using Generative Adversarial Network (GAN) models to reinitialize
the system with novel seed files. We assess performance based on the temporal
rate at which we produce novel and unseen code paths. We compare this approach
to seed file generation from a random draw of bytes observed in the training
seed files. The code path lengths and variations were not sufficiently diverse
to fully replace AFL input generation. However, augmenting native AFL with
these additional code paths demonstrated improvements over AFL alone.
Specifically, experiments showed the GAN was faster and more effective than the
LSTM and out-performed a random augmentation strategy, as measured by the
number of unique code paths discovered. GAN helps AFL discover 14.23% more code
paths than the random strategy in the same amount of CPU time, finds 6.16% more
unique code paths, and finds paths that are on average 13.84% longer. Using GAN
shows promise as a reinitialization strategy for AFL to help the fuzzer
exercise deep paths in software.
Labelled network subgraphs reveal stylistic subtleties in written texts
Authors: Vanessa Q. Marinho, Graeme Hirst, Diego R. Amancio
Date: 2017-05-01 00:00:00
Tasks: Authorship Attribution, Machine Translation, Translation

The vast amount of data and increase of computational capacity have allowed
the analysis of texts from several perspectives, including the representation
of texts as complex networks. Nodes of the network represent the words, and
edges represent some relationship, usually word co-occurrence. Even though
networked representations have been applied to study some tasks, such
approaches are not usually combined with traditional models relying upon
statistical paradigms. Because networked models are able to grasp textual
patterns, we devised a hybrid classifier, called labelled subgraphs, that
combines the frequency of common words with small structures found in the
topology of the network, known as motifs. Our approach is illustrated in two
contexts, authorship attribution and translationese identification. In the
former, a set of novels written by different authors is analyzed. To identify
translationese, texts from the Canadian Hansard and the European parliament
were classified as to original and translated instances. Our results suggest
that labelled subgraphs are able to represent texts and it should be further
explored in other tasks, such as the analysis of text complexity, language
proficiency, and machine translation.
Metric Learning-based Generative Adversarial Network
Authors: Zi-Yi Dou
Date: 2017-11-08 00:00:00
Tasks: Generative Adversarial Network, Metric Learning

Generative Adversarial Networks (GANs), as a framework for estimating
generative models via an adversarial process, have attracted huge attention and
have proven to be powerful in a variety of tasks. However, training GANs is
well known for being delicate and unstable, partially caused by its sig- moid
cross entropy loss function for the discriminator. To overcome such a problem,
many researchers directed their attention on various ways to measure how close
the model distribution and real distribution are and have applied dif- ferent
metrics as their objective functions. In this paper, we propose a novel
framework to train GANs based on distance metric learning and we call it Metric
Learning-based Gener- ative Adversarial Network (MLGAN). The discriminator of
MLGANs can dynamically learn an appropriate metric, rather than a static one,
to measure the distance between generated samples and real samples. Afterwards,
MLGANs update the generator under the newly learned metric. We evaluate our ap-
proach on several representative datasets and the experimen- tal results
demonstrate that MLGANs can achieve superior performance compared with several
existing state-of-the-art approaches. We also empirically show that MLGANs
could increase the stability of training GANs.
Estimating Mixture Entropy with Pairwise Distances
Authors: Artemy Kolchinsky, Brendan D. Tracey
Date: 2017-06-08 00:00:00
Tasks: Form

Mixture distributions arise in many parametric and non-parametric settings --
for example, in Gaussian mixture models and in non-parametric estimation. It is
often necessary to compute the entropy of a mixture, but, in most cases, this
quantity has no closed-form expression, making some form of approximation
necessary. We propose a family of estimators based on a pairwise distance
function between mixture components, and show that this estimator class has
many attractive properties. For many distributions of interest, the proposed
estimators are efficient to compute, differentiable in the mixture parameters,
and become exact when the mixture components are clustered. We prove this
family includes lower and upper bounds on the mixture entropy. The Chernoff
$\alpha$-divergence gives a lower bound when chosen as the distance function,
with the Bhattacharyya distance providing the tightest lower bound for
components that are symmetric and members of a location family. The
Kullback-Leibler divergence gives an upper bound when used as the distance
function. We provide closed-form expressions of these bounds for mixtures of
Gaussians, and discuss their applications to the estimation of mutual
information. We then demonstrate that our bounds are significantly tighter than
well-known existing bounds using numeric simulations. This estimator class is
very useful in optimization problems involving maximization/minimization of
entropy and mutual information, such as MaxEnt and rate distortion problems.
Block-Sparse Recurrent Neural Networks
Authors: Sharan Narang, Eric Undersander, Gregory Diamos
Date: 2017-11-08 00:00:00
Proceeding: ICLR 2018 1
Tasks: Language Modelling, Machine Translation, speech-recognition, Speech Recognition, Translation

Recurrent Neural Networks (RNNs) are used in state-of-the-art models in
domains such as speech recognition, machine translation, and language
modelling. Sparsity is a technique to reduce compute and memory requirements of
deep learning models. Sparse RNNs are easier to deploy on devices and high-end
server processors. Even though sparse operations need less compute and memory
relative to their dense counterparts, the speed-up observed by using sparse
operations is less than expected on different hardware platforms. In order to
address this issue, we investigate two different approaches to induce block
sparsity in RNNs: pruning blocks of weights in a layer and using group lasso
regularization to create blocks of weights with zeros. Using these techniques,
we demonstrate that we can create block-sparse RNNs with sparsity ranging from
80% to 90% with small loss in accuracy. This allows us to reduce the model size
by roughly 10x. Additionally, we can prune a larger dense network to recover
this loss in accuracy while maintaining high block sparsity and reducing the
overall parameter count. Our technique works with a variety of block sizes up
to 32x32. Block-sparse RNNs eliminate overheads related to data storage and
irregular memory accesses while increasing hardware efficiency compared to
unstructured sparsity.
Equivalence between LINE and Matrix Factorization
Authors: Qiao Wang, Zheng Wang, Xiaojun Ye
Date: 2017-07-19 00:00:00
Tasks: Network Embedding

LINE [1], as an efficient network embedding method, has shown its
effectiveness in dealing with large-scale undirected, directed, and/or weighted
networks. Particularly, it proposes to preserve both the local structure
(represented by First-order Proximity) and global structure (represented by
Second-order Proximity) of the network. In this study, we prove that LINE with
these two proximities (LINE(1st) and LINE(2nd)) are actually factoring two
different matrices separately. Specifically, LINE(1st) is factoring a matrix M
(1), whose entries are the doubled Pointwise Mutual Information (PMI) of vertex
pairs in undirected networks, shifted by a constant. LINE(2nd) is factoring a
matrix M (2), whose entries are the PMI of vertex and context pairs in directed
networks, shifted by a constant. We hope this finding would provide a basis for
further extensions and generalizations of LINE.
Linear regression without correspondence
Authors: Daniel Hsu, Kevin Shi, Xiaorui Sun
Date: 2017-05-19 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression

This article considers algorithmic and statistical aspects of linear
regression when the correspondence between the covariates and the responses is
unknown. First, a fully polynomial-time approximation scheme is given for the
natural least squares optimization problem in any constant dimension. Next, in
an average-case and noise-free setting where the responses exactly correspond
to a linear function of i.i.d. draws from a standard multivariate normal
distribution, an efficient algorithm based on lattice basis reduction is shown
to exactly recover the unknown linear function in arbitrary dimension. Finally,
lower bounds on the signal-to-noise ratio are established for approximate
recovery of the unknown linear function by any estimator.
Hidden Markov Random Field Iterative Closest Point
Authors: John Stechschulte, Christoffer Heckman
Date: 2017-11-07 00:00:00

When registering point clouds resolved from an underlying 2-D pixel
structure, such as those resulting from structured light and flash LiDAR
sensors, or stereo reconstruction, it is expected that some points in one cloud
do not have corresponding points in the other cloud, and that these would occur
together, such as along an edge of the depth map. In this work, a hidden Markov
random field model is used to capture this prior within the framework of the
iterative closest point algorithm. The EM algorithm is used to estimate the
distribution parameters and the hidden component memberships. Experiments are
presented demonstrating that this method outperforms several other outlier
rejection methods when the point clouds have low or moderate overlap.
Evaluation of Croatian Word Embeddings
Authors: Lukas Svoboda, Slobodan Beliga
Date: 2017-11-06 00:00:00
Proceeding: LREC 2018 5
Tasks: Word Embeddings

Croatian is poorly resourced and highly inflected language from Slavic
language family. Nowadays, research is focusing mostly on English. We created a
new word analogy corpus based on the original English Word2vec word analogy
corpus and added some of the specific linguistic aspects from Croatian
language. Next, we created Croatian WordSim353 and RG65 corpora for a basic
evaluation of word similarities. We compared created corpora on two popular
word representation models, based on Word2Vec tool and fastText tool. Models
has been trained on 1.37B tokens training data corpus and tested on a new
robust Croatian word analogy corpus. Results show that models are able to
create meaningful word representation. This research has shown that free word
order and the higher morphological complexity of Croatian language influences
the quality of resulting word embeddings.
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
Authors: Abhishek Kumar, Prasanna Sattigeri, Avinash Balakrishnan
Date: 2017-11-02 00:00:00
Proceeding: ICLR 2018 1
Tasks: Disentanglement, Variational Inference

Disentangled representations, where the higher level data generative factors
are reflected in disjoint latent dimensions, offer several benefits such as
ease of deriving invariant representations, transferability to other tasks,
interpretability, etc. We consider the problem of unsupervised learning of
disentangled representations from large pool of unlabeled observations, and
propose a variational inference based approach to infer disentangled latent
factors. We introduce a regularizer on the expectation of the approximate
posterior over observed data that encourages the disentanglement. We also
propose a new disentanglement metric which is better aligned with the
qualitative disentanglement observed in the decoder's output. We empirically
observe significant improvement over existing methods in terms of both
disentanglement and data likelihood (reconstruction quality).
UCB Exploration via Q-Ensembles
Authors: Richard Y. Chen, Szymon Sidor, Pieter Abbeel, John Schulman
Date: 2017-06-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We show how an ensemble of $Q^*$-functions can be leveraged for more
effective exploration in deep reinforcement learning. We build on well
established algorithms from the bandit setting, and adapt them to the
$Q$-learning setting. We propose an exploration strategy based on
upper-confidence bounds (UCB). Our experiments show significant gains on the
Atari benchmark.
Tangent: Automatic Differentiation Using Source Code Transformation in Python
Authors: Bart van Merriënboer, Alexander B. Wiltschko, Dan Moldovan
Date: 2017-11-07 00:00:00
Tasks: BIG-bench Machine Learning

Automatic differentiation (AD) is an essential primitive for machine learning
programming systems. Tangent is a new library that performs AD using source
code transformation (SCT) in Python. It takes numeric functions written in a
syntactic subset of Python and NumPy as input, and generates new Python
functions which calculate a derivative. This approach to automatic
differentiation is different from existing packages popular in machine
learning, such as TensorFlow and Autograd. Advantages are that Tangent
generates gradient code in Python which is readable by the user, easy to
understand and debug, and has no runtime overhead. Tangent also introduces
abstractions for easily injecting logic into the generated gradient code,
further improving usability.