pwc-archive/papers-with-abstracts huggingface dataset:
Training Bit Fully Convolutional Network for Fast Semantic Segmentation
Authors: He Wen, Shuchang Zhou, Zhe Liang, Yuxiang Zhang, Dieqiao Feng, Xinyu Zhou, Cong Yao
Date: 2016-12-01 00:00:00
Tasks: CPU, Segmentation, Semantic Segmentation

Fully convolutional neural networks give accurate, per-pixel prediction for
input images and have applications like semantic segmentation. However, a
typical FCN usually requires lots of floating point computation and large
run-time memory, which effectively limits its usability. We propose a method to
train Bit Fully Convolution Network (BFCN), a fully convolutional neural
network that has low bit-width weights and activations. Because most of its
computation-intensive convolutions are accomplished between low bit-width
numbers, a BFCN can be accelerated by an efficient bit-convolution
implementation. On CPU, the dot product operation between two bit vectors can
be reduced to bitwise operations and popcounts, which can offer much higher
throughput than 32-bit multiplications and additions.
  To validate the effectiveness of BFCN, we conduct experiments on the PASCAL
VOC 2012 semantic segmentation task and Cityscapes. Our BFCN with 1-bit weights
and 2-bit activations, which runs 7.8x faster on CPU or requires less than 1\%
resources on FPGA, can achieve comparable performance as the 32-bit
counterpart.
HyperNetworks
Authors: David Ha, Andrew Dai, Quoc V. Le
Date: 2016-09-27 00:00:00
Tasks: Handwriting generation, Language Modelling, Machine Translation, Translation

This work explores hypernetworks: an approach of using a one network, also
known as a hypernetwork, to generate the weights for another network.
Hypernetworks provide an abstraction that is similar to what is found in
nature: the relationship between a genotype - the hypernetwork - and a
phenotype - the main network. Though they are also reminiscent of HyperNEAT in
evolution, our hypernetworks are trained end-to-end with backpropagation and
thus are usually faster. The focus of this work is to make hypernetworks useful
for deep convolutional networks and long recurrent networks, where
hypernetworks can be viewed as relaxed form of weight-sharing across layers.
Our main result is that hypernetworks can generate non-shared weights for LSTM
and achieve near state-of-the-art results on a variety of sequence modelling
tasks including character-level language modelling, handwriting generation and
neural machine translation, challenging the weight-sharing paradigm for
recurrent networks. Our results also show that hypernetworks applied to
convolutional networks still achieve respectable results for image recognition
tasks compared to state-of-the-art baseline models while requiring fewer
learnable parameters.
KGEval: Estimating Accuracy of Automatically Constructed Knowledge Graphs
Authors: Prakhar Ojha, Partha Talukdar
Date: 2016-10-21 00:00:00
Tasks: Diversity, Knowledge Graphs

Automatic construction of large knowledge graphs (KG) by mining web-scale
text datasets has received considerable attention recently. Estimating accuracy
of such automatically constructed KGs is a challenging problem due to their
size and diversity. This important problem has largely been ignored in prior
research we fill this gap and propose KGEval. KGEval binds facts of a KG using
coupling constraints and crowdsources the facts that infer correctness of large
parts of the KG. We demonstrate that the objective optimized by KGEval is
submodular and NP-hard, allowing guarantees for our approximation algorithm.
Through extensive experiments on real-world datasets, we demonstrate that
KGEval is able to estimate KG accuracy more accurately compared to other
competitive baselines, while requiring significantly lesser number of human
evaluations.
Adversarial Images for Variational Autoencoders
Authors: Pedro Tabacof, Julia Tavares, Eduardo Valle
Date: 2016-12-01 00:00:00
Tasks: Adversarial Attack

We investigate adversarial attacks for autoencoders. We propose a procedure
that distorts the input image to mislead the autoencoder in reconstructing a
completely different target image. We attack the internal latent
representations, attempting to make the adversarial input produce an internal
representation as similar as possible as the target's. We find that
autoencoders are much more robust to the attack than classifiers: while some
examples have tolerably small input distortion, and reasonable similarity to
the target image, there is a quasi-linear trade-off between those aims. We
report results on MNIST and SVHN datasets, and also test regular deterministic
autoencoders, reaching similar conclusions in all cases. Finally, we show that
the usual adversarial attack for classifiers, while being much easier, also
presents a direct proportion between distortion on the input, and misdirection
on the output. That proportionality however is hidden by the normalization of
the output, which maps a linear layer into non-linear probabilities.
A New Method for Classification of Datasets for Data Mining
Authors: Singh Vijendra, Hemjyotsana Parashar, Nisha Vasudeva
Date: 2016-12-01 00:00:00
Tasks: Classification, General Classification

Decision tree is an important method for both induction research and data
mining, which is mainly used for model classification and prediction. ID3
algorithm is the most widely used algorithm in the decision tree so far. In
this paper, the shortcoming of ID3's inclining to choose attributes with many
values is discussed, and then a new decision tree algorithm which is improved
version of ID3. In our proposed algorithm attributes are divided into groups
and then we apply the selection measure 5 for these groups. If information gain
is not good then again divide attributes values into groups. These steps are
done until we get good classification/misclassification ratio. The proposed
algorithms classify the data sets more accurately and efficiently.
Domain Adaptation for Named Entity Recognition in Online Media with Word Embeddings
Authors: Vivek Kulkarni, Yashar Mehdad, Troy Chevalier
Date: 2016-12-01 00:00:00
Tasks: Domain Adaptation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Word Embeddings

Content on the Internet is heterogeneous and arises from various domains like
News, Entertainment, Finance and Technology. Understanding such content
requires identifying named entities (persons, places and organizations) as one
of the key steps. Traditionally Named Entity Recognition (NER) systems have
been built using available annotated datasets (like CoNLL, MUC) and demonstrate
excellent performance. However, these models fail to generalize onto other
domains like Sports and Finance where conventions and language use can differ
significantly. Furthermore, several domains do not have large amounts of
annotated labeled data for training robust Named Entity Recognition models. A
key step towards this challenge is to adapt models learned on domains where
large amounts of annotated training data are available to domains with scarce
annotated data.
  In this paper, we propose methods to effectively adapt models learned on one
domain onto other domains using distributed word representations. First we
analyze the linguistic variation present across domains to identify key
linguistic insights that can boost performance across domains. We propose
methods to capture domain specific semantics of word usage in addition to
global semantics. We then demonstrate how to effectively use such domain
specific knowledge to learn NER models that outperform previous baselines in
the domain adaptation setting.
Less is More: Learning Prominent and Diverse Topics for Data Summarization
Authors: Jian Tang, Cheng Li, Ming Zhang, Qiaozhu Mei
Date: 2016-11-29 00:00:00
Tasks: Data Summarization, Topic Models

Statistical topic models efficiently facilitate the exploration of
large-scale data sets. Many models have been developed and broadly used to
summarize the semantic structure in news, science, social media, and digital
humanities. However, a common and practical objective in data exploration tasks
is not to enumerate all existing topics, but to quickly extract representative
ones that broadly cover the content of the corpus, i.e., a few topics that
serve as a good summary of the data. Most existing topic models fit exactly the
same number of topics as a user specifies, which have imposed an unnecessary
burden to the users who have limited prior knowledge. We instead propose new
models that are able to learn fewer but more representative topics for the
purpose of data summarization. We propose a reinforced random walk that allows
prominent topics to absorb tokens from similar and smaller topics, thus
enhances the diversity among the top topics extracted. With this reinforced
random walk as a general process embedded in classical topic models, we obtain
\textit{diverse topic models} that are able to extract the most prominent and
diverse topics from data. The inference procedures of these diverse topic
models remain as simple and efficient as the classical models. Experimental
results demonstrate that the diverse topic models not only discover topics that
better summarize the data, but also require minimal prior knowledge of the
users.
Collective Decision Dynamics in Group Evacuation: Behavioral Experiment and Machine Learning Models
Authors: Chantal Nguyen, Fangqiu Han, Kimberly J. Schlesinger, Izzeddin Gür, Jean M. Carlson
Date: 2016-06-17 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

Identifying factors that affect human decision making and quantifying their
influence remain essential and challenging tasks for the design and
implementation of social and technological communication systems. We report
results of a behavioral experiment involving decision making in the face of an
impending natural disaster. In a controlled laboratory setting, we characterize
individual and group evacuation decision making influenced by several key
factors, including the likelihood of the disaster, available shelter capacity,
group size, and group decision protocol. Our results show that success in
individual decision making is not a strong predictor of group performance. We
use an artificial neural network trained on the collective behavior of subjects
to predict individual and group outcomes. Overall model accuracy increases with
the inclusion of a subject-specific performance parameter based on laboratory
trials that captures individual differences. In parallel, we demonstrate that
the social media activity of individual subjects, specifically their Facebook
use, can be used to generate an alternative individual personality profile that
leads to comparable model accuracy. Quantitative characterization and
prediction of collective decision making is crucial for the development of
effective policies to guide the action of populations in the face of threat or
uncertainty.
Large Scale SfM with the Distributed Camera Model
Authors: Chris Sweeney, Victor Fragoso, Tobias Hollerer, Matthew Turk
Date: 2016-07-13 00:00:00
Tasks: model

We introduce the distributed camera model, a novel model for
Structure-from-Motion (SfM). This model describes image observations in terms
of light rays with ray origins and directions rather than pixels. As such, the
proposed model is capable of describing a single camera or multiple cameras
simultaneously as the collection of all light rays observed. We show how the
distributed camera model is a generalization of the standard camera model and
describe a general formulation and solution to the absolute camera pose problem
that works for standard or distributed cameras. The proposed method computes a
solution that is up to 8 times more efficient and robust to rotation
singularities in comparison with gDLS. Finally, this method is used in an novel
large-scale incremental SfM pipeline where distributed cameras are accurately
and robustly merged together. This pipeline is a direct generalization of
traditional incremental SfM; however, instead of incrementally adding one
camera at a time to grow the reconstruction the reconstruction is grown by
adding a distributed camera. Our pipeline produces highly accurate
reconstructions efficiently by avoiding the need for many bundle adjustment
iterations and is capable of computing a 3D model of Rome from over 15,000
images in just 22 minutes.
Logarithmic Time One-Against-Some
Authors: Hal Daume III, Nikos Karampatziakis, John Langford, Paul Mineiro
Date: 2016-06-15 00:00:00
Proceeding: ICML 2017 8
Tasks: Binary Classification, Classification, General Classification

We create a new online reduction of multiclass classification to binary
classification for which training and prediction time scale logarithmically
with the number of classes. Compared to previous approaches, we obtain
substantially better statistical performance for two reasons: First, we prove a
tighter and more complete boosting theorem, and second we translate the results
more directly into an algorithm. We show that several simple techniques give
rise to an algorithm that can compete with one-against-all in both space and
predictive power while offering exponential improvements in speed when the
number of classes is large.
Robust Optimization for Tree-Structured Stochastic Network Design
Authors: Xiaojian Wu, Akshat Kumar, Daniel Sheldon, Shlomo Zilberstein
Date: 2016-12-01 00:00:00

Stochastic network design is a general framework for optimizing network
connectivity. It has several applications in computational sustainability
including spatial conservation planning, pre-disaster network preparation, and
river network optimization. A common assumption in previous work has been made
that network parameters (e.g., probability of species colonization) are
precisely known, which is unrealistic in real- world settings. We therefore
address the robust river network design problem where the goal is to optimize
river connectivity for fish movement by removing barriers. We assume that fish
passability probabilities are known only imprecisely, but are within some
interval bounds. We then develop a planning approach that computes the policies
with either high robust ratio or low regret. Empirically, our approach scales
well to large river networks. We also provide insights into the solutions
generated by our robust approach, which has significantly higher robust ratio
than the baseline solution with mean parameter estimates.
Noise-Tolerant Life-Long Matrix Completion via Adaptive Sampling
Authors: Maria-Florina Balcan, Hongyang Zhang
Date: 2016-12-01 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Matrix Completion

We study the problem of recovering an incomplete $m\times n$ matrix of rank
$r$ with columns arriving online over time. This is known as the problem of
life-long matrix completion, and is widely applied to recommendation system,
computer vision, system identification, etc. The challenge is to design
provable algorithms tolerant to a large amount of noises, with small sample
complexity. In this work, we give algorithms achieving strong guarantee under
two realistic noise models. In bounded deterministic noise, an adversary can
add any bounded yet unstructured noise to each column. For this problem, we
present an algorithm that returns a matrix of a small error, with sample
complexity almost as small as the best prior results in the noiseless case. For
sparse random noise, where the corrupted columns are sparse and drawn randomly,
we give an algorithm that exactly recovers an $\mu_0$-incoherent matrix by
probability at least $1-\delta$ with sample complexity as small as
$O\left(\mu_0rn\log (r/\delta)\right)$. This result advances the
state-of-the-art work and matches the lower bound in a worst case. We also
study the scenario where the hidden matrix lies on a mixture of subspaces and
show that the sample complexity can be even smaller. Our proposed algorithms
perform well experimentally in both synthetic and real-world datasets.
Optimizing Quantiles in Preference-based Markov Decision Processes
Authors: Hugo Gilbert, Paul Weng, Yan Xu
Date: 2016-12-01 00:00:00

In the Markov decision process model, policies are usually evaluated by
expected cumulative rewards. As this decision criterion is not always suitable,
we propose in this paper an algorithm for computing a policy optimal for the
quantile criterion. Both finite and infinite horizons are considered. Finally
we experimentally evaluate our approach on random MDPs and on a data center
control problem.
Are Accuracy and Robustness Correlated?
Authors: Andras Rozsa, Manuel Günther, Terrance E. Boult
Date: 2016-10-14 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Object Recognition

Machine learning models are vulnerable to adversarial examples formed by
applying small carefully chosen perturbations to inputs that cause unexpected
classification errors. In this paper, we perform experiments on various
adversarial example generation approaches with multiple deep convolutional
neural networks including Residual Networks, the best performing models on
ImageNet Large-Scale Visual Recognition Challenge 2015. We compare the
adversarial example generation techniques with respect to the quality of the
produced images, and measure the robustness of the tested machine learning
models to adversarial examples. Finally, we conduct large-scale experiments on
cross-model adversarial portability. We find that adversarial examples are
mostly transferable across similar network topologies, and we demonstrate that
better machine learning models are less vulnerable to adversarial examples.
Texture Enhancement via High-Resolution Style Transfer for Single-Image Super-Resolution
Authors: Il Jun Ahn, Woo Hyun Nam
Date: 2016-12-01 00:00:00
Tasks: Image Super-Resolution, Style Transfer, Super-Resolution, Vocal Bursts Intensity Prediction

Recently, various deep-neural-network (DNN)-based approaches have been
proposed for single-image super-resolution (SISR). Despite their promising
results on major structure regions such as edges and lines, they still suffer
from limited performance on texture regions that consist of very complex and
fine patterns. This is because, during the acquisition of a low-resolution (LR)
image via down-sampling, these regions lose most of the high frequency
information necessary to represent the texture details. In this paper, we
present a novel texture enhancement framework for SISR to effectively improve
the spatial resolution in the texture regions as well as edges and lines. We
call our method, high-resolution (HR) style transfer algorithm. Our framework
consists of three steps: (i) generate an initial HR image from an interpolated
LR image via an SISR algorithm, (ii) generate an HR style image from the
initial HR image via down-scaling and tiling, and (iii) combine the HR style
image with the initial HR image via a customized style transfer algorithm.
Here, the HR style image is obtained by down-scaling the initial HR image and
then repetitively tiling it into an image of the same size as the HR image.
This down-scaling and tiling process comes from the idea that texture regions
are often composed of small regions that similar in appearance albeit sometimes
different in scale. This process creates an HR style image that is rich in
details, which can be used to restore high-frequency texture details back into
the initial HR image via the style transfer algorithm. Experimental results on
a number of texture datasets show that our proposed HR style transfer algorithm
provides more visually pleasing results compared with competitive methods.
Non-Convex Projected Gradient Descent for Generalized Low-Rank Tensor Regression
Authors: Han Chen, Garvesh Raskutti, Ming Yuan
Date: 2016-11-30 00:00:00
Tasks: regression

In this paper, we consider the problem of learning high-dimensional tensor
regression problems with low-rank structure. One of the core challenges
associated with learning high-dimensional models is computation since the
underlying optimization problems are often non-convex. While convex relaxations
could lead to polynomial-time algorithms they are often slow in practice. On
the other hand, limited theoretical guarantees exist for non-convex methods. In
this paper we provide a general framework that provides theoretical guarantees
for learning high-dimensional tensor regression models under different low-rank
structural assumptions using the projected gradient descent algorithm applied
to a potentially non-convex constraint set $\Theta$ in terms of its
\emph{localized Gaussian width}. We juxtapose our theoretical results for
non-convex projected gradient descent algorithms with previous results on
regularized convex approaches. The two main differences between the convex and
non-convex approach are: (i) from a computational perspective whether the
non-convex projection operator is computable and whether the projection has
desirable contraction properties and (ii) from a statistical upper bound
perspective, the non-convex approach has a superior rate for a number of
examples. We provide three concrete examples of low-dimensional structure which
address these issues and explain the pros and cons for the non-convex and
convex approaches. We supplement our theoretical results with simulations which
show that, under several common settings of generalized low rank tensor
regression, the projected gradient descent approach is superior both in terms
of statistical error and run-time provided the step-sizes of the projected
descent algorithm are suitably chosen.
Co-adaptive learning over a countable space
Authors: Michael Rabadi
Date: 2016-11-29 00:00:00
Tasks: Decoder, Recommendation Systems

Co-adaptation is a special form of on-line learning where an algorithm
$\mathcal{A}$ must assist an unknown algorithm $\mathcal{B}$ to perform some
task. This is a general framework and has applications in recommendation
systems, search, education, and much more. Today, the most common use of
co-adaptive algorithms is in brain-computer interfacing (BCI), where algorithms
help patients gain and maintain control over prosthetic devices. While previous
studies have shown strong empirical results Kowalski et al. (2013); Orsborn et
al. (2014) or have been studied in specific examples Merel et al. (2013, 2015),
there is no general analysis of the co-adaptive learning problem. Here we will
study the co-adaptive learning problem in the online, closed-loop setting. We
will prove that, with high probability, co-adaptive learning is guaranteed to
outperform learning with a fixed decoder as long as a particular condition is
met.
SLA Violation Prediction In Cloud Computing: A Machine Learning Perspective
Authors: Reyhane Askari Hemmat, Abdelhakim Hafid
Date: 2016-11-30 00:00:00
Tasks: BIG-bench Machine Learning, Cloud Computing, General Classification

Service level agreement (SLA) is an essential part of cloud systems to ensure
maximum availability of services for customers. With a violation of SLA, the
provider has to pay penalties. In this paper, we explore two machine learning
models: Naive Bayes and Random Forest Classifiers to predict SLA violations.
Since SLA violations are a rare event in the real world (~0.2 %), the
classification task becomes more challenging. In order to overcome these
challenges, we use several re-sampling methods. We find that random forests
with SMOTE-ENN re-sampling have the best performance among other methods with
the accuracy of 99.88 % and F_1 score of 0.9980.