pwc-archive/papers-with-abstracts huggingface dataset:
Clustering Semi-Random Mixtures of Gaussians
Authors: Pranjal Awasthi, Aravindan Vijayaraghavan
Date: 2017-11-23 00:00:00
Proceeding: ICML 2018 7
Tasks: Clustering

Gaussian mixture models (GMM) are the most widely used statistical model for
the $k$-means clustering problem and form a popular framework for clustering in
machine learning and data analysis. In this paper, we propose a natural
semi-random model for $k$-means clustering that generalizes the Gaussian
mixture model, and that we believe will be useful in identifying robust
algorithms. In our model, a semi-random adversary is allowed to make arbitrary
"monotone" or helpful changes to the data generated from the Gaussian mixture
model.
  Our first contribution is a polynomial time algorithm that provably recovers
the ground-truth up to small classification error w.h.p., assuming certain
separation between the components. Perhaps surprisingly, the algorithm we
analyze is the popular Lloyd's algorithm for $k$-means clustering that is the
method-of-choice in practice. Our second result complements the upper bound by
giving a nearly matching information-theoretic lower bound on the number of
misclassified points incurred by any $k$-means clustering algorithm on the
semi-random model.
Empower Sequence Labeling with Task-Aware Neural Language Model
Authors: Liyuan Liu, Jingbo Shang, Frank F. Xu, Xiang Ren, Huan Gui, Jian Peng, Jiawei Han
Date: 2017-09-13 00:00:00
Tasks: GPU, Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Part-Of-Speech Tagging, Transfer Learning, Word Embeddings

Linguistic sequence labeling is a general modeling approach that encompasses
a variety of problems, such as part-of-speech tagging and named entity
recognition. Recent advances in neural networks (NNs) make it possible to build
reliable models without handcrafted features. However, in many cases, it is
hard to obtain sufficient annotations to train these models. In this study, we
develop a novel neural framework to extract abundant knowledge hidden in raw
texts to empower the sequence labeling task. Besides word-level knowledge
contained in pre-trained word embeddings, character-aware neural language
models are incorporated to extract character-level knowledge. Transfer learning
techniques are further adopted to mediate different components and guide the
language model towards the key knowledge. Comparing to previous methods, these
task-specific knowledge allows us to adopt a more concise model and conduct
more efficient training. Different from most transfer learning methods, the
proposed framework does not rely on any additional supervision. It extracts
knowledge from self-contained order information of training sequences.
Extensive experiments on benchmark datasets demonstrate the effectiveness of
leveraging character-level knowledge and the efficiency of co-training. For
example, on the CoNLL03 NER task, model training completes in about 6 hours on
a single GPU, reaching F1 score of 91.71$\pm$0.10 without using any extra
annotation.
Deep Learning for Real-Time Crime Forecasting and its Ternarization
Authors: Bao Wang, Penghang Yin, Andrea L. Bertozzi, P. Jeffrey Brantingham, Stanley J. Osher, Jack Xin
Date: 2017-11-23 00:00:00
Tasks: Deep Learning

Real-time crime forecasting is important. However, accurate prediction of
when and where the next crime will happen is difficult. No known physical model
provides a reasonable approximation to such a complex system. Historical crime
data are sparse in both space and time and the signal of interests is weak. In
this work, we first present a proper representation of crime data. We then
adapt the spatial temporal residual network on the well represented data to
predict the distribution of crime in Los Angeles at the scale of hours in
neighborhood-sized parcels. These experiments as well as comparisons with
several existing approaches to prediction demonstrate the superiority of the
proposed model in terms of accuracy. Finally, we present a ternarization
technique to address the resource consumption issue for its deployment in real
world. This work is an extension of our short conference proceeding paper [Wang
et al, Arxiv 1707.03340].
A Dictionary Approach to Identifying Transient RFI
Authors: Daniel Czech, Amit Mishra, Michael Inggs
Date: 2017-11-23 00:00:00

As radio telescopes become more sensitive, the damaging effects of radio
frequency interference (RFI) become more apparent. Near radio telescope arrays,
RFI sources are often easily removed or replaced; the challenge lies in
identifying them. Transient (impulsive) RFI is particularly difficult to
identify. We propose a novel dictionary-based approach to transient RFI
identification. RFI events are treated as sequences of sub-events, drawn from
particular labelled classes. We demonstrate an automated method of extracting
and labelling sub-events using a dataset of transient RFI. A dictionary of
labels may be used in conjunction with hidden Markov models to identify the
sources of RFI events reliably. We attain improved classification accuracy over
traditional approaches such as SVMs or a na\"ive kNN classifier. Finally, we
investigate why transient RFI is difficult to classify. We show that cluster
separation in the principal components domain is influenced by the mains supply
phase for certain sources.
Measurement-Adaptive Sparse Image Sampling and Recovery
Authors: Ali Taimori, Farokh Marvasti
Date: 2017-06-09 00:00:00
Tasks: Compressive Sensing

This paper presents an adaptive and intelligent sparse model for digital
image sampling and recovery. In the proposed sampler, we adaptively determine
the number of required samples for retrieving image based on
space-frequency-gradient information content of image patches. By leveraging
texture in space, sparsity locations in DCT domain, and directional
decomposition of gradients, the sampler structure consists of a combination of
uniform, random, and nonuniform sampling strategies. For reconstruction, we
model the recovery problem as a two-state cellular automaton to iteratively
restore image with scalable windows from generation to generation. We
demonstrate the recovery algorithm quickly converges after a few generations
for an image with arbitrary degree of texture. For a given number of
measurements, extensive experiments on standard image-sets, infra-red, and
mega-pixel range imaging devices show that the proposed measurement matrix
considerably increases the overall recovery performance, or equivalently
decreases the number of sampled pixels for a specific recovery quality compared
to random sampling matrix and Gaussian linear combinations employed by the
state-of-the-art compressive sensing methods. In practice, the proposed
measurement-adaptive sampling/recovery framework includes various applications
from intelligent compressive imaging-based acquisition devices to computer
vision and graphics, and image processing technology. Simulation codes are
available online for reproduction purposes.
Controlling Physical Attributes in GAN-Accelerated Simulation of Electromagnetic Calorimeters
Authors: Luke de Oliveira, Michela Paganini, Benjamin Nachman
Date: 2017-11-23 00:00:00
Tasks: Attribute, Generative Adversarial Network

High-precision modeling of subatomic particle interactions is critical for
many fields within the physical sciences, such as nuclear physics and high
energy particle physics. Most simulation pipelines in the sciences are
computationally intensive -- in a variety of scientific fields, Generative
Adversarial Networks have been suggested as a solution to speed up the forward
component of simulation, with promising results. An important component of any
simulation system for the sciences is the ability to condition on any number of
physically meaningful latent characteristics that can effect the forward
generation procedure. We introduce an auxiliary task to the training of a
Generative Adversarial Network on particle showers in a multi-layer
electromagnetic calorimeter, which allows our model to learn an attribute-aware
conditioning mechanism.
Machine Learning Algorithms for $b$-Jet Tagging at the ATLAS Experiment
Authors: Michela Paganini
Date: 2017-11-23 00:00:00
Tasks: BIG-bench Machine Learning, Jet Tagging

The separation of $b$-quark initiated jets from those coming from lighter
quark flavors ($b$-tagging) is a fundamental tool for the ATLAS physics program
at the CERN Large Hadron Collider. The most powerful $b$-tagging algorithms
combine information from low-level taggers, exploiting reconstructed track and
vertex information, into machine learning classifiers. The potential of modern
deep learning techniques is explored using simulated events, and compared to
that achievable from more traditional classifiers such as boosted decision
trees.
Learning to Predict with Highly Granular Temporal Data: Estimating individual behavioral profiles with smart meter data
Authors: Anastasia Ushakova, Slava J. Mikhaylov
Date: 2017-11-15 00:00:00
Tasks: Time Series, Time Series Analysis

Big spatio-temporal datasets, available through both open and administrative
data sources, offer significant potential for social science research. The
magnitude of the data allows for increased resolution and analysis at
individual level. While there are recent advances in forecasting techniques for
highly granular temporal data, little attention is given to segmenting the time
series and finding homogeneous patterns. In this paper, it is proposed to
estimate behavioral profiles of individuals' activities over time using
Gaussian Process-based models. In particular, the aim is to investigate how
individuals or groups may be clustered according to the model parameters. Such
a Bayesian non-parametric method is then tested by looking at the
predictability of the segments using a combination of models to fit different
parts of the temporal profiles. Model validity is then tested on a set of
holdout data. The dataset consists of half hourly energy consumption records
from smart meters from more than 100,000 households in the UK and covers the
period from 2015 to 2016. The methodological approach developed in the paper
may be easily applied to datasets of similar structure and granularity, for
example social media data, and may lead to improved accuracy in the prediction
of social dynamics and behavior.
Practical Hash Functions for Similarity Estimation and Dimensionality Reduction
Authors: Søren Dahlgaard, Mathias Bæk Tejs Knudsen, Mikkel Thorup
Date: 2017-11-23 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Dimensionality Reduction

Hashing is a basic tool for dimensionality reduction employed in several
aspects of machine learning. However, the perfomance analysis is often carried
out under the abstract assumption that a truly random unit cost hash function
is used, without concern for which concrete hash function is employed. The
concrete hash function may work fine on sufficiently random input. The question
is if it can be trusted in the real world when faced with more structured
input.
  In this paper we focus on two prominent applications of hashing, namely
similarity estimation with the one permutation hashing (OPH) scheme of Li et
al. [NIPS'12] and feature hashing (FH) of Weinberger et al. [ICML'09], both of
which have found numerous applications, i.e. in approximate near-neighbour
search with LSH and large-scale classification with SVM.
  We consider mixed tabulation hashing of Dahlgaard et al.[FOCS'15] which was
proved to perform like a truly random hash function in many applications,
including OPH. Here we first show improved concentration bounds for FH with
truly random hashing and then argue that mixed tabulation performs similar for
sparse input. Our main contribution, however, is an experimental comparison of
different hashing schemes when used inside FH, OPH, and LSH.
  We find that mixed tabulation hashing is almost as fast as the
multiply-mod-prime scheme ax+b mod p. Mutiply-mod-prime is guaranteed to work
well on sufficiently random data, but we demonstrate that in the above
applications, it can lead to bias and poor concentration on both real-world and
synthetic data. We also compare with the popular MurmurHash3, which has no
proven guarantees. Mixed tabulation and MurmurHash3 both perform similar to
truly random hashing in our experiments. However, mixed tabulation is 40%
faster than MurmurHash3, and it has the proven guarantee of good performance on
all possible input.
SPINE: SParse Interpretable Neural Embeddings
Authors: Anant Subramanian, Danish Pruthi, Harsh Jhamtani, Taylor Berg-Kirkpatrick, Eduard Hovy
Date: 2017-11-23 00:00:00
Tasks: Denoising, Word Embeddings

Prediction without justification has limited utility. Much of the success of
neural models can be attributed to their ability to learn rich, dense and
expressive representations. While these representations capture the underlying
complexity and latent trends in the data, they are far from being
interpretable. We propose a novel variant of denoising k-sparse autoencoders
that generates highly efficient and interpretable distributed word
representations (word embeddings), beginning with existing word representations
from state-of-the-art methods like GloVe and word2vec. Through large scale
human evaluation, we report that our resulting word embedddings are much more
interpretable than the original GloVe and word2vec embeddings. Moreover, our
embeddings outperform existing popular word embeddings on a diverse suite of
benchmark downstream tasks.
A Genetic Algorithm Approach for ImageRepresentation Learning through Color Quantization
Authors: Érico M. Pereira, Ricardo da S. Torres, Jefersson A. dos Santos
Date: 2017-11-18 00:00:00
Tasks: Content-Based Image Retrieval, Image Description, Image Retrieval, Information Retrieval, Quantization, Representation Learning, Retrieval

Over the last decades, hand-crafted feature extractors have been used to encode image visual properties into feature vectors. Recently, data-driven feature learning approaches have been successfully explored as alternatives for producing more representative visual features. In this work, we combine both research venues, focusing on the color quantization problem. We propose two data-driven approaches to learn image representations through the search for optimized quantization schemes, which lead to more effective feature extraction algorithms and compact representations. Our strategy employs Genetic Algorithm, a soft-computing apparatus successfully utilized in Information-retrieval-related optimization problems. We hypothesize that changing the quantization affects the quality of image description approaches, leading to effective and efficient representations. We evaluate our approaches in content-based image retrieval tasks, considering eight well-known datasets with different visual properties. Results indicate that the approach focused on representation effectiveness outperformed baselines in all tested scenarios. The other approach, which also considers the size of created representations, produced competitive results keeping or even reducing the dimensionality of feature vectors up to 25%.
3D Based Landmark Tracker Using Superpixels Based Segmentation for Neuroscience and Biomechanics Studies
Authors: Omid Haji Maghsoudi, Andrew Spence
Date: 2017-11-23 00:00:00
Tasks: Position, Superpixels

Examining locomotion has improved our basic understanding of motor control
and aided in treating motor impairment. Mice and rats are premier models of
human disease and increasingly the model systems of choice for basic
neuroscience. High frame rates (250 Hz) are needed to quantify the kinematics
of these running rodents. Manual tracking, especially for multiple markers,
becomes time-consuming and impossible for large sample sizes. Therefore, the
need for automatic segmentation of these markers has grown in recent years.
Here, we address this need by presenting a method to segment the markers using
the SLIC superpixel method. The 2D coordinates on the image plane are projected
to a 3D domain using direct linear transform (DLT) and a 3D Kalman filter has
been used to predict the position of markers based on the speed and position of
markers from the previous frames. Finally, a probabilistic function is used to
find the best match among superpixels. The method is evaluated for different
difficulties for tracking of the markers and it achieves 95% correct labeling
of markers.
Diversity-Promoting Bayesian Learning of Latent Variable Models
Authors: Pengtao Xie, Jun Zhu, Eric P. Xing
Date: 2017-11-23 00:00:00
Tasks: Diversity, Mixture-of-Experts, Variational Inference

To address three important issues involved in latent variable models (LVMs),
including capturing infrequent patterns, achieving small-sized but expressive
models and alleviating overfitting, several studies have been devoted to
"diversifying" LVMs, which aim at encouraging the components in LVMs to be
diverse. Most existing studies fall into a frequentist-style regularization
framework, where the components are learned via point estimation. In this
paper, we investigate how to "diversify" LVMs in the paradigm of Bayesian
learning. We propose two approaches that have complementary advantages. One is
to define a diversity-promoting mutual angular prior which assigns larger
density to components with larger mutual angles and use this prior to affect
the posterior via Bayes' rule. We develop two efficient approximate posterior
inference algorithms based on variational inference and MCMC sampling. The
other approach is to impose diversity-promoting regularization directly over
the post-data distribution of components. We also extend our approach to
"diversify" Bayesian nonparametric models where the number of components is
infinite. A sampling algorithm based on slice sampling and Hamiltonian Monte
Carlo is developed. We apply these methods to "diversify" Bayesian mixture of
experts model and infinite latent feature model. Experiments on various
datasets demonstrate the effectiveness and efficiency of our methods.
DeepPainter: Painter Classification Using Deep Convolutional Autoencoders
Authors: Eli David, Nathan S. Netanyahu
Date: 2017-11-23 00:00:00
Tasks: Classification, General Classification

In this paper we describe the problem of painter classification, and propose
a novel approach based on deep convolutional autoencoder neural networks. While
previous approaches relied on image processing and manual feature extraction
from paintings, our approach operates on the raw pixel level, without any
preprocessing or manual feature extraction. We first train a deep convolutional
autoencoder on a dataset of paintings, and subsequently use it to initialize a
supervised convolutional neural network for the classification phase.
  The proposed approach substantially outperforms previous methods, improving
the previous state-of-the-art for the 3-painter classification problem from
90.44% accuracy (previous state-of-the-art) to 96.52% accuracy, i.e., a 63%
reduction in error rate.
Online and Batch Supervised Background Estimation via L1 Regression
Authors: Aritra Dutta, Peter Richtarik
Date: 2017-11-23 00:00:00
Tasks: regression

We propose a surprisingly simple model for supervised video background
estimation. Our model is based on $\ell_1$ regression. As existing methods for
$\ell_1$ regression do not scale to high-resolution videos, we propose several
simple and scalable methods for solving the problem, including iteratively
reweighted least squares, a homotopy method, and stochastic gradient descent.
We show through extensive experiments that our model and methods match or
outperform the state-of-the-art online and batch methods in virtually all
quantitative and qualitative measures.
DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem
Authors: Dror Sholomon, Eli David, Nathan S. Netanyahu
Date: 2017-11-23 00:00:00

This paper introduces the first deep neural network-based estimation metric
for the jigsaw puzzle problem. Given two puzzle piece edges, the neural network
predicts whether or not they should be adjacent in the correct assembly of the
puzzle, using nothing but the pixels of each piece. The proposed metric
exhibits an extremely high precision even though no manual feature extraction
is performed. When incorporated into an existing puzzle solver, the solution's
accuracy increases significantly, achieving thereby a new state-of-the-art
standard.
Data-Free Knowledge Distillation for Deep Neural Networks
Authors: Raphael Gontijo Lopes, Stefano Fenu, Thad Starner
Date: 2017-10-19 00:00:00
Tasks: Data-free Knowledge Distillation, Knowledge Distillation, Model Compression

Recent advances in model compression have provided procedures for compressing
large neural networks to a fraction of their original size while retaining most
if not all of their accuracy. However, all of these approaches rely on access
to the original training set, which might not always be possible if the network
to be compressed was trained on a very large dataset, or on a dataset whose
release poses privacy or safety concerns as may be the case for biometrics
tasks. We present a method for data-free knowledge distillation, which is able
to compress deep neural networks trained on large-scale datasets to a fraction
of their size leveraging only some extra metadata to be provided with a
pretrained model release. We also explore different kinds of metadata that can
be used with our method, and discuss tradeoffs involved in using each of them.
DeepSign: Deep Learning for Automatic Malware Signature Generation and Classification
Authors: Eli David, Nathan S. Netanyahu
Date: 2017-11-21 00:00:00
Tasks: Classification, Deep Learning, Denoising, General Classification, Malware Detection

This paper presents a novel deep learning based method for automatic malware
signature generation and classification. The method uses a deep belief network
(DBN), implemented with a deep stack of denoising autoencoders, generating an
invariant compact representation of the malware behavior. While conventional
signature and token based methods for malware detection do not detect a
majority of new variants for existing malware, the results presented in this
paper show that signatures generated by the DBN allow for an accurate
classification of new malware variants. Using a dataset containing hundreds of
variants for several major malware families, our method achieves 98.6%
classification accuracy using the signatures generated by the DBN. The
presented method is completely agnostic to the type of malware behavior that is
logged (e.g., API calls and their parameters, registry entries, websites and
ports accessed, etc.), and can use any raw input from a sandbox to successfully
train the deep neural network which is used to generate malware signatures.
Boosted Cascaded Convnets for Multilabel Classification of Thoracic Diseases in Chest Radiographs
Authors: Pulkit Kumar, Monika Grewal, Muktabh Mayank Srivastava
Date: 2017-11-23 00:00:00
Tasks: Deep Learning, General Classification, Lung Disease Classification

Chest X-ray is one of the most accessible medical imaging technique for
diagnosis of multiple diseases. With the availability of ChestX-ray14, which is
a massive dataset of chest X-ray images and provides annotations for 14
thoracic diseases; it is possible to train Deep Convolutional Neural Networks
(DCNN) to build Computer Aided Diagnosis (CAD) systems. In this work, we
experiment a set of deep learning models and present a cascaded deep neural
network that can diagnose all 14 pathologies better than the baseline and is
competitive with other published methods. Our work provides the quantitative
results to answer following research questions for the dataset: 1) What loss
functions to use for training DCNN from scratch on ChestX-ray14 dataset that
demonstrates high class imbalance and label co occurrence? 2) How to use
cascading to model label dependency and to improve accuracy of the deep
learning model?