pwc-archive/papers-with-abstracts huggingface dataset:
Some techniques in density estimation
Authors: Hassan Ashtiani, Abbas Mehrabian
Date: 2018-01-11 00:00:00
Tasks: BIG-bench Machine Learning, Density Estimation

Density estimation is an interdisciplinary topic at the intersection of
statistics, theoretical computer science and machine learning. We review some
old and new techniques for bounding the sample complexity of estimating
densities of continuous distributions, focusing on the class of mixtures of
Gaussians and its subclasses. In particular, we review the main techniques used
to prove the new sample complexity bounds for mixtures of Gaussians by
Ashtiani, Ben-David, Harvey, Liaw, Mehrabian, and Plan arXiv:1710.05209.
Parallelizing Linear Recurrent Neural Nets Over Sequence Length
Authors: Eric Martin, Chris Cundy
Date: 2017-09-12 00:00:00
Proceeding: ICLR 2018 1

Recurrent neural networks (RNNs) are widely used to model sequential data but
their non-linear dependencies between sequence elements prevent parallelizing
training over sequence length. We show the training of RNNs with only linear
sequential dependencies can be parallelized over the sequence length using the
parallel scan algorithm, leading to rapid training on long sequences even with
small minibatch size. We develop a parallel linear recurrence CUDA kernel and
show that it can be applied to immediately speed up training and inference of
several state of the art RNN architectures by up to 9x. We abstract recent work
on linear RNNs into a new framework of linear surrogate RNNs and develop a
linear surrogate model for the long short-term memory unit, the GILR-LSTM, that
utilizes parallel linear recurrence. We extend sequence learning to new
extremely long sequence regimes that were previously out of reach by
successfully training a GILR-LSTM on a synthetic sequence classification task
with a one million timestep dependency.
Training Probabilistic Spiking Neural Networks with First-to-spike Decoding
Authors: Alireza Bagheri, Osvaldo Simeone, Bipin Rajendran
Date: 2017-10-29 00:00:00
Tasks: Classification, Early  Classification, General Classification

Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at
harnessing the energy efficiency of spike-domain processing by building on
computing elements that operate on, and exchange, spikes. In this paper, the
problem of training a two-layer SNN is studied for the purpose of
classification, under a Generalized Linear Model (GLM) probabilistic neural
model that was previously considered within the computational neuroscience
literature. Conventional classification rules for SNNs operate offline based on
the number of output spikes at each output neuron. In contrast, a novel
training method is proposed here for a first-to-spike decoding rule, whereby
the SNN can perform an early classification decision once spike firing is
detected at an output neuron. Numerical results bring insights into the optimal
parameter selection for the GLM neuron and on the accuracy-complexity trade-off
performance of conventional and first-to-spike decoding.
FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference
Authors: Tianyu Wang, Marco Morucci, M. Usaid Awan, Yameng Liu, Sudeepa Roy, Cynthia Rudin, Alexander Volfovsky
Date: 2017-07-19 00:00:00
Tasks: Causal Inference, Management

A classical problem in causal inference is that of matching, where treatment units need to be matched to control units based on covariate information. In this work, we propose a method that computes high quality almost-exact matches for high-dimensional categorical datasets. This method, called FLAME (Fast Large-scale Almost Matching Exactly), learns a distance metric for matching using a hold-out training data set. In order to perform matching efficiently for large datasets, FLAME leverages techniques that are natural for query processing in the area of database management, and two implementations of FLAME are provided: the first uses SQL queries and the second uses bit-vector techniques. The algorithm starts by constructing matches of the highest quality (exact matches on all covariates), and successively eliminates variables in order to match exactly on as many variables as possible, while still maintaining interpretable high-quality matches and balance between treatment and control groups. We leverage these high quality matches to estimate conditional average treatment effects (CATEs). Our experiments show that FLAME scales to huge datasets with millions of observations where existing state-of-the-art methods fail, and that it achieves significantly better performance than other matching methods.
Guide Actor-Critic for Continuous Control
Authors: Voot Tangkaratt, Abbas Abdolmaleki, Masashi Sugiyama
Date: 2017-05-22 00:00:00
Proceeding: ICLR 2018 1
Tasks: continuous-control, Continuous Control, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Actor-critic methods solve reinforcement learning problems by updating a
parameterized policy known as an actor in a direction that increases an
estimate of the expected return known as a critic. However, existing
actor-critic methods only use values or gradients of the critic to update the
policy parameter. In this paper, we propose a novel actor-critic method called
the guide actor-critic (GAC). GAC firstly learns a guide actor that locally
maximizes the critic and then it updates the policy parameter based on the
guide actor by supervised learning. Our main theoretical contributions are two
folds. First, we show that GAC updates the guide actor by performing
second-order optimization in the action space where the curvature matrix is
based on the Hessians of the critic. Second, we show that the deterministic
policy gradient method is a special case of GAC when the Hessians are ignored.
Through experiments, we show that our method is a promising reinforcement
learning method for continuous controls.
Deep Unsupervised Learning of Visual Similarities
Authors: Artsiom Sanakoyeu, Miguel A. Bautista, Björn Ommer
Date: 2018-02-22 00:00:00

Exemplar learning of visual similarities in an unsupervised manner is a
problem of paramount importance to Computer Vision. In this context, however,
the recent breakthrough in deep learning could not yet unfold its full
potential. With only a single positive sample, a great imbalance between one
positive and many negatives, and unreliable relationships between most samples,
training of Convolutional Neural networks is impaired. In this paper we use
weak estimates of local similarities and propose a single optimization problem
to extract batches of samples with mutually consistent relations. Conflicting
relations are distributed over different batches and similar samples are
grouped into compact groups. Learning visual similarities is then framed as a
sequence of categorization tasks. The CNN then consolidates transitivity
relations within and between groups and learns a single representation for all
samples without the need for labels. The proposed unsupervised approach has
shown competitive performance on detailed posture analysis and object
classification.
Adversarial Occlusion-aware Face Detection
Authors: Yujia Chen, Lingxiao Song, Ran He
Date: 2017-09-15 00:00:00
Tasks: Face Detection, Occluded Face Detection

Occluded face detection is a challenging detection task due to the large
appearance variations incurred by various real-world occlusions. This paper
introduces an Adversarial Occlusion-aware Face Detector (AOFD) by
simultaneously detecting occluded faces and segmenting occluded areas.
Specifically, we employ an adversarial training strategy to generate
occlusion-like face features that are difficult for a face detector to
recognize. Occlusion mask is predicted simultaneously while detecting occluded
faces and the occluded area is utilized as an auxiliary instead of being
regarded as a hindrance. Moreover, the supervisory signals from the
segmentation branch will reversely affect the features, aiding in detecting
heavily-occluded faces accordingly. Consequently, AOFD is able to find the
faces with few exposed facial landmarks with very high confidences and keeps
high detection accuracy even for masked faces. Extensive experiments
demonstrate that AOFD not only significantly outperforms state-of-the-art
methods on the MAFA occluded face detection dataset, but also achieves
competitive detection accuracy on benchmark dataset for general face detection
such as FDDB.
Entropy Rate Estimation for Markov Chains with Large State Space
Authors: Yanjun Han, Jiantao Jiao, Chuan-Zheng Lee, Tsachy Weissman, Yihong Wu, Tiancheng Yu
Date: 2018-02-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Language Modeling, Language Modelling

Estimating the entropy based on data is one of the prototypical problems in
distribution property testing and estimation. For estimating the Shannon
entropy of a distribution on $S$ elements with independent samples,
[Paninski2004] showed that the sample complexity is sublinear in $S$, and
[Valiant--Valiant2011] showed that consistent estimation of Shannon entropy is
possible if and only if the sample size $n$ far exceeds $\frac{S}{\log S}$. In
this paper we consider the problem of estimating the entropy rate of a
stationary reversible Markov chain with $S$ states from a sample path of $n$
observations. We show that:
  (1) As long as the Markov chain mixes not too slowly, i.e., the relaxation
time is at most $O(\frac{S}{\ln^3 S})$, consistent estimation is achievable
when $n \gg \frac{S^2}{\log S}$.
  (2) As long as the Markov chain has some slight dependency, i.e., the
relaxation time is at least $1+\Omega(\frac{\ln^2 S}{\sqrt{S}})$, consistent
estimation is impossible when $n \lesssim \frac{S^2}{\log S}$.
  Under both assumptions, the optimal estimation accuracy is shown to be
$\Theta(\frac{S^2}{n \log S})$. In comparison, the empirical entropy rate
requires at least $\Omega(S^2)$ samples to be consistent, even when the Markov
chain is memoryless. In addition to synthetic experiments, we also apply the
estimators that achieve the optimal sample complexity to estimate the entropy
rate of the English language in the Penn Treebank and the Google One Billion
Words corpora, which provides a natural benchmark for language modeling and
relates it directly to the widely used perplexity measure.
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
Authors: Jiantao Jiao, Weihao Gao, Yanjun Han
Date: 2017-11-23 00:00:00
Proceeding: NeurIPS 2018 12

We analyze the Kozachenko--Leonenko (KL) nearest neighbor estimator for the
differential entropy. We obtain the first uniform upper bound on its
performance over H\"older balls on a torus without assuming any conditions on
how close the density could be from zero. Accompanying a new minimax lower
bound over the H\"older ball, we show that the KL estimator is achieving the
minimax rates up to logarithmic factors without cognizance of the smoothness
parameter $s$ of the H\"older ball for $s\in (0,2]$ and arbitrary dimension
$d$, rendering it the first estimator that provably satisfies this property.
Federated Meta-Learning with Fast Convergence and Efficient Communication
Authors: Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, Xiuqiang He
Date: 2018-02-22 00:00:00
Tasks: Federated Learning, Meta-Learning, Recommendation Systems

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.
Unsupervised Pretraining for Sequence to Sequence Learning
Authors: Prajit Ramachandran, Peter J. Liu, Quoc V. Le
Date: 2016-11-08 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Abstractive Text Summarization, Decoder, Machine Translation, Translation

This work presents a general unsupervised learning method to improve the
accuracy of sequence to sequence (seq2seq) models. In our method, the weights
of the encoder and decoder of a seq2seq model are initialized with the
pretrained weights of two language models and then fine-tuned with labeled
data. We apply this method to challenging benchmarks in machine translation and
abstractive summarization and find that it significantly improves the
subsequent supervised models. Our main result is that pretraining improves the
generalization of seq2seq models. We achieve state-of-the art results on the
WMT English$\rightarrow$German task, surpassing a range of methods using both
phrase-based machine translation and neural machine translation. Our method
achieves a significant improvement of 1.3 BLEU from the previous best models on
both WMT'14 and WMT'15 English$\rightarrow$German. We also conduct human
evaluations on abstractive summarization and find that our method outperforms a
purely supervised learning baseline in a statistically significant manner.
Multi-Sensor Integration for Indoor 3D Reconstruction
Authors: Jacky C. K. Chow
Date: 2018-02-22 00:00:00
Tasks: 3D Reconstruction

Outdoor maps and navigation information delivered by modern services and
technologies like Google Maps and Garmin navigators have revolutionized the
lifestyle of many people. Motivated by the desire for similar navigation
systems for indoor usage from consumers, advertisers, emergency
rescuers/responders, etc., many indoor environments such as shopping malls,
museums, casinos, airports, transit stations, offices, and schools need to be
mapped. Typically, the environment is first reconstructed by capturing many
point clouds from various stations and defining their spatial relationships.
Currently, there is a lack of an accurate, rigorous, and speedy method for
relating point clouds in indoor, urban, satellite-denied environments. This
thesis presents a novel and automatic way for fusing calibrated point clouds
obtained using a terrestrial laser scanner and the Microsoft Kinect by
integrating them with a low-cost inertial measurement unit. The developed
system, titled the Scannect, is the first joint static-kinematic indoor 3D
mapper.
Multimodal Named Entity Recognition for Short Social Media Posts
Authors: Seungwhan Moon, Leonardo Neves, Vitor Carvalho
Date: 2018-02-22 00:00:00
Proceeding: NAACL 2018 6
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

We introduce a new task called Multimodal Named Entity Recognition (MNER) for
noisy user-generated data such as tweets or Snapchat captions, which comprise
short text with accompanying images. These social media posts often come in
inconsistent or incomplete syntax and lexical notations with very limited
surrounding textual contexts, bringing significant challenges for NER. To this
end, we create a new dataset for MNER called SnapCaptions (Snapchat
image-caption pairs submitted to public and crowd-sourced stories with fully
annotated named entities). We then build upon the state-of-the-art Bi-LSTM
word/character based NER models with 1) a deep image network which incorporates
relevant visual context to augment textual information, and 2) a generic
modality-attention module which learns to attenuate irrelevant modalities while
amplifying the most informative ones to extract contexts from, adaptive to each
sample and token. The proposed MNER model with modality attention significantly
outperforms the state-of-the-art text-only NER models by successfully
leveraging provided visual contexts, opening up potential applications of MNER
on myriads of social media platforms.
Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics
Authors: Arindam Jati, Panayiotis Georgiou
Date: 2018-02-22 00:00:00
Tasks: Speaker Identification, Speaker Recognition, Speaker Verification, speech-recognition, Speech Recognition

Learning speaker-specific features is vital in many applications like speaker
recognition, diarization and speech recognition. This paper provides a novel
approach, we term Neural Predictive Coding (NPC), to learn speaker-specific
characteristics in a completely unsupervised manner from large amounts of
unlabeled training data that even contain many non-speech events and
multi-speaker audio streams. The NPC framework exploits the proposed short-term
active-speaker stationarity hypothesis which assumes two temporally-close short
speech segments belong to the same speaker, and thus a common representation
that can encode the commonalities of both the segments, should capture the
vocal characteristics of that speaker. We train a convolutional deep siamese
network to produce "speaker embeddings" by learning to separate `same' vs
`different' speaker pairs which are generated from an unlabeled data of audio
streams. Two sets of experiments are done in different scenarios to evaluate
the strength of NPC embeddings and compare with state-of-the-art in-domain
supervised methods. First, two speaker identification experiments with
different context lengths are performed in a scenario with comparatively
limited within-speaker channel variability. NPC embeddings are found to perform
the best at short duration experiment, and they provide complementary
information to i-vectors for full utterance experiments. Second, a large scale
speaker verification task having a wide range of within-speaker channel
variability is adopted as an upper-bound experiment where comparisons are drawn
with in-domain supervised methods.
Modeling Spatiotemporal Factors Associated With Sentiment on Twitter: Synthesis and Suggestions for Improving the Identification of Localized Deviations
Authors: Zubair Shah, Paige Martin, Enrico Coiera, Kenneth D. Mandl, Adam G. Dunn
Date: 2018-02-22 00:00:00
Tasks: Sentiment Analysis

Background: Studies examining how sentiment on social media varies depending on timing and location appear to produce inconsistent results, making it hard to design systems that use sentiment to detect localized events for public health applications. Objective: The aim of this study was to measure how common timing and location confounders explain variation in sentiment on Twitter. Methods: Using a dataset of 16.54 million English-language tweets from 100 cities posted between July 13 and November 30, 2017, we estimated the positive and negative sentiment for each of the cities using a dictionary-based sentiment analysis and constructed models to explain the differences in sentiment using time of day, day of week, weather, city, and interaction type (conversations or broadcasting) as factors and found that all factors were independently associated with sentiment. Results: In the full multivariable model of positive (Pearson r in test data 0.236; 95\% CI 0.231-0.241) and negative (Pearson r in test data 0.306; 95\% CI 0.301-0.310) sentiment, the city and time of day explained more of the variance than weather and day of week. Models that account for these confounders produce a different distribution and ranking of important events compared with models that do not account for these confounders. Conclusions: In public health applications that aim to detect localized events by aggregating sentiment across populations of Twitter users, it is worthwhile accounting for baseline differences before looking for unexpected changes.
xView: Objects in Context in Overhead Imagery
Authors: Darius Lam, Richard Kuzma, Kevin McGee, Samuel Dooley, Michael Laielli, Matthew Klaric, Yaroslav Bulatov, Brendan McCord
Date: 2018-02-22 00:00:00
Tasks: Object, object-detection, Object Detection, Object Detection In Aerial Images

We introduce a new large-scale dataset for the advancement of object
detection techniques and overhead object detection research. This satellite
imagery dataset enables research progress pertaining to four key computer
vision frontiers. We utilize a novel process for geospatial category detection
and bounding box annotation with three stages of quality control. Our data is
collected from WorldView-3 satellites at 0.3m ground sample distance, providing
higher resolution imagery than most public satellite imagery datasets. We
compare xView to other object detection datasets in both natural and overhead
imagery domains and then provide a baseline analysis using the Single Shot
MultiBox Detector. xView is one of the largest and most diverse publicly
available object-detection datasets to date, with over 1 million objects across
60 classes in over 1,400 km^2 of imagery.
Driver Hand Localization and Grasp Analysis: A Vision-based Real-time Approach
Authors: Siddharth, Akshay Rangesh, Eshed Ohn-Bar, Mohan M. Trivedi
Date: 2018-02-22 00:00:00
Tasks: Hand Detection

Extracting hand regions and their grasp information from images robustly in
real-time is critical for occupants' safety and in-vehicular infotainment
applications. It must however, be noted that naturalistic driving scenes suffer
from rapidly changing illumination and occlusion. This is aggravated by the
fact that hands are highly deformable objects, and change in appearance
frequently. This work addresses the task of accurately localizing driver hands
and classifying the grasp state of each hand. We use a fast ConvNet to first
detect likely hand regions. Next, a pixel-based skin classifier that takes into
account the global illumination changes is used to refine the hand detections
and remove false positives. This step generates a pixel-level mask for each
hand. Finally, we study each such masked regions and detect if the driver is
grasping the wheel, or in some cases a mobile phone. Through evaluation we
demonstrate that our method can outperform state-of-the-art pixel based hand
detectors, while running faster (at 35 fps) than other deep ConvNet based
frameworks even for grasp analysis. Hand mask cues are shown to be crucial when
analyzing a set of driver hand gestures (wheel/mobile phone grasp and no-grasp)
in naturalistic driving settings. The proposed detection and localization
pipeline hence can act as a general framework for real-time hand detection and
gesture classification.
Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detection
Authors: Avi Ben-Cohen, Eyal Klang, Stephen P. Raskin, Shelly Soffer, Simona Ben-Haim, Eli Konen, Michal Marianne Amitai, Hayit Greenspan
Date: 2018-02-21 00:00:00
Tasks: Generative Adversarial Network, Lesion Detection

In this work we present a novel system for generation of virtual PET images
using CT scans. We combine a fully convolutional network (FCN) with a
conditional generative adversarial network (GAN) to generate simulated PET data
from given input CT data. The synthesized PET can be used for false-positive
reduction in lesion detection solutions. Clinically, such solutions may enable
lesion detection and drug treatment evaluation in a CT-only environment, thus
reducing the need for the more expensive and radioactive PET/CT scan. Our
dataset includes 60 PET/CT scans from Sheba Medical center. We used 23 scans
for training and 37 for testing. Different schemes to achieve the synthesized
output were qualitatively compared. Quantitative evaluation was conducted using
an existing lesion detection software, combining the synthesized PET as a false
positive reduction layer for the detection of malignant lesions in the liver.
Current results look promising showing a 28% reduction in the average false
positive per case from 2.9 to 2.1. The suggested solution is comprehensive and
can be expanded to additional body organs, and different modalities.