pwc-archive/papers-with-abstracts huggingface dataset:
Hierarchical Clustering: Objective Functions and Algorithms
Authors: Vincent Cohen-Addad, Varun Kanade, Frederik Mallmann-Trenn, Claire Mathieu
Date: 2017-04-07 00:00:00
Tasks: Clustering, Combinatorial Optimization, Stochastic Block Model

Hierarchical clustering is a recursive partitioning of a dataset into
clusters at an increasingly finer granularity. Motivated by the fact that most
work on hierarchical clustering was based on providing algorithms, rather than
optimizing a specific objective, Dasgupta framed similarity-based hierarchical
clustering as a combinatorial optimization problem, where a `good' hierarchical
clustering is one that minimizes some cost function. He showed that this cost
function has certain desirable properties.
  We take an axiomatic approach to defining `good' objective functions for both
similarity and dissimilarity-based hierarchical clustering. We characterize a
set of "admissible" objective functions (that includes Dasgupta's one) that
have the property that when the input admits a `natural' hierarchical
clustering, it has an optimal value.
  Equipped with a suitable objective function, we analyze the performance of
practical algorithms, as well as develop better algorithms. For
similarity-based hierarchical clustering, Dasgupta showed that the divisive
sparsest-cut approach achieves an $O(\log^{3/2} n)$-approximation. We give a
refined analysis of the algorithm and show that it in fact achieves an
$O(\sqrt{\log n})$-approx. (Charikar and Chatziafratis independently proved
that it is a $O(\sqrt{\log n})$-approx.). This improves upon the LP-based
$O(\log n)$-approx. of Roy and Pokutta. For dissimilarity-based hierarchical
clustering, we show that the classic average-linkage algorithm gives a factor 2
approx., and provide a simple and better algorithm that gives a factor 3/2
approx..
  Finally, we consider `beyond-worst-case' scenario through a generalisation of
the stochastic block model for hierarchical clustering. We show that Dasgupta's
cost function has desirable properties for these inputs and we provide a simple
1 + o(1)-approximation in this setting.
Quantum ensembles of quantum classifiers
Authors: Maria Schuld, Francesco Petruccione
Date: 2017-04-07 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Medical Diagnosis, Quantum Machine Learning

Quantum machine learning witnesses an increasing amount of quantum algorithms
for data-driven decision making, a problem with potential applications ranging
from automated image recognition to medical diagnosis. Many of those algorithms
are implementations of quantum classifiers, or models for the classification of
data inputs with a quantum computer. Following the success of collective
decision making with ensembles in classical machine learning, this paper
introduces the concept of quantum ensembles of quantum classifiers. Creating
the ensemble corresponds to a state preparation routine, after which the
quantum classifiers are evaluated in parallel and their combined decision is
accessed by a single-qubit measurement. This framework naturally allows for
exponentially large ensembles in which -- similar to Bayesian learning -- the
individual classifiers do not have to be trained. As an example, we analyse an
exponentially large quantum ensemble in which each classifier is weighed
according to its performance in classifying the training data, leading to new
results for quantum as well as classical machine learning.
Optimal algorithms for smooth and strongly convex distributed optimization in networks
Authors: Kevin Scaman, Francis Bach, Sébastien Bubeck, Yin Tat Lee, Laurent Massoulié
Date: 2017-02-28 00:00:00
Proceeding: ICML 2017 8
Tasks: Distributed Optimization, regression

In this paper, we determine the optimal convergence rates for strongly convex
and smooth distributed optimization in two settings: centralized and
decentralized communications over a network. For centralized (i.e.
master/slave) algorithms, we show that distributing Nesterov's accelerated
gradient descent is optimal and achieves a precision $\varepsilon > 0$ in time
$O(\sqrt{\kappa_g}(1+\Delta\tau)\ln(1/\varepsilon))$, where $\kappa_g$ is the
condition number of the (global) function to optimize, $\Delta$ is the diameter
of the network, and $\tau$ (resp. $1$) is the time needed to communicate values
between two neighbors (resp. perform local computations). For decentralized
algorithms based on gossip, we provide the first optimal algorithm, called the
multi-step dual accelerated (MSDA) method, that achieves a precision
$\varepsilon > 0$ in time
$O(\sqrt{\kappa_l}(1+\frac{\tau}{\sqrt{\gamma}})\ln(1/\varepsilon))$, where
$\kappa_l$ is the condition number of the local functions and $\gamma$ is the
(normalized) eigengap of the gossip matrix used for communication between
nodes. We then verify the efficiency of MSDA against state-of-the-art methods
for two problems: least-squares regression and classification by logistic
regression.
Partial Face Detection in the Mobile Domain
Authors: Upal Mahbub, Sayantan Sarkar, Rama Chellappa
Date: 2017-04-07 00:00:00
Tasks: Binary Classification, Data Augmentation, Face Detection, General Classification, regression

Generic face detection algorithms do not perform well in the mobile domain
due to significant presence of occluded and partially visible faces. One
promising technique to handle the challenge of partial faces is to design face
detectors based on facial segments. In this paper two different approaches of
facial segment-based face detection are discussed, namely, proposal-based
detection and detection by end-to-end regression. Methods that follow the first
approach rely on generating face proposals that contain facial segment
information. The three detectors following this approach, namely Facial
Segment-based Face Detector (FSFD), SegFace and DeepSegFace, discussed in this
paper, perform binary classification on each proposal based on features learned
from facial segments. The process of proposal generation, however, needs to be
handled separately, which can be very time consuming, and is not truly
necessary given the nature of the active authentication problem. Hence a novel
algorithm, Deep Regression-based User Image Detector (DRUID) is proposed, which
shifts from the classification to the regression paradigm, thus obviating the
need for proposal generation. DRUID has an unique network architecture with
customized loss functions, is trained using a relatively small amount of data
by utilizing a novel data augmentation scheme and is fast since it outputs the
bounding boxes of a face and its segments in a single pass. Being robust to
occlusion by design, the facial segment-based face detection methods,
especially DRUID show superior performance over other state-of-the-art face
detectors in terms of precision-recall and ROC curve on two mobile face
datasets.
Improving content marketing processes with the approaches by artificial intelligence
Authors: Utku Kose, Selcuk Sert
Date: 2017-04-07 00:00:00
Tasks: Marketing

Content marketing is todays one of the most remarkable approaches in the
context of marketing processes of companies. Value of this kind of marketing
has improved in time, thanks to the latest developments regarding to computer
and communication technologies. Nowadays, especially social media based
platforms have a great importance on enabling companies to design multimedia
oriented, interactive content. But on the other hand, there is still something
more to do for improved content marketing approaches. In this context,
objective of this study is to focus on intelligent content marketing, which can
be done by using artificial intelligence. Artificial Intelligence is todays one
of the most remarkable research fields and it can be used easily as
multidisciplinary. So, this study has aimed to discuss about its potential on
improving content marketing. In detail, the study has enabled readers to
improve their awareness about the intersection point of content marketing and
artificial intelligence. Furthermore, the authors have introduced some example
models of intelligent content marketing, which can be achieved by using current
Web technologies and artificial intelligence techniques.
Learning from Noisy Labels with Distillation
Authors: Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, Li-Jia Li
Date: 2017-03-07 00:00:00
Proceeding: ICCV 2017 10

The ability of learning from noisy labels is very useful in many visual
recognition tasks, as a vast amount of data with noisy labels are relatively
easy to obtain. Traditionally, the label noises have been treated as
statistical outliers, and approaches such as importance re-weighting and
bootstrap have been proposed to alleviate the problem. According to our
observation, the real-world noisy labels exhibit multi-mode characteristics as
the true labels, rather than behaving like independent random outliers. In this
work, we propose a unified distillation framework to use side information,
including a small clean dataset and label relations in knowledge graph, to
"hedge the risk" of learning from noisy labels. Furthermore, unlike the
traditional approaches evaluated based on simulated label noises, we propose a
suite of new benchmark datasets, in Sports, Species and Artifacts domains, to
evaluate the task of learning from noisy labels in the practical setting. The
empirical study demonstrates the effectiveness of our proposed method in all
the domains.
Conceptualization Topic Modeling
Authors: Yi-Kun Tang, Xian-Ling Mao, He-Yan Huang, Guihua Wen
Date: 2017-04-07 00:00:00
Tasks: Topic Models

Recently, topic modeling has been widely used to discover the abstract topics
in text corpora. Most of the existing topic models are based on the assumption
of three-layer hierarchical Bayesian structure, i.e. each document is modeled
as a probability distribution over topics, and each topic is a probability
distribution over words. However, the assumption is not optimal. Intuitively,
it's more reasonable to assume that each topic is a probability distribution
over concepts, and then each concept is a probability distribution over words,
i.e. adding a latent concept layer between topic layer and word layer in
traditional three-layer assumption. In this paper, we verify the proposed
assumption by incorporating the new assumption in two representative topic
models, and obtain two novel topic models. Extensive experiments were conducted
among the proposed models and corresponding baselines, and the results show
that the proposed models significantly outperform the baselines in terms of
case study and perplexity, which means the new assumption is more reasonable
than traditional one.
Variance Reduction for Distributed Stochastic Gradient Descent
Authors: Soham De, Gavin Taylor, Tom Goldstein
Date: 2015-12-05 00:00:00
Tasks: Stochastic Optimization

Variance reduction (VR) methods boost the performance of stochastic gradient
descent (SGD) by enabling the use of larger, constant stepsizes and preserving
linear convergence rates. However, current variance reduced SGD methods require
either high memory usage or an exact gradient computation (using the entire
dataset) at the end of each epoch. This limits the use of VR methods in
practical distributed settings. In this paper, we propose a variance reduction
method, called VR-lite, that does not require full gradient computations or
extra storage. We explore distributed synchronous and asynchronous variants
that are scalable and remain stable with low communication frequency. We
empirically compare both the sequential and distributed algorithms to
state-of-the-art stochastic optimization methods, and find that our proposed
algorithms perform favorably to other stochastic methods.
"RAPID" Regions-of-Interest Detection In Big Histopathological Images
Authors: Li Sulimowicz, Ishfaq Ahmad
Date: 2017-04-07 00:00:00
Tasks: Superpixels

The sheer volume and size of histopathological images (e.g.,10^6 MPixel)
underscores the need for faster and more accurate Regions-of-interest (ROI)
detection algorithms. In this paper, we propose such an algorithm, which has
four main components that help achieve greater accuracy and faster speed:
First, while using coarse-to-fine topology preserving segmentation as the
baseline, the proposed algorithm uses a superpixel regularity optimization
scheme for avoiding irregular and extremely small superpixels. Second, the
proposed technique employs a prediction strategy to focus only on important
superpixels at finer image levels. Third, the algorithm reuses the information
gained from the coarsest image level at other finer image levels. Both the
second and the third components drastically lower the complexity. Fourth, the
algorithm employs a highly effective parallelization scheme using adap- tive
data partitioning, which gains high speedup. Experimental results, conducted on
the BSD500 [1] and 500 whole-slide histological images from the National Lung
Screening Trial (NLST)1 dataset, confirm that the proposed algorithm gained 13
times speedup compared with the baseline, and around 160 times compared with
SLIC [11], without losing accuracy.
Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks
Authors: Mohammad Javad Shafiee, Elnaz Barshan, Alexander Wong
Date: 2017-04-07 00:00:00
Tasks: Object Categorization, object-detection, Object Detection

A promising paradigm for achieving highly efficient deep neural networks is
the idea of evolutionary deep intelligence, which mimics biological evolution
processes to progressively synthesize more efficient networks. A crucial design
factor in evolutionary deep intelligence is the genetic encoding scheme used to
simulate heredity and determine the architectures of offspring networks. In
this study, we take a deeper look at the notion of synaptic cluster-driven
evolution of deep neural networks which guides the evolution process towards
the formation of a highly sparse set of synaptic clusters in offspring
networks. Utilizing a synaptic cluster-driven genetic encoding, the
probabilistic encoding of synaptic traits considers not only individual
synaptic properties but also inter-synaptic relationships within a deep neural
network. This process results in highly sparse offspring networks which are
particularly tailored for parallel computational devices such as GPUs and deep
neural network accelerator chips. Comprehensive experimental results using four
well-known deep neural network architectures (LeNet-5, AlexNet, ResNet-56, and
DetectNet) on two different tasks (object categorization and object detection)
demonstrate the efficiency of the proposed method. Cluster-driven genetic
encoding scheme synthesizes networks that can achieve state-of-the-art
performance with significantly smaller number of synapses than that of the
original ancestor network. ($\sim$125-fold decrease in synapses for MNIST).
Furthermore, the improved cluster efficiency in the generated offspring
networks ($\sim$9.71-fold decrease in clusters for MNIST and a $\sim$8.16-fold
decrease in clusters for KITTI) is particularly useful for accelerated
performance on parallel computing hardware architectures such as those in GPUs
and deep neural network accelerator chips.
Conversation Modeling on Reddit using a Graph-Structured LSTM
Authors: Vicky Zayats, Mari Ostendorf
Date: 2017-04-07 00:00:00
Proceeding: TACL 2018 1

This paper presents a novel approach for modeling threaded discussions on
social media using a graph-structured bidirectional LSTM which represents both
hierarchical and temporal conversation structure. In experiments with a task of
predicting popularity of comments in Reddit discussions, the proposed model
outperforms a node-independent architecture for different sets of input
features. Analyses show a benefit to the model over the full course of the
discussion, improving detection in both early and late stages. Further, the use
of language cues with the bidirectional tree state updates helps with
identifying controversial comments.
Efficient Distributed SGD with Variance Reduction
Authors: Soham De, Tom Goldstein
Date: 2015-12-09 00:00:00
Tasks: Stochastic Optimization

Stochastic Gradient Descent (SGD) has become one of the most popular
optimization methods for training machine learning models on massive datasets.
However, SGD suffers from two main drawbacks: (i) The noisy gradient updates
have high variance, which slows down convergence as the iterates approach the
optimum, and (ii) SGD scales poorly in distributed settings, typically
experiencing rapidly decreasing marginal benefits as the number of workers
increases. In this paper, we propose a highly parallel method, CentralVR, that
uses error corrections to reduce the variance of SGD gradient updates, and
scales linearly with the number of worker nodes. CentralVR enjoys low iteration
complexity, provably linear convergence rates, and exhibits linear performance
gains up to hundreds of cores for massive datasets. We compare CentralVR to
state-of-the-art parallel stochastic optimization methods on a variety of
models and datasets, and find that our proposed methods exhibit stronger
scaling than other SGD variants.
Convolutional Neural Pyramid for Image Processing
Authors: Xiaoyong Shen, Ying-Cong Chen, Xin Tao, Jiaya Jia
Date: 2017-04-07 00:00:00
Tasks: Colorization, Image Enhancement, Image Restoration, regression

We propose a principled convolutional neural pyramid (CNP) framework for
general low-level vision and image processing tasks. It is based on the
essential finding that many applications require large receptive fields for
structure understanding. But corresponding neural networks for regression
either stack many layers or apply large kernels to achieve it, which is
computationally very costly. Our pyramid structure can greatly enlarge the
field while not sacrificing computation efficiency. Extra benefit includes
adaptive network depth and progressive upsampling for quasi-realtime testing on
VGA-size input. Our method profits a broad set of applications, such as
depth/RGB image restoration, completion, noise/artifact removal, edge
refinement, image filtering, image enhancement and colorization.
Boundary-aware Instance Segmentation
Authors: Zeeshan Hayder, Xuming He, Mathieu Salzmann
Date: 2016-12-09 00:00:00
Proceeding: CVPR 2017 7
Tasks: Instance Segmentation, Object, Object Proposal Generation, Segmentation, Semantic Segmentation

We address the problem of instance-level semantic segmentation, which aims at
jointly detecting, segmenting and classifying every individual object in an
image. In this context, existing methods typically propose candidate objects,
usually as bounding boxes, and directly predict a binary mask within each such
proposal. As a consequence, they cannot recover from errors in the object
candidate generation process, such as too small or shifted boxes.
  In this paper, we introduce a novel object segment representation based on
the distance transform of the object masks. We then design an object mask
network (OMN) with a new residual-deconvolution architecture that infers such a
representation and decodes it into the final binary object mask. This allows us
to predict masks that go beyond the scope of the bounding boxes and are thus
robust to inaccurate object candidates. We integrate our OMN into a Multitask
Network Cascade framework, and learn the resulting boundary-aware instance
segmentation (BAIS) network in an end-to-end manner. Our experiments on the
PASCAL VOC 2012 and the Cityscapes datasets demonstrate the benefits of our
approach, which outperforms the state-of-the-art in both object proposal
generation and instance segmentation.
An Efficient Pseudo-likelihood Method for Sparse Binary Pairwise Markov Network Estimation
Authors: Sinong Geng, Zhaobin Kuang, David Page
Date: 2017-02-27 00:00:00
Tasks: regression

The pseudo-likelihood method is one of the most popular algorithms for
learning sparse binary pairwise Markov networks. In this paper, we formulate
the $L_1$ regularized pseudo-likelihood problem as a sparse multiple logistic
regression problem. In this way, many insights and optimization procedures for
sparse logistic regression can be applied to the learning of discrete Markov
networks. Specifically, we use the coordinate descent algorithm for generalized
linear models with convex penalties, combined with strong screening rules, to
solve the pseudo-likelihood problem with $L_1$ regularization. Therefore a
substantial speedup without losing any accuracy can be achieved. Furthermore,
this method is more stable than the node-wise logistic regression approach on
unbalanced high-dimensional data when penalized by small regularization
parameters. Thorough numerical experiments on simulated data and real world
data demonstrate the advantages of the proposed method.
What do different evaluation metrics tell us about saliency models?
Authors: Zoya Bylinskii, Tilke Judd, Aude Oliva, Antonio Torralba, Frédo Durand
Date: 2016-04-12 00:00:00

How best to evaluate a saliency model's ability to predict where humans look
in images is an open research question. The choice of evaluation metric depends
on how saliency is defined and how the ground truth is represented. Metrics
differ in how they rank saliency models, and this results from how false
positives and false negatives are treated, whether viewing biases are accounted
for, whether spatial deviations are factored in, and how the saliency maps are
pre-processed. In this paper, we provide an analysis of 8 different evaluation
metrics and their properties. With the help of systematic experiments and
visualizations of metric computations, we add interpretability to saliency
scores and more transparency to the evaluation of saliency models. Building off
the differences in metric properties and behaviors, we make recommendations for
metric selections under specific assumptions and for specific applications.
Big Batch SGD: Automated Inference using Adaptive Batch Sizes
Authors: Soham De, Abhay Yadav, David Jacobs, Tom Goldstein
Date: 2016-10-18 00:00:00

Classical stochastic gradient methods for optimization rely on noisy gradient
approximations that become progressively less accurate as iterates approach a
solution. The large noise and small signal in the resulting gradients makes it
difficult to use them for adaptive stepsize selection and automatic stopping.
We propose alternative "big batch" SGD schemes that adaptively grow the batch
size over time to maintain a nearly constant signal-to-noise ratio in the
gradient approximation. The resulting methods have similar convergence rates to
classical SGD, and do not require convexity of the objective. The high fidelity
gradients enable automated learning rate selection and do not require stepsize
decay. Big batch methods are thus easily automated and can run with little or
no oversight.
Tacotron: Towards End-to-End Speech Synthesis
Authors: Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, Rif A. Saurous
Date: 2017-03-29 00:00:00
Tasks: Audio Synthesis, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

A text-to-speech synthesis system typically consists of multiple stages, such
as a text analysis frontend, an acoustic model and an audio synthesis module.
Building these components often requires extensive domain expertise and may
contain brittle design choices. In this paper, we present Tacotron, an
end-to-end generative text-to-speech model that synthesizes speech directly
from characters. Given <text, audio> pairs, the model can be trained completely
from scratch with random initialization. We present several key techniques to
make the sequence-to-sequence framework perform well for this challenging task.
Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English,
outperforming a production parametric system in terms of naturalness. In
addition, since Tacotron generates speech at the frame level, it's
substantially faster than sample-level autoregressive methods.