pwc-archive/papers-with-abstracts huggingface dataset:
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification
Authors: Igor Gitman, Boris Ginsburg
Date: 2017-09-24 00:00:00
Tasks: General Classification, image-classification, Image Classification

Batch normalization (BN) has become a de facto standard for training deep
convolutional networks. However, BN accounts for a significant fraction of
training run-time and is difficult to accelerate, since it is a
memory-bandwidth bounded operation. Such a drawback of BN motivates us to
explore recently proposed weight normalization algorithms (WN algorithms), i.e.
weight normalization, normalization propagation and weight normalization with
translated ReLU. These algorithms don't slow-down training iterations and were
experimentally shown to outperform BN on relatively small networks and
datasets. However, it is not clear if these algorithms could replace BN in
practical, large-scale applications. We answer this question by providing a
detailed comparison of BN and WN algorithms using ResNet-50 network trained on
ImageNet. We found that although WN achieves better training accuracy, the
final test accuracy is significantly lower ($\approx 6\%$) than that of BN.
This result demonstrates the surprising strength of the BN regularization
effect which we were unable to compensate for using standard regularization
techniques like dropout and weight decay. We also found that training of deep
networks with WN algorithms is significantly less stable compared to BN,
limiting their practical applications.
A New Spectral Clustering Algorithm
Authors: W. R. Casper, Balu Nadiga
Date: 2017-10-07 00:00:00
Tasks: Clustering

We present a new clustering algorithm that is based on searching for natural
gaps in the components of the lowest energy eigenvectors of the Laplacian of a
graph. In comparing the performance of the proposed method with a set of other
popular methods (KMEANS, spectral-KMEANS, and an agglomerative method) in the
context of the Lancichinetti-Fortunato-Radicchi (LFR) Benchmark for undirected
weighted overlapping networks, we find that the new method outperforms the
other spectral methods considered in certain parameter regimes. Finally, in an
application to climate data involving one of the most important modes of
interannual climate variability, the El Nino Southern Oscillation phenomenon,
we demonstrate the ability of the new algorithm to readily identify different
flavors of the phenomenon.
Texture Fuzzy Segmentation using Skew Divergence Adaptive Affinity Functions
Authors: José F. S. Neto, Waldson P. N. Leandro, Matheus A. Gadelha, Tiago S. Santos, Bruno M. Carvalho, Edgar Garduño
Date: 2017-10-07 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Digital image segmentation is the process of assigning distinct labels to
different objects in a digital image, and the fuzzy segmentation algorithm has
been successfully used in the segmentation of images from a wide variety of
sources. However, the traditional fuzzy segmentation algorithm fails to segment
objects that are characterized by textures whose patterns cannot be
successfully described by simple statistics computed over a very restricted
area. In this paper, we propose an extension of the fuzzy segmentation
algorithm that uses adaptive textural affinity functions to perform the
segmentation of such objects on bidimensional images. The adaptive affinity
functions compute their appropriate neighborhood size as they compute the
texture descriptors surrounding the seed spels (spatial elements), according to
the characteristics of the texture being processed. The algorithm then segments
the image with an appropriate neighborhood for each object. We performed
experiments on mosaic images that were composed using images from the Brodatz
database, and compared our results with the ones produced by a recently
published texture segmentation algorithm, showing the applicability of our
method.
Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images
Authors: Ebrahim Karami, Siva Prasad, Mohamed Shehata
Date: 2017-10-07 00:00:00

Fast and robust image matching is a very important task with various
applications in computer vision and robotics. In this paper, we compare the
performance of three different image matching techniques, i.e., SIFT, SURF, and
ORB, against different kinds of transformations and deformations such as
scaling, rotation, noise, fish eye distortion, and shearing. For this purpose,
we manually apply different types of transformations on original images and
compute the matching evaluation parameters such as the number of key points in
images, the matching rate, and the execution time required for each algorithm
and we will show that which algorithm is the best more robust against each kind
of distortion. Index Terms-Image matching, scale invariant feature transform
(SIFT), speed up robust feature (SURF), robust independent elementary features
(BRIEF), oriented FAST, rotated BRIEF (ORB).
Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm
Authors: Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, Sune Lehmann
Date: 2017-08-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Diversity, Sarcasm Detection, Sentiment Analysis, Transfer Learning

NLP tasks are often limited by scarcity of manually annotated data. In social
media sentiment analysis and related tasks, researchers have therefore used
binarized emoticons and specific hashtags as forms of distant supervision. Our
paper shows that by extending the distant supervision to a more diverse set of
noisy labels, the models can learn richer representations. Through emoji
prediction on a dataset of 1246 million tweets containing one of 64 common
emojis we obtain state-of-the-art performance on 8 benchmark datasets within
sentiment, emotion and sarcasm detection using a single pretrained model. Our
analyses confirm that the diversity of our emotional labels yield a performance
improvement over previous distant supervision approaches.
Group Sparse CNNs for Question Classification with Answer Sets
Authors: Mingbo Ma, Liang Huang, Bing Xiang, Bo-Wen Zhou
Date: 2017-10-07 00:00:00
Proceeding: ACL 2017 7
Tasks: Classification, General Classification

Question classification is an important task with wide applications. However,
traditional techniques treat questions as general sentences, ignoring the
corresponding answer data. In order to consider answer information into
question modeling, we first introduce novel group sparse autoencoders which
refine question representation by utilizing group information in the answer
set. We then propose novel group sparse CNNs which naturally learn question
representation with respect to their answers by implanting group sparse
autoencoders into traditional CNNs. The proposed model significantly outperform
strong baselines on four datasets.
Estimating latent feature-feature interactions in large feature-rich graphs
Authors: Corrado Monti, Paolo Boldi
Date: 2016-12-03 00:00:00

Real-world complex networks describe connections between objects; in reality,
those objects are often endowed with some kind of features. How does the
presence or absence of such features interplay with the network link structure?
Although the situation here described is truly ubiquitous, there is a limited
body of research dealing with large graphs of this kind. Many previous works
considered homophily as the only possible transmission mechanism translating
node features into links. Other authors, instead, developed more sophisticated
models, that are able to handle complex feature interactions, but are unfit to
scale to very large networks. We expand on the MGJ model, where interactions
between pairs of features can foster or discourage link formation. In this
work, we will investigate how to estimate the latent feature-feature
interactions in this model. We shall propose two solutions: the first one
assumes feature independence and it is essentially based on Naive Bayes; the
second one, which relaxes the independence assumption assumption, is based on
perceptrons. In fact, we show it is possible to cast the model equation in
order to see it as the prediction rule of a perceptron. We analyze how
classical results for the perceptrons can be interpreted in this context; then,
we define a fast and simple perceptron-like algorithm for this task, which can
process $10^8$ links in minutes. We then compare these two techniques, first
with synthetic datasets that follows our model, gaining evidence that the Naive
independence assumptions are detrimental in practice. Secondly, we consider a
real, large-scale citation network where each node (i.e., paper) can be
described by different types of characteristics; there, our algorithm can
assess how well each set of features can explain the links, and thus finding
meaningful latent feature-feature interactions.
Interactive Learning of State Representation through Natural Language Instruction and Explanation
Authors: Qiaozi Gao, Lanbo She, Joyce Y. Chai
Date: 2017-10-07 00:00:00

One significant simplification in most previous work on robot learning is the
closed-world assumption where the robot is assumed to know ahead of time a
complete set of predicates describing the state of the physical world. However,
robots are not likely to have a complete model of the world especially when
learning a new task. To address this problem, this extended abstract gives a
brief introduction to our on-going work that aims to enable the robot to
acquire new state representations through language communication with humans.
End to End Deep Neural Network Frequency Demodulation of Speech Signals
Authors: Dan Elbaz, Michael Zibulevsky
Date: 2017-04-06 00:00:00

Frequency modulation (FM) is a form of radio broadcasting which is widely
used nowadays and has been for almost a century. We suggest a
software-defined-radio (SDR) receiver for FM demodulation that adopts an
end-to-end learning based approach and utilizes the prior information of
transmitted speech message in the demodulation process. The receiver detects
and enhances speech from the in-phase and quadrature components of its base
band version. The new system yields high performance detection for both
acoustical disturbances, and communication channel noise and is foreseen to
out-perform the established methods for low signal to noise ratio (SNR)
conditions in both mean square error and in perceptual evaluation of speech
quality score.
Asymptotic Allocation Rules for a Class of Dynamic Multi-armed Bandit Problems
Authors: T. W. U. Madhushani, D. H. S. Maithripala, N. E. Leonard
Date: 2017-10-02 00:00:00

This paper presents a class of Dynamic Multi-Armed Bandit problems where the
reward can be modeled as the noisy output of a time varying linear stochastic
dynamic system that satisfies some boundedness constraints. The class allows
many seemingly different problems with time varying option characteristics to
be considered in a single framework. It also opens up the possibility of
considering many new problems of practical importance. For instance it affords
the simultaneous consideration of temporal option unavailabilities and the
depen- dencies between options with time varying option characteristics in a
seamless manner. We show that, for this class of problems, the combination of
any Upper Confidence Bound type algorithm with any efficient reward estimator
for the expected reward ensures the logarithmic bounding of the expected
cumulative regret. We demonstrate the versatility of the approach by the
explicit consideration of a new example of practical interest.
Nonsparse learning with latent variables
Authors: Zemin Zheng, Jinchi Lv, Wei. Lin
Date: 2017-10-07 00:00:00
Tasks: Model Selection, Sparse Learning

As a popular tool for producing meaningful and interpretable models,
large-scale sparse learning works efficiently when the underlying structures
are indeed or close to sparse. However, naively applying the existing
regularization methods can result in misleading outcomes due to model
misspecification. In particular, the direct sparsity assumption on coefficient
vectors has been questioned in real applications. Therefore, we consider
nonsparse learning with the conditional sparsity structure that the coefficient
vector becomes sparse after taking out the impacts of certain unobservable
latent variables. A new methodology of nonsparse learning with latent variables
(NSL) is proposed to simultaneously recover the significant observable
predictors and latent factors as well as their effects. We explore a common
latent family incorporating population principal components and derive the
convergence rates of both sample principal components and their score vectors
that hold for a wide class of distributions. With the properly estimated latent
variables, properties including model selection consistency and oracle
inequalities under various prediction and estimation losses are established for
the proposed methodology. Our new methodology and results are evidenced by
simulation and real data examples.
Classification of Time-Series Images Using Deep Convolutional Neural Networks
Authors: Nima Hatami, Yann Gavet, Johan Debayle
Date: 2017-10-02 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

Convolutional Neural Networks (CNN) has achieved a great success in image
recognition task by automatically learning a hierarchical feature
representation from raw data. While the majority of Time-Series Classification
(TSC) literature is focused on 1D signals, this paper uses Recurrence Plots
(RP) to transform time-series into 2D texture images and then take advantage of
the deep CNN classifier. Image representation of time-series introduces
different feature types that are not available for 1D signals, and therefore
TSC can be treated as texture image recognition task. CNN model also allows
learning different levels of representations together with a classifier,
jointly and automatically. Therefore, using RP and CNN in a unified framework
is expected to boost the recognition rate of TSC. Experimental results on the
UCR time-series classification archive demonstrate competitive accuracy of the
proposed approach, compared not only to the existing deep architectures, but
also to the state-of-the art TSC algorithms.
Correlated Equilibria for Approximate Variational Inference in MRFs
Authors: Luis E. Ortiz, Boshen Wang, Ze Gong
Date: 2016-04-10 00:00:00
Tasks: Fairness, Variational Inference

Almost all of the work in graphical models for game theory has mirrored
previous work in probabilistic graphical models. Our work considers the
opposite direction: Taking advantage of recent advances in equilibrium
computation for probabilistic inference. We present formulations of inference
problems in Markov random fields (MRFs) as computation of equilibria in a
certain class of game-theoretic graphical models. We concretely establishes the
precise connection between variational probabilistic inference in MRFs and
correlated equilibria. No previous work exploits recent theoretical and
empirical results from the literature on algorithmic and computational game
theory on the tractable, polynomial-time computation of exact or approximate
correlated equilibria in graphical games with arbitrary, loopy graph structure.
We discuss how to design new algorithms with equally tractable guarantees for
the computation of approximate variational inference in MRFs. Also, inspired by
a previously stated game-theoretic view of state-of-the-art tree-reweighed
(TRW) message-passing techniques for belief inference as zero-sum game, we
propose a different, general-sum potential game to design approximate
fictitious-play techniques. We perform synthetic experiments evaluating our
proposed approximation algorithms with standard methods and TRW on several
classes of classical Ising models (i.e., with binary random variables). We also
evaluate the algorithms using Ising models learned from the MNIST dataset. Our
experiments show that our global approach is competitive, particularly shinning
in a class of Ising models with constant, "highly attractive" edge-weights, in
which it is often better than all other alternatives we evaluated. With a
notable exception, our more local approach was not as effective. Yet, in
fairness, almost all of the alternatives are often no better than a simple
baseline: estimate 0.5.
Memetic search for identifying critical nodes in sparse graphs
Authors: Yangming Zhou, Jin-Kao Hao, Fred Glover
Date: 2017-05-11 00:00:00

Critical node problems involve identifying a subset of critical nodes from an
undirected graph whose removal results in optimizing a pre-defined measure over
the residual graph. As useful models for a variety of practical applications,
these problems are computational challenging. In this paper, we study the
classic critical node problem (CNP) and introduce an effective memetic
algorithm for solving CNP. The proposed algorithm combines a double
backbone-based crossover operator (to generate promising offspring solutions),
a component-based neighborhood search procedure (to find high-quality local
optima) and a rank-based pool updating strategy (to guarantee a healthy
population). Specially, the component-based neighborhood search integrates two
key techniques, i.e., two-phase node exchange strategy and node weighting
scheme. The double backbone-based crossover extends the idea of general
backbone-based crossovers. Extensive evaluations on 42 synthetic and real-world
benchmark instances show that the proposed algorithm discovers 21 new upper
bounds and matches 18 previous best-known upper bounds. We also demonstrate the
relevance of our algorithm for effectively solving a variant of the classic
CNP, called the cardinality-constrained critical node problem. Finally, we
investigate the usefulness of each key algorithmic component.
Ranking and Selection with Covariates for Personalized Decision Making
Authors: Haihui Shen, L. Jeff Hong, Xiaowei Zhang
Date: 2017-10-07 00:00:00
Tasks: Decision Making, Experimental Design

We consider a problem of ranking and selection via simulation in the context of personalized decision making, where the best alternative is not universal but varies as a function of some observable covariates. The goal of ranking and selection with covariates (R&S-C) is to use simulation samples to obtain a selection policy that specifies the best alternative with certain statistical guarantee for subsequent individuals upon observing their covariates. A linear model is proposed to capture the relationship between the mean performance of an alternative and the covariates. Under the indifference-zone formulation, we develop two-stage procedures for both homoscedastic and heteroscedastic simulation errors, respectively, and prove their statistical validity in terms of average probability of correct selection. We also generalize the well-known slippage configuration, and prove that the generalized slippage configuration is the least favorable configuration for our procedures. Extensive numerical experiments are conducted to investigate the performance of the proposed procedures, the experimental design issue, and the robustness to the linearity assumption. Finally, we demonstrate the usefulness of R&S-C via a case study of selecting the best treatment regimen in the prevention of esophageal cancer. We find that by leveraging disease-related personal information, R&S-C can substantially improve patients' expected quality-adjusted life years by providing patient-specific treatment regimen.
Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification
Authors: Qiqi Xiao, Hao Luo, Chi Zhang
Date: 2017-10-02 00:00:00
Tasks: Metric Learning, Person Re-Identification, Triplet

Person re-identification (ReID) is an important task in computer vision.
Recently, deep learning with a metric learning loss has become a common
framework for ReID. In this paper, we also propose a new metric learning loss
with hard sample mining called margin smaple mining loss (MSML) which can
achieve better accuracy compared with other metric learning losses, such as
triplet loss. In experi- ments, our proposed methods outperforms most of the
state-of-the-art algorithms on Market1501, MARS, CUHK03 and CUHK-SYSU.
Ranking and Selection as Stochastic Control
Authors: Yijie Peng, Edwin K. P. Chong, Chun-Hung Chen, Michael C. Fu
Date: 2017-10-07 00:00:00

Under a Bayesian framework, we formulate the fully sequential sampling and
selection decision in statistical ranking and selection as a stochastic control
problem, and derive the associated Bellman equation. Using value function
approximation, we derive an approximately optimal allocation policy. We show
that this policy is not only computationally efficient but also possesses both
one-step-ahead and asymptotic optimality for independent normal sampling
distributions. Moreover, the proposed allocation policy is easily generalizable
in the approximate dynamic programming paradigm.
Deep Self-Paced Learning for Person Re-Identification
Authors: Sanping Zhou, Jinjun Wang, Deyu Meng, Xiaomeng Xin, Yubing Li, Yihong Gong, Nanning Zheng
Date: 2017-10-07 00:00:00
Tasks: Person Re-Identification, Triplet

Person re-identification (Re-ID) usually suffers from noisy samples with
background clutter and mutual occlusion, which makes it extremely difficult to
distinguish different individuals across the disjoint camera views. In this
paper, we propose a novel deep self-paced learning (DSPL) algorithm to
alleviate this problem, in which we apply a self-paced constraint and symmetric
regularization to help the relative distance metric training the deep neural
network, so as to learn the stable and discriminative features for person
Re-ID. Firstly, we propose a soft polynomial regularizer term which can derive
the adaptive weights to samples based on both the training loss and model age.
As a result, the high-confidence fidelity samples will be emphasized and the
low-confidence noisy samples will be suppressed at early stage of the whole
training process. Such a learning regime is naturally implemented under a
self-paced learning (SPL) framework, in which samples weights are adaptively
updated based on both model age and sample loss using an alternative
optimization method. Secondly, we introduce a symmetric regularizer term to
revise the asymmetric gradient back-propagation derived by the relative
distance metric, so as to simultaneously minimize the intra-class distance and
maximize the inter-class distance in each triplet unit. Finally, we build a
part-based deep neural network, in which the features of different body parts
are first discriminately learned in the lower convolutional layers and then
fused in the higher fully connected layers. Experiments on several benchmark
datasets have demonstrated the superior performance of our method as compared
with the state-of-the-art approaches.