pwc-archive/papers-with-abstracts huggingface dataset:
"Did I Say Something Wrong?" A Word-Level Analysis of Wikipedia Articles for Deletion Discussions
Authors: Michael Ruster
Date: 2016-03-25 00:00:00
Tasks: Articles

This thesis focuses on gaining linguistic insights into textual discussions
on a word level. It was of special interest to distinguish messages that
constructively contribute to a discussion from those that are detrimental to
them. Thereby, we wanted to determine whether "I"- and "You"-messages are
indicators for either of the two discussion styles. These messages are nowadays
often used in guidelines for successful communication. Although their effects
have been successfully evaluated multiple times, a large-scale analysis has
never been conducted.
  Thus, we used Wikipedia Articles for Deletion (short: AfD) discussions
together with the records of blocked users and developed a fully automated
creation of an annotated data set. In this data set, messages were labelled
either constructive or disruptive. We applied binary classifiers to the data to
determine characteristic words for both discussion styles. Thereby, we also
investigated whether function words like pronouns and conjunctions play an
important role in distinguishing the two.
  We found that "You"-messages were a strong indicator for disruptive messages
which matches their attributed effects on communication. However, we found
"I"-messages to be indicative for disruptive messages as well which is contrary
to their attributed effects. The importance of function words could neither be
confirmed nor refuted. Other characteristic words for either communication
style were not found. Yet, the results suggest that a different model might
represent disruptive and constructive messages in textual discussions better.
Hamiltonian Monte Carlo Without Detailed Balance
Authors: Jascha Sohl-Dickstein, Mayur Mudigonda, Michael R. DeWeese
Date: 2014-09-18 00:00:00

We present a method for performing Hamiltonian Monte Carlo that largely
eliminates sample rejection for typical hyperparameters. In situations that
would normally lead to rejection, instead a longer trajectory is computed until
a new state is reached that can be accepted. This is achieved using Markov
chain transitions that satisfy the fixed point equation, but do not satisfy
detailed balance. The resulting algorithm significantly suppresses the random
walk behavior and wasted function evaluations that are typically the
consequence of update rejection. We demonstrate a greater than factor of two
improvement in mixing time on three test problems. We release the source code
as Python and MATLAB packages.
An Empirical Study of Dimensional Reduction Techniques for Facial Action Units Detection
Authors: Zhuo Hui, Wen-Sheng Chu
Date: 2016-03-25 00:00:00
Tasks: Action Unit Detection, Computational Efficiency, Dimensionality Reduction

Biologically inspired features, such as Gabor filters, result in very high
dimensional measurement. Does reducing the dimensionality of the feature space
afford advantages beyond computational efficiency? Do some approaches to
dimensionality reduction (DR) yield improved action unit detection? To answer
these questions, we compared DR approaches in two relatively large databases of
spontaneous facial behavior (45 participants in total with over 2 minutes of
FACS-coded video per participant). Facial features were tracked and aligned
using active appearance models (AAM). SIFT and Gabor features were extracted
from local facial regions. We compared linear (PCA and KPCA), manifold (LPP and
LLE), supervised (LDA and KDA) and hybrid approaches (LSDA) to DR with respect
to AU detection. For further comparison, a no-DR control condition was included
as well. Linear support vector machine classifiers with independent train and
test sets were used for AU detection. AU detection was quantified using area
under the ROC curve and F1. Baseline results for PCA with Gabor features were
comparable with previous research. With some notable exceptions, DR improved AU
detection relative to no-DR. Locality embedding approaches proved vulnerable to
\emph{out-of-sample} problems. Gradient-based SIFT lead to better AU detection
than the filter-based Gabor features. For area under the curve, few differences
were found between linear and other DR approaches. For F1, results were mixed.
For both metrics, the pattern of results varied among action units. These
findings suggest that action unit detection may be optimized by using specific
DR for specific action units. PCA and LDA were the most efficient approaches;
KDA was the least efficient.
On the Detection of Mixture Distributions with applications to the Most Biased Coin Problem
Authors: Kevin Jamieson, Daniel Haas, Ben Recht
Date: 2016-03-25 00:00:00
Tasks: Anomaly Detection

This paper studies the trade-off between two different kinds of pure
exploration: breadth versus depth. The most biased coin problem asks how many
total coin flips are required to identify a "heavy" coin from an infinite bag
containing both "heavy" coins with mean $\theta_1 \in (0,1)$, and "light" coins
with mean $\theta_0 \in (0,\theta_1)$, where heavy coins are drawn from the bag
with probability $\alpha \in (0,1/2)$. The key difficulty of this problem lies
in distinguishing whether the two kinds of coins have very similar means, or
whether heavy coins are just extremely rare. This problem has applications in
crowdsourcing, anomaly detection, and radio spectrum search. Chandrasekaran et.
al. (2014) recently introduced a solution to this problem but it required
perfect knowledge of $\theta_0,\theta_1,\alpha$. In contrast, we derive
algorithms that are adaptive to partial or absent knowledge of the problem
parameters. Moreover, our techniques generalize beyond coins to more general
instances of infinitely many armed bandit problems. We also prove lower bounds
that show our algorithm's upper bounds are tight up to $\log$ factors, and on
the way characterize the sample complexity of differentiating between a single
parametric distribution and a mixture of two such distributions. As a result,
these bounds have surprising implications both for solutions to the most biased
coin problem and for anomaly detection when only partial information about the
parameters is known.
Resnet in Resnet: Generalizing Residual Architectures
Authors: Sasha Targ, Diogo Almeida, Kevin Lyman
Date: 2016-03-25 00:00:00

Residual networks (ResNets) have recently achieved state-of-the-art on
challenging computer vision tasks. We introduce Resnet in Resnet (RiR): a deep
dual-stream architecture that generalizes ResNets and standard CNNs and is
easily implemented with no computational overhead. RiR consistently improves
performance over ResNets, outperforms architectures with similar amounts of
augmentation on CIFAR-10, and establishes a new state-of-the-art on CIFAR-100.
On the Simultaneous Preservation of Privacy and Community Structure in Anonymized Networks
Authors: Daniel Cullina, Kushagra Singhal, Negar Kiyavash, Prateek Mittal
Date: 2016-03-25 00:00:00
Tasks: Community Detection, Stochastic Block Model

We consider the problem of performing community detection on a network, while
maintaining privacy, assuming that the adversary has access to an auxiliary
correlated network. We ask the question "Does there exist a regime where the
network cannot be deanonymized perfectly, yet the community structure could be
learned?." To answer this question, we derive information theoretic converses
for the perfect deanonymization problem using the Stochastic Block Model and
edge sub-sampling. We also provide an almost tight achievability result for
perfect deanonymization.
  We also evaluate the performance of percolation based deanonymization
algorithm on Stochastic Block Model data-sets that satisfy the conditions of
our converse. Although our converse applies to exact deanonymization, the
algorithm fails drastically when the conditions of the converse are met.
Additionally, we study the effect of edge sub-sampling on the community
structure of a real world dataset. Results show that the dataset falls under
the purview of the idea of this paper. There results suggest that it may be
possible to prove stronger partial deanonymizability converses, which would
enable better privacy guarantees.
Perturbed Iterate Analysis for Asynchronous Stochastic Optimization
Authors: Horia Mania, Xinghao Pan, Dimitris Papailiopoulos, Benjamin Recht, Kannan Ramchandran, Michael. I. Jordan
Date: 2015-07-24 00:00:00
Tasks: Stochastic Optimization

We introduce and analyze stochastic optimization methods where the input to
each gradient update is perturbed by bounded noise. We show that this framework
forms the basis of a unified approach to analyze asynchronous implementations
of stochastic optimization algorithms.In this framework, asynchronous
stochastic optimization algorithms can be thought of as serial methods
operating on noisy inputs. Using our perturbed iterate framework, we provide
new analyses of the Hogwild! algorithm and asynchronous stochastic coordinate
descent, that are simpler than earlier analyses, remove many assumptions of
previous models, and in some cases yield improved upper bounds on the
convergence rates. We proceed to apply our framework to develop and analyze
KroMagnon: a novel, parallel, sparse stochastic variance-reduced gradient
(SVRG) algorithm. We demonstrate experimentally on a 16-core machine that the
sparse and parallel version of SVRG is in some cases more than four orders of
magnitude faster than the standard SVRG algorithm.
Modular Decomposition and Analysis of Registration based Trackers
Authors: Abhineet Singh, Ankush Roy, Xi Zhang, Martin Jagersand
Date: 2016-03-03 00:00:00

This paper presents a new way to study registration based trackers by
decomposing them into three constituent sub modules: appearance model, state
space model and search method. It is often the case that when a new tracker is
introduced in literature, it only contributes to one or two of these sub
modules while using existing methods for the rest. Since these are often
selected arbitrarily by the authors, they may not be optimal for the new
method. In such cases, our breakdown can help to experimentally find the best
combination of methods for these sub modules while also providing a framework
within which the contributions of the new tracker can be clearly demarcated and
thus studied better. We show how existing trackers can be broken down using the
suggested methodology and compare the performance of the default configuration
chosen by the authors against other possible combinations to demonstrate the
new insights that can be gained by such an approach. We also present an open
source system that provides a convenient interface to plug in a new method for
any sub module and test it against all possible combinations of methods for the
other two sub modules while also serving as a fast and efficient solution for
practical tracking requirements.
Developing Quantum Annealer Driven Data Discovery
Authors: Joseph Dulny III, Michael Kim
Date: 2016-03-25 00:00:00
Tasks: BIG-bench Machine Learning, Seizure prediction

Machine learning applications are limited by computational power. In this
paper, we gain novel insights into the application of quantum annealing (QA) to
machine learning (ML) through experiments in natural language processing (NLP),
seizure prediction, and linear separability testing. These experiments are
performed on QA simulators and early-stage commercial QA hardware and compared
to an unprecedented number of traditional ML techniques. We extend QBoost, an
early implementation of a binary classifier that utilizes a quantum annealer,
via resampling and ensembling of predicted probabilities to produce a more
robust class estimator. To determine the strengths and weaknesses of this
approach, resampled QBoost (RQBoost) is tested across several datasets and
compared to QBoost and traditional ML. We show and explain how QBoost in
combination with a commercial QA device are unable to perfectly separate binary
class data which is linearly separable via logistic regression with shrinkage.
We further explore the performance of RQBoost in the space of NLP and seizure
prediction and find QA-enabled ML using QBoost and RQBoost is outperformed by
traditional techniques. Additionally, we provide a detailed discussion of
algorithmic constraints and trade-offs imposed by the use of this QA hardware.
Through these experiments, we provide unique insights into the state of quantum
ML via boosting and the use of quantum annealing hardware that are valuable to
institutions interested in applying QA to problems in ML and beyond.
Unsupervised Category Discovery via Looped Deep Pseudo-Task Optimization Using a Large Scale Radiology Image Database
Authors: Xiaosong Wang, Le Lu, Hoo-chang Shin, Lauren Kim, Isabella Nogues, Jianhua Yao, Ronald Summers
Date: 2016-03-25 00:00:00
Tasks: Clustering

Obtaining semantic labels on a large scale radiology image database (215,786
key images from 61,845 unique patients) is a prerequisite yet bottleneck to
train highly effective deep convolutional neural network (CNN) models for image
recognition. Nevertheless, conventional methods for collecting image labels
(e.g., Google search followed by crowd-sourcing) are not applicable due to the
formidable difficulties of medical annotation tasks for those who are not
clinically trained. This type of image labeling task remains non-trivial even
for radiologists due to uncertainty and possible drastic inter-observer
variation or inconsistency.
  In this paper, we present a looped deep pseudo-task optimization procedure
for automatic category discovery of visually coherent and clinically semantic
(concept) clusters. Our system can be initialized by domain-specific (CNN
trained on radiology images and text report derived labels) or generic
(ImageNet based) CNN models. Afterwards, a sequence of pseudo-tasks are
exploited by the looped deep image feature clustering (to refine image labels)
and deep CNN training/classification using new labels (to obtain more task
representative deep features). Our method is conceptually simple and based on
the hypothesized "convergence" of better labels leading to better trained CNN
models which in turn feed more effective deep image features to facilitate more
meaningful clustering/labels. We have empirically validated the convergence and
demonstrated promising quantitative and qualitative results. Category labels of
significantly higher quality than those in previous work are discovered. This
allows for further investigation of the hierarchical semantic nature of the
given large-scale radiology image database.
Unsupervised Transductive Domain Adaptation
Authors: Ozan Sener, Hyun Oh Song, Ashutosh Saxena, Silvio Savarese
Date: 2016-02-10 00:00:00
Tasks: Domain Adaptation, Object Recognition, Unsupervised Domain Adaptation

Supervised learning with large scale labeled datasets and deep layered models
has made a paradigm shift in diverse areas in learning and recognition.
However, this approach still suffers generalization issues under the presence
of a domain shift between the training and the test data distribution. In this
regard, unsupervised domain adaptation algorithms have been proposed to
directly address the domain shift problem. In this paper, we approach the
problem from a transductive perspective. We incorporate the domain shift and
the transductive target inference into our framework by jointly solving for an
asymmetric similarity metric and the optimal transductive target label
assignment. We also show that our model can easily be extended for deep feature
learning in order to learn features which are discriminative in the target
domain. Our experiments show that the proposed method significantly outperforms
state-of-the-art algorithms in both object recognition and digit classification
experiments by a large margin.
Object Recognition Based on Amounts of Unlabeled Data
Authors: Fuqiang Liu, Fukun Bi, Liang Chen
Date: 2016-03-25 00:00:00
Tasks: Object, Object Recognition

This paper proposes a novel semi-supervised method on object recognition.
First, based on Boost Picking, a universal algorithm, Boost Picking Teaching
(BPT), is proposed to train an effective binary-classifier just using a few
labeled data and amounts of unlabeled data. Then, an ensemble strategy is
detailed to synthesize multiple BPT-trained binary-classifiers to be a
high-performance multi-classifier. The rationality of the strategy is also
analyzed in theory. Finally, the proposed method is tested on two databases,
CIFAR-10 and CIFAR-100. Using 2% labeled data and 98% unlabeled data, the
accuracies of the proposed method on the two data sets are 78.39% and 50.77%
respectively.
Generalized system identification with stable spline kernels
Authors: Aleksandr Y. Aravkin, James V. Burke, Gianluigi Pillonetto
Date: 2013-09-30 00:00:00

Regularized least-squares approaches have been successfully applied to linear
system identification. Recent approaches use quadratic penalty terms on the
unknown impulse response defined by stable spline kernels, which control model
space complexity by leveraging regularity and bounded-input bounded-output
stability. This paper extends linear system identification to a wide class of
nonsmooth stable spline estimators, where regularization functionals and data
misfits can be selected from a rich set of piecewise linear-quadratic (PLQ)
penalties. This class includes the 1-norm, Huber, and Vapnik, in addition to
the least-squares penalty.
  By representing penalties through their conjugates, the modeler can specify
any piecewise linear-quadratic penalty for misfit and regularizer, as well as
inequality constraints on the response. The interior-point solver we implement
(IPsolve) is locally quadratically convergent, with $O(\min(m,n)^2(m+n))$
arithmetic operations per iteration, where $n$ the number of unknown impulse
response coefficients and $m$ the number of observed output measurements.
IPsolve is competitive with available alternatives for system identification.
This is shown by a comparison with TFOCS, libSVM, and the FISTA algorithm. The
code is open source (https://github.com/saravkin/IPsolve).
  The impact of the approach for system identification is illustrated with
numerical experiments featuring robust formulations for contaminated data,
relaxation systems, nonnegativity and unimodality constraints on the impulse
response, and sparsity promoting regularization. Incorporating constraints
yields particularly significant improvements.
Transfer Learning Based on AdaBoost for Feature Selection from Multiple ConvNet Layer Features
Authors: Jumabek Alikhanov, Myeong Hyeon Ga, Seunghyun Ko, Geun-Sik Jo
Date: 2016-02-01 00:00:00
Tasks: feature selection, Transfer Learning

Convolutional Networks (ConvNets) are powerful models that learn hierarchies
of visual features, which could also be used to obtain image representations
for transfer learning. The basic pipeline for transfer learning is to first
train a ConvNet on a large dataset (source task) and then use feed-forward
units activation of the trained ConvNet as image representation for smaller
datasets (target task). Our key contribution is to demonstrate superior
performance of multiple ConvNet layer features over single ConvNet layer
features. Combining multiple ConvNet layer features will result in more complex
feature space with some features being repetitive. This requires some form of
feature selection. We use AdaBoost with single stumps to implicitly select only
distinct features that are useful towards classification from concatenated
ConvNet features. Experimental results show that using multiple ConvNet layer
activation features instead of single ConvNet layer features consistently will
produce superior performance. Improvements becomes significant as we increase
the distance between source task and the target task.
A Novel Biologically Mechanism-Based Visual Cognition Model--Automatic Extraction of Semantics, Formation of Integrated Concepts and Re-selection Features for Ambiguity
Authors: Peijie Yin, Hong Qiao, Wei Wu, Lu Qi, YinLin Li, Shanlin Zhong, Bo Zhang
Date: 2016-03-25 00:00:00
Tasks: Object Recognition

Integration between biology and information science benefits both fields.
Many related models have been proposed, such as computational visual cognition
models, computational motor control models, integrations of both and so on. In
general, the robustness and precision of recognition is one of the key problems
for object recognition models.
  In this paper, inspired by features of human recognition process and their
biological mechanisms, a new integrated and dynamic framework is proposed to
mimic the semantic extraction, concept formation and feature re-selection in
human visual processing. The main contributions of the proposed model are as
follows:
  (1) Semantic feature extraction: Local semantic features are learnt from
episodic features that are extracted from raw images through a deep neural
network;
  (2) Integrated concept formation: Concepts are formed with local semantic
information and structural information learnt through network.
  (3) Feature re-selection: When ambiguity is detected during recognition
process, distinctive features according to the difference between ambiguous
candidates are re-selected for recognition.
  Experimental results on hand-written digits and facial shape dataset show
that, compared with other methods, the new proposed model exhibits higher
robustness and precision for visual recognition, especially in the condition
when input samples are smantic ambiguous. Meanwhile, the introduced biological
mechanisms further strengthen the interaction between neuroscience and
information science.
Hybridization of Expectation-Maximization and K-Means Algorithms for Better Clustering Performance
Authors: D. Raja Kishor, N. B. Venkateswarlu
Date: 2016-03-25 00:00:00
Tasks: Clustering

The present work proposes hybridization of Expectation-Maximization (EM) and
K-Means techniques as an attempt to speed-up the clustering process. Though
both K-Means and EM techniques look into different areas, K-means can be viewed
as an approximate way to obtain maximum likelihood estimates for the means.
Along with the proposed algorithm for hybridization, the present work also
experiments with the Standard EM algorithm. Six different datasets are used for
the experiments of which three are synthetic datasets. Clustering fitness and
Sum of Squared Errors (SSE) are computed for measuring the clustering
performance. In all the experiments it is observed that the proposed algorithm
for hybridization of EM and K-Means techniques is consistently taking less
execution time with acceptable Clustering Fitness value and less SSE than the
standard EM algorithm. It is also observed that the proposed algorithm is
producing better clustering results than the Cluster package of Purdue
University.
The Benefit of Multitask Representation Learning
Authors: Andreas Maurer, Massimiliano Pontil, Bernardino Romera-Paredes
Date: 2015-05-23 00:00:00
Tasks: Representation Learning

We discuss a general method to learn data representations from multiple
tasks. We provide a justification for this method in both settings of multitask
learning and learning-to-learn. The method is illustrated in detail in the
special case of linear feature learning. Conditions on the theoretical
advantage offered by multitask representation learning over independent task
learning are established. In particular, focusing on the important example of
half-space learning, we derive the regime in which multitask representation
learning is beneficial over independent task learning, as a function of the
sample size, the number of tasks and the intrinsic data dimensionality. Other
potential applications of our results include multitask feature learning in
reproducing kernel Hilbert spaces and multilayer, deep networks.