pwc-archive/papers-with-abstracts huggingface dataset:
Random Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints
Authors: Mengdi Wang, Yi-Chen Chen, Jialin Liu, Yuantao Gu
Date: 2015-11-12 00:00:00

Consider convex optimization problems subject to a large number of
constraints. We focus on stochastic problems in which the objective takes the
form of expected values and the feasible set is the intersection of a large
number of convex sets. We propose a class of algorithms that perform both
stochastic gradient descent and random feasibility updates simultaneously. At
every iteration, the algorithms sample a number of projection points onto a
randomly selected small subsets of all constraints. Three feasibility update
schemes are considered: averaging over random projected points, projecting onto
the most distant sample, projecting onto a special polyhedral set constructed
based on sample points. We prove the almost sure convergence of these
algorithms, and analyze the iterates' feasibility error and optimality error,
respectively. We provide new convergence rate benchmarks for stochastic
first-order optimization with many constraints. The rate analysis and numerical
experiments reveal that the algorithm using the polyhedral-set projection
scheme is the most efficient one within known algorithms.
Complexity of the Description Logic ALCM
Authors: Monica Martinez, Edelweis Rohrer, Paula Severi
Date: 2015-11-12 00:00:00

In this paper we show that the problem of checking consistency of a knowledge
base in the Description Logic ALCM is ExpTime-complete. The M stands for
meta-modelling as defined by Motz, Rohrer and Severi. To show our main result,
we define an ExpTime Tableau algorithm as an extension of an algorithm for
checking consistency of a knowledge base in ALC by Nguyen and Szalas.
Automatic Content-Aware Color and Tone Stylization
Authors: Joon-Young Lee, Kalyan Sunkavalli, Zhe Lin, Xiaohui Shen, In So Kweon
Date: 2015-11-12 00:00:00
Proceeding: CVPR 2016 6
Tasks: Style Transfer

We introduce a new technique that automatically generates diverse, visually
compelling stylizations for a photograph in an unsupervised manner. We achieve
this by learning style ranking for a given input using a large photo collection
and selecting a diverse subset of matching styles for final style transfer. We
also propose a novel technique that transfers the global color and tone of the
chosen exemplars to the input photograph while avoiding the common visual
artifacts produced by the existing style transfer methods. Together, our style
selection and transfer techniques produce compelling, artifact-free results on
a wide range of input photographs, and a user study shows that our results are
preferred over other techniques.
Explicit Knowledge-based Reasoning for Visual Question Answering
Authors: Peng Wang, Qi Wu, Chunhua Shen, Anton Van Den Hengel, Anthony Dick
Date: 2015-11-09 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We describe a method for visual question answering which is capable of
reasoning about contents of an image on the basis of information extracted from
a large-scale knowledge base. The method not only answers natural language
questions using concepts not contained in the image, but can provide an
explanation of the reasoning by which it developed its answer. The method is
capable of answering far more complex questions than the predominant long
short-term memory-based approach, and outperforms it significantly in the
testing. We also provide a dataset and a protocol by which to evaluate such
methods, thus addressing one of the key issues in general visual ques- tion
answering.
Scaling up Greedy Causal Search for Continuous Variables
Authors: Joseph D. Ramsey
Date: 2015-07-28 00:00:00

As standardly implemented in R or the Tetrad program, causal search
algorithms used most widely or effectively by scientists have severe
dimensionality constraints that make them inappropriate for big data problems
without sacrificing accuracy. However, implementation improvements are
possible. We explore optimizations for the Greedy Equivalence Search that allow
search on 50,000-variable problems in 13 minutes for sparse models with 1000
samples on a four-processor, 16G laptop computer. We finish a problem with 1000
samples on 1,000,000 variables in 18 hours for sparse models on a supercomputer
node at the Pittsburgh Supercomputing Center with 40 processors and 384 G RAM.
The same algorithm can be applied to discrete data, with a slower discrete
score, though the discrete implementation currently does not scale as well in
our experiments; we have managed to scale up to about 10,000 variables in
sparse models with 1000 samples.
The Wilson Machine for Image Modeling
Authors: Saeed Saremi, Terrence J. Sejnowski
Date: 2015-10-27 00:00:00

Learning the distribution of natural images is one of the hardest and most
important problems in machine learning. The problem remains open, because the
enormous complexity of the structures in natural images spans all length
scales. We break down the complexity of the problem and show that the hierarchy
of structures in natural images fuels a new class of learning algorithms based
on the theory of critical phenomena and stochastic processes. We approach this
problem from the perspective of the theory of critical phenomena, which was
developed in condensed matter physics to address problems with infinite
length-scale fluctuations, and build a framework to integrate the criticality
of natural images into a learning algorithm. The problem is broken down by
mapping images into a hierarchy of binary images, called bitplanes. In this
representation, the top bitplane is critical, having fluctuations in structures
over a vast range of scales. The bitplanes below go through a gradual
stochastic heating process to disorder. We turn this representation into a
directed probabilistic graphical model, transforming the learning problem into
the unsupervised learning of the distribution of the critical bitplane and the
supervised learning of the conditional distributions for the remaining
bitplanes. We learnt the conditional distributions by logistic regression in a
convolutional architecture. Conditioned on the critical binary image, this
simple architecture can generate large, natural-looking images, with many
shades of gray, without the use of hidden units, unprecedented in the studies
of natural images. The framework presented here is a major step in bringing
criticality and stochastic processes to machine learning and in studying
natural image statistics.
Deep Multimodal Semantic Embeddings for Speech and Images
Authors: David Harwath, James Glass
Date: 2015-11-11 00:00:00
Tasks: Image Retrieval

In this paper, we present a model which takes as input a corpus of images
with relevant spoken captions and finds a correspondence between the two
modalities. We employ a pair of convolutional neural networks to model visual
objects and speech signals at the word level, and tie the networks together
with an embedding and alignment model which learns a joint semantic space over
both modalities. We evaluate our model using image search and annotation tasks
on the Flickr8k dataset, which we augmented by collecting a corpus of 40,000
spoken captions using Amazon Mechanical Turk.
Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?
Authors: Hervé Cardot, David Degras
Date: 2015-11-11 00:00:00
Tasks: Astronomy, Stochastic Optimization, Vocal Bursts Intensity Prediction

In the current context of data explosion, online techniques that do not
require storing all data in memory are indispensable to routinely perform tasks
like principal component analysis (PCA). Recursive algorithms that update the
PCA with each new observation have been studied in various fields of research
and found wide applications in industrial monitoring, computer vision,
astronomy, and latent semantic indexing, among others. This work provides
guidance for selecting an online PCA algorithm in practice. We present the main
approaches to online PCA, namely, perturbation techniques, incremental methods,
and stochastic optimization, and compare their statistical accuracy,
computation time, and memory requirements using artificial and real data.
Extensions to missing data and to functional data are discussed. All studied
algorithms are available in the R package onlinePCA on CRAN.
Taxonomy of Pathways to Dangerous AI
Authors: Roman V. Yampolskiy
Date: 2015-11-10 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

In order to properly handle a dangerous Artificially Intelligent (AI) system
it is important to understand how the system came to be in such a state. In
popular culture (science fiction movies/books) AIs/Robots became self-aware and
as a result rebel against humanity and decide to destroy it. While it is one
possible scenario, it is probably the least likely path to appearance of
dangerous AI. In this work, we survey, classify and analyze a number of
circumstances, which might lead to arrival of malicious AI. To the best of our
knowledge, this is the first attempt to systematically classify types of
pathways leading to malevolent AI. Previous relevant work either surveyed
specific goals/meta-rules which might lead to malevolent behavior in AIs
(\"Ozkural, 2014) or reviewed specific undesirable behaviors AGIs can exhibit
at different stages of its development (Alexey Turchin, July 10 2015, July 10,
2015).
A Continuous Max-Flow Approach to Cyclic Field Reconstruction
Authors: John S. H. Baxter, Jonathan McLeod, Terry M. Peters
Date: 2015-11-11 00:00:00
Tasks: Image Reconstruction

Reconstruction of an image from noisy data using Markov Random Field theory
has been explored by both the graph-cuts and continuous max-flow community in
the form of the Potts and Ishikawa models. However, neither model takes into
account the particular cyclic topology of specific intensity types such as the
hue in natural colour images, or the phase in complex valued MRI. This paper
presents \textit{cyclic continuous max-flow} image reconstruction which models
the intensity being reconstructed as having a fundamentally cyclic topology.
This model complements the Ishikawa model in that it is designed with image
reconstruction in mind, having the topology of the intensity space inherent in
the model while being readily extendable to an arbitrary intensity resolution.
Bayesian group latent factor analysis with structured sparsity
Authors: Shiwen Zhao, Chuan Gao, Sayan Mukherjee, Barbara E. Engelhardt
Date: 2014-11-11 00:00:00
Tasks: parameter estimation

Latent factor models are the canonical statistical tool for exploratory
analyses of low-dimensional linear structure for an observation matrix with p
features across n samples. We develop a structured Bayesian group factor
analysis model that extends the factor model to multiple coupled observation
matrices; in the case of two observations, this reduces to a Bayesian model of
canonical correlation analysis. The main contribution of this work is to
carefully define a structured Bayesian prior that encourages both element-wise
and column-wise shrinkage and leads to desirable behavior on high-dimensional
data. In particular, our model puts a structured prior on the joint factor
loading matrix, regularizing at three levels, which enables element-wise
sparsity and unsupervised recovery of latent factors corresponding to
structured variance across arbitrary subsets of the observations. In addition,
our structured prior allows for both dense and sparse latent factors so that
covariation among either all features or only a subset of features can both be
recovered. We use fast parameter-expanded expectation-maximization for
parameter estimation in this model. We validate our method on both simulated
data with substantial structure and real data, comparing against a number of
state-of-the-art approaches. These results illustrate useful properties of our
model, including i) recovering sparse signal in the presence of dense effects;
ii) the ability to scale naturally to large numbers of observations; iii)
flexible observation- and factor-specific regularization to recover factors
with a wide variety of sparsity levels and percentage of variance explained;
and iv) tractable inference that scales to modern genomic and document data
sizes.
Fast, Provable Algorithms for Isotonic Regression in all $\ell_{p}$-norms
Authors: Rasmus Kyng, Anup Rao, Sushant Sachdeva
Date: 2015-07-02 00:00:00
Tasks: All, regression

Given a directed acyclic graph $G,$ and a set of values $y$ on the vertices,
the Isotonic Regression of $y$ is a vector $x$ that respects the partial order
described by $G,$ and minimizes $||x-y||,$ for a specified norm. This paper
gives improved algorithms for computing the Isotonic Regression for all
weighted $\ell_{p}$-norms with rigorous performance guarantees. Our algorithms
are quite practical, and their variants can be implemented to run fast in
practice.
One Scan 1-Bit Compressed Sensing
Authors: Ping Li
Date: 2015-03-08 00:00:00
Tasks: compressed sensing

Based on $\alpha$-stable random projections with small $\alpha$, we develop a
simple algorithm for compressed sensing (sparse signal recovery) by utilizing
only the signs (i.e., 1-bit) of the measurements. Using only 1-bit information
of the measurements results in substantial cost reduction in collection,
storage, communication, and decoding for compressed sensing. The proposed
algorithm is efficient in that the decoding procedure requires only one scan of
the coordinates. Our analysis can precisely show that, for a $K$-sparse signal
of length $N$, $12.3K\log N/\delta$ measurements (where $\delta$ is the
confidence) would be sufficient for recovering the support and the signs of the
signal. While the method is very robust against typical measurement noises, we
also provide the analysis of the scheme under random flipping of the signs of
the measurements.
  \noindent Compared to the well-known work on 1-bit marginal regression (which
can also be viewed as a one-scan method), the proposed algorithm requires
orders of magnitude fewer measurements. Compared to 1-bit Iterative Hard
Thresholding (IHT) (which is not a one-scan algorithm), our method is still
significantly more accurate. Furthermore, the proposed method is reasonably
robust against random sign flipping while IHT is known to be very sensitive to
this type of noise.
DataGrinder: Fast, Accurate, Fully non-Parametric Classification Approach Using 2D Convex Hulls
Authors: Mohammad Khabbaz
Date: 2015-11-11 00:00:00
Tasks: Classification, General Classification, Management

It has been a long time, since data mining technologies have made their ways
to the field of data management. Classification is one of the most important
data mining tasks for label prediction, categorization of objects into groups,
advertisement and data management. In this paper, we focus on the standard
classification problem which is predicting unknown labels in Euclidean space.
Most efforts in Machine Learning communities are devoted to methods that use
probabilistic algorithms which are heavy on Calculus and Linear Algebra. Most
of these techniques have scalability issues for big data, and are hardly
parallelizable if they are to maintain their high accuracies in their standard
form. Sampling is a new direction for improving scalability, using many small
parallel classifiers. In this paper, rather than conventional sampling methods,
we focus on a discrete classification algorithm with O(n) expected running
time. Our approach performs a similar task as sampling methods. However, we use
column-wise sampling of data, rather than the row-wise sampling used in the
literature. In either case, our algorithm is completely deterministic. Our
algorithm, proposes a way of combining 2D convex hulls in order to achieve high
classification accuracy as well as scalability in the same time. First, we
thoroughly describe and prove our O(n) algorithm for finding the convex hull of
a point set in 2D. Then, we show with experiments our classifier model built
based on this idea is very competitive compared with existing sophisticated
classification algorithms included in commercial statistical applications such
as MATLAB.
Federated Optimization:Distributed Optimization Beyond the Datacenter
Authors: Jakub Konečný, Brendan Mcmahan, Daniel Ramage
Date: 2015-11-11 00:00:00
Tasks: Distributed Optimization

We introduce a new and increasingly relevant setting for distributed
optimization in machine learning, where the data defining the optimization are
distributed (unevenly) over an extremely large number of \nodes, but the goal
remains to train a high-quality centralized model. We refer to this setting as
Federated Optimization. In this setting, communication efficiency is of utmost
importance.
  A motivating example for federated optimization arises when we keep the
training data locally on users' mobile devices rather than logging it to a data
center for training. Instead, the mobile devices are used as nodes performing
computation on their local data in order to update a global model. We suppose
that we have an extremely large number of devices in our network, each of which
has only a tiny fraction of data available totally; in particular, we expect
the number of data points available locally to be much smaller than the number
of devices. Additionally, since different users generate data with different
patterns, we assume that no device has a representative sample of the overall
distribution.
  We show that existing algorithms are not suitable for this setting, and
propose a new algorithm which shows encouraging experimental results. This work
also sets a path for future research needed in the context of federated
optimization.
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Authors: Pingbo Pan, Zhongwen Xu, Yi Yang, Fei Wu, Yueting Zhuang
Date: 2015-11-11 00:00:00
Proceeding: CVPR 2016 6
Tasks: image-classification, Image Classification, Video Captioning

Recently, deep learning approach, especially deep Convolutional Neural
Networks (ConvNets), have achieved overwhelming accuracy with fast processing
speed for image classification. Incorporating temporal structure with deep
ConvNets for video representation becomes a fundamental problem for video
content analysis. In this paper, we propose a new approach, namely Hierarchical
Recurrent Neural Encoder (HRNE), to exploit temporal information of videos.
Compared to recent video representation inference approaches, this paper makes
the following three contributions. First, our HRNE is able to efficiently
exploit video temporal structure in a longer range by reducing the length of
input information flow, and compositing multiple consecutive inputs at a higher
level. Second, computation operations are significantly lessened while
attaining more non-linearity. Third, HRNE is able to uncover temporal
transitions between frame chunks with different granularities, i.e., it can
model the temporal transitions between frames as well as the transitions
between segments. We apply the new method to video captioning where temporal
information plays a crucial role. Experiments demonstrate that our method
outperforms the state-of-the-art on video captioning benchmarks. Notably, even
using a single network with only RGB stream as input, HRNE beats all the recent
systems which combine multiple inputs, such as RGB ConvNet plus 3D ConvNet.
Instantaneous Modelling and Reverse Engineering of DataConsistent Prime Models in Seconds!
Authors: Michael A. Idowu
Date: 2015-11-11 00:00:00
Tasks: parameter estimation, Time Series, Time Series Analysis

A theoretical framework that supports automated construction of dynamic prime
models purely from experimental time series data has been invented and
developed, which can automatically generate (construct) data-driven models of
any time series data in seconds. This has resulted in the formulation and
formalisation of new reverse engineering and dynamic methods for automated
systems modelling of complex systems, including complex biological, financial,
control, and artificial neural network systems. The systems/model theory behind
the invention has been formalised as a new, effective and robust system
identification strategy complementary to process-based modelling. The proposed
dynamic modelling and network inference solutions often involve tackling
extremely difficult parameter estimation challenges, inferring unknown
underlying network structures, and unsupervised formulation and construction of
smart and intelligent ODE models of complex systems. In underdetermined
conditions, i.e., cases of dealing with how best to instantaneously and rapidly
construct data-consistent prime models of unknown (or well-studied) complex
system from small-sized time series data, inference of unknown underlying
network of interaction is more challenging. This article reports a robust
step-by-step mathematical and computational analysis of the entire prime model
construction process that determines a model from data in less than a minute.
A Directional Diffusion Algorithm for Inpainting
Authors: Jan Deriu, Rolf Jagerman, Kai-En Tsay
Date: 2015-11-11 00:00:00
Tasks: Image Inpainting

The problem of inpainting involves reconstructing the missing areas of an
image. Inpainting has many applications, such as reconstructing old damaged
photographs or removing obfuscations from images. In this paper we present the
directional diffusion algorithm for inpainting. Typical diffusion algorithms
are bad at propagating edges from the image into the unknown masked regions.
The directional diffusion algorithm improves on the regular diffusion algorithm
by reconstructing edges more accurately. It scores better than regular
diffusion when reconstructing images that are obfuscated by a text mask.
Granger Causality in Multi-variate Time Series using a Time Ordered Restricted Vector Autoregressive Model
Authors: Elsa Siggiridou, Dimitris Kugiumtzis
Date: 2015-11-11 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Specificity, Time Series, Time Series Analysis

Granger causality has been used for the investigation of the inter-dependence
structure of the underlying systems of multi-variate time series. In
particular, the direct causal effects are commonly estimated by the conditional
Granger causality index (CGCI). In the presence of many observed variables and
relatively short time series, CGCI may fail because it is based on vector
autoregressive models (VAR) involving a large number of coefficients to be
estimated. In this work, the VAR is restricted by a scheme that modifies the
recently developed method of backward-in-time selection (BTS) of the lagged
variables and the CGCI is combined with BTS. Further, the proposed approach is
compared favorably to other restricted VAR representations, such as the
top-down strategy, the bottom-up strategy, and the least absolute shrinkage and
selection operator (LASSO), in terms of sensitivity and specificity of CGCI.
This is shown by using simulations of linear and nonlinear, low and
high-dimensional systems and different time series lengths. For nonlinear
systems, CGCI from the restricted VAR representations are compared with
analogous nonlinear causality indices. Further, CGCI in conjunction with BTS
and other restricted VAR representations is applied to multi-channel scalp
electroencephalogram (EEG) recordings of epileptic patients containing
epileptiform discharges. CGCI on the restricted VAR, and BTS in particular,
could track the changes in brain connectivity before, during and after
epileptiform discharges, which was not possible using the full VAR
representation.