pwc-archive/papers-with-abstracts huggingface dataset:
Variable Annealing Length and Parallelism in Simulated Annealing
Authors: Vincent A. Cicirello
Date: 2017-09-08 00:00:00
Tasks: Scheduling

In this paper, we propose: (a) a restart schedule for an adaptive simulated
annealer, and (b) parallel simulated annealing, with an adaptive and
parameter-free annealing schedule. The foundation of our approach is the
Modified Lam annealing schedule, which adaptively controls the temperature
parameter to track a theoretically ideal rate of acceptance of neighboring
states. A sequential implementation of Modified Lam simulated annealing is
almost parameter-free. However, it requires prior knowledge of the annealing
length. We eliminate this parameter using restarts, with an exponentially
increasing schedule of annealing lengths. We then extend this restart schedule
to parallel implementation, executing several Modified Lam simulated annealers
in parallel, with varying initial annealing lengths, and our proposed parallel
annealing length schedule. To validate our approach, we conduct experiments on
an NP-Hard scheduling problem with sequence-dependent setup constraints. We
compare our approach to fixed length restarts, both sequentially and in
parallel. Our results show that our approach can achieve substantial
performance gains, throughout the course of the run, demonstrating our approach
to be an effective anytime algorithm.
A Simple LSTM model for Transition-based Dependency Parsing
Authors: Mohab El-karef, Bernd Bohnet
Date: 2017-08-29 00:00:00
Tasks: Dependency Parsing, Transition-Based Dependency Parsing

We present a simple LSTM-based transition-based dependency parser. Our model
is composed of a single LSTM hidden layer replacing the hidden layer in the
usual feed-forward network architecture. We also propose a new initialization
method that uses the pre-trained weights from a feed-forward neural network to
initialize our LSTM-based model. We also show that using dropout on the input
layer has a positive effect on performance. Our final parser achieves a 93.06%
unlabeled and 91.01% labeled attachment score on the Penn Treebank. We
additionally replace LSTMs with GRUs and Elman units in our model and explore
the effectiveness of our initialization method on individual gates constituting
all three types of RNN units.
Language Modeling with Gated Convolutional Networks
Authors: Yann N. Dauphin, Angela Fan, Michael Auli, David Grangier
Date: 2016-12-23 00:00:00
Proceeding: ICML 2017 8
Tasks: Language Modeling, Language Modelling, Sentence

The pre-dominant approach to language modeling to date is based on recurrent
neural networks. Their success on this task is often linked to their ability to
capture unbounded context. In this paper we develop a finite context approach
through stacked convolutions, which can be more efficient since they allow
parallelization over sequential tokens. We propose a novel simplified gating
mechanism that outperforms Oord et al (2016) and investigate the impact of key
architectural decisions. The proposed approach achieves state-of-the-art on the
WikiText-103 benchmark, even though it features long-term dependencies, as well
as competitive results on the Google Billion Words benchmark. Our model reduces
the latency to score a sentence by an order of magnitude compared to a
recurrent baseline. To our knowledge, this is the first time a non-recurrent
approach is competitive with strong recurrent models on these large scale
language tasks.
Benchmarking Super-Resolution Algorithms on Real Data
Authors: Thomas Köhler, Michel Bätz, Farzad Naderi, André Kaup, Andreas K. Maier, Christian Riess
Date: 2017-09-08 00:00:00
Tasks: Benchmarking, Super-Resolution

Over the past decades, various super-resolution (SR) techniques have been
developed to enhance the spatial resolution of digital images. Despite the
great number of methodical contributions, there is still a lack of comparative
validations of SR under practical conditions, as capturing real ground truth
data is a challenging task. Therefore, current studies are either evaluated 1)
on simulated data or 2) on real data without a pixel-wise ground truth.
  To facilitate comprehensive studies, this paper introduces the publicly
available Super-Resolution Erlangen (SupER) database that includes real
low-resolution images along with high-resolution ground truth data. Our
database comprises image sequences with more than 20k images captured from 14
scenes under various types of motions and photometric conditions. The datasets
cover four spatial resolution levels using camera hardware binning. With this
database, we benchmark 15 single-image and multi-frame SR algorithms. Our
experiments quantitatively analyze SR accuracy and robustness under realistic
conditions including independent object and camera motion or photometric
variations.
Learning Texture Manifolds with the Periodic Spatial GAN
Authors: Urs Bergmann, Nikolay Jetchev, Roland Vollgraf
Date: 2017-05-18 00:00:00
Proceeding: ICML 2017 8
Tasks: Image Generation, Texture Synthesis

This paper introduces a novel approach to texture synthesis based on
generative adversarial networks (GAN) (Goodfellow et al., 2014). We extend the
structure of the input noise distribution by constructing tensors with
different types of dimensions. We call this technique Periodic Spatial GAN
(PSGAN). The PSGAN has several novel abilities which surpass the current state
of the art in texture synthesis. First, we can learn multiple textures from
datasets of one or more complex large images. Second, we show that the image
generation with PSGANs has properties of a texture manifold: we can smoothly
interpolate between samples in the structured noise space and generate novel
samples, which lie perceptually between the textures of the original dataset.
In addition, we can also accurately learn periodical textures. We make multiple
experiments which show that PSGANs can flexibly handle diverse texture and
image data sources. Our method is highly scalable and it can generate output
images of arbitrary large size.
Roll-back Hamiltonian Monte Carlo
Authors: Kexin Yi, Finale Doshi-Velez
Date: 2017-09-08 00:00:00
Tasks: Bayesian Inference, Computational Efficiency

We propose a new framework for Hamiltonian Monte Carlo (HMC) on truncated
probability distributions with smooth underlying density functions. Traditional
HMC requires computing the gradient of potential function associated with the
target distribution, and therefore does not perform its full power on truncated
distributions due to lack of continuity and differentiability. In our
framework, we introduce a sharp sigmoid factor in the density function to
approximate the probability drop at the truncation boundary. The target
potential function is approximated by a new potential which smoothly extends to
the entire sample space. HMC is then performed on the approximate potential.
While our method is easy to implement and applies to a wide range of problems,
it also achieves comparable computational efficiency on various sampling tasks
compared to other baseline methods. RBHMC also gives rise to a new approach for
Bayesian inference on constrained spaces.
Uncertainty measurement with belief entropy on interference effect in Quantum-Like Bayesian Networks
Authors: Zhiming Huang, Lin Yang, Wen Jiang
Date: 2017-09-08 00:00:00

Social dilemmas have been regarded as the essence of evolution game theory,
in which the prisoner's dilemma game is the most famous metaphor for the
problem of cooperation. Recent findings revealed people's behavior violated the
Sure Thing Principle in such games. Classic probability methodologies have
difficulty explaining the underlying mechanisms of people's behavior. In this
paper, a novel quantum-like Bayesian Network was proposed to accommodate the
paradoxical phenomenon. The special network can take interference into
consideration, which is likely to be an efficient way to describe the
underlying mechanism. With the assistance of belief entropy, named as Deng
entropy, the paper proposes Belief Distance to render the model practical.
Tested with empirical data, the proposed model is proved to be predictable and
effective.
CLaC at SemEval-2016 Task 11: Exploring linguistic and psycho-linguistic Features for Complex Word Identification
Authors: Elnaz Davoodi, Leila Kosseim
Date: 2017-09-08 00:00:00
Proceeding: SEMEVAL 2016 6
Tasks: Complex Word Identification

This paper describes the system deployed by the CLaC-EDLK team to the
"SemEval 2016, Complex Word Identification task". The goal of the task is to
identify if a given word in a given context is "simple" or "complex". Our
system relies on linguistic features and cognitive complexity. We used several
supervised models, however the Random Forest model outperformed the others.
Overall our best configuration achieved a G-score of 68.8% in the task, ranking
our system 21 out of 45.
Combining LSTM and Latent Topic Modeling for Mortality Prediction
Authors: Yohan Jo, Lisa Lee, Shruti Palaskar
Date: 2017-09-08 00:00:00
Tasks: Mortality Prediction, Prediction

There is a great need for technologies that can predict the mortality of
patients in intensive care units with both high accuracy and accountability. We
present joint end-to-end neural network architectures that combine long
short-term memory (LSTM) and a latent topic model to simultaneously train a
classifier for mortality prediction and learn latent topics indicative of
mortality from textual clinical notes. For topic interpretability, the topic
modeling layer has been carefully designed as a single-layer network with
constraints inspired by LDA. Experiments on the MIMIC-III dataset show that our
models significantly outperform prior models that are based on LDA topics in
mortality prediction. However, we achieve limited success with our method for
interpreting topics from the trained models by looking at the neural network
weights.
Globally Normalized Reader
Authors: Jonathan Raiman, John Miller
Date: 2017-09-08 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Data Augmentation, Question Answering, Sentence, valid

Rapid progress has been made towards question answering (QA) systems that can
extract answers from text. Existing neural approaches make use of expensive
bi-directional attention mechanisms or score all possible answer spans,
limiting scalability. We propose instead to cast extractive QA as an iterative
search problem: select the answer's sentence, start word, and end word. This
representation reduces the space of each search step and allows computation to
be conditionally allocated to promising search paths. We show that globally
normalizing the decision process and back-propagating through beam search makes
this representation viable and learning efficient. We empirically demonstrate
the benefits of this approach using our model, Globally Normalized Reader
(GNR), which achieves the second highest single model performance on the
Stanford Question Answering Dataset (68.4 EM, 76.21 F1 dev) and is 24.7x faster
than bi-attention-flow. We also introduce a data-augmentation method to produce
semantically valid examples by aligning named entities to a knowledge base and
swapping them with new entities of the same type. This method improves the
performance of all models considered in this work and is of independent
interest for a variety of NLP tasks.
Ultimate Intelligence Part III: Measures of Intelligence, Perception and Intelligent Agents
Authors: Eray Özkural
Date: 2017-09-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose that operator induction serves as an adequate model of perception.
We explain how to reduce universal agent models to operator induction. We
propose a universal measure of operator induction fitness, and show how it can
be used in a reinforcement learning model and a homeostasis (self-preserving)
agent based on the free energy principle. We show that the action of the
homeostasis agent can be explained by the operator induction model.
Gigamachine: incremental machine learning on desktop computers
Authors: Eray Özkural
Date: 2017-09-08 00:00:00
Tasks: BIG-bench Machine Learning

We present a concrete design for Solomonoff's incremental machine learning
system suitable for desktop computers. We use R5RS Scheme and its standard
library with a few omissions as the reference machine. We introduce a Levin
Search variant based on a stochastic Context Free Grammar together with new
update algorithms that use the same grammar as a guiding probability
distribution for incremental machine learning. The updates include adjusting
production probabilities, re-using previous solutions, learning programming
idioms and discovery of frequent subprograms. The issues of extending the a
priori probability distribution and bootstrapping are discussed. We have
implemented a good portion of the proposed algorithms. Experiments with toy
problems show that the update algorithms work as expected.
A Statistical Comparison of Some Theories of NP Word Order
Authors: Richard Futrell, Roger Levy, Matthew Dryer
Date: 2017-09-08 00:00:00
Tasks: regression

A frequent object of study in linguistic typology is the order of elements
{demonstrative, adjective, numeral, noun} in the noun phrase. The goal is to
predict the relative frequencies of these orders across languages. Here we use
Poisson regression to statistically compare some prominent accounts of this
variation. We compare feature systems derived from Cinque (2005) to feature
systems given in Cysouw (2010) and Dryer (in prep). In this setting, we do not
find clear reasons to prefer the model of Cinque (2005) or Dryer (in prep), but
we find both of these models have substantially better fit to the typological
data than the model from Cysouw (2010).
Detecting Hands in Egocentric Videos: Towards Action Recognition
Authors: Alejandro Cartas, Mariella Dimiccoli, Petia Radeva
Date: 2017-09-08 00:00:00
Tasks: Action Recognition, Hand Detection, Temporal Action Localization

Recently, there has been a growing interest in analyzing human daily
activities from data collected by wearable cameras. Since the hands are
involved in a vast set of daily tasks, detecting hands in egocentric images is
an important step towards the recognition of a variety of egocentric actions.
However, besides extreme illumination changes in egocentric images, hand
detection is not a trivial task because of the intrinsic large variability of
hand appearance. We propose a hand detector that exploits skin modeling for
fast hand proposal generation and Convolutional Neural Networks for hand
recognition. We tested our method on UNIGE-HANDS dataset and we showed that the
proposed approach achieves competitive hand detection results.
Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses
Authors: Chandrasekhar Bhagavatula, Chenchen Zhu, Khoa Luu, Marios Savvides
Date: 2017-07-18 00:00:00
Proceeding: ICCV 2017 10
Tasks: Face Alignment

Facial alignment involves finding a set of landmark points on an image with a
known semantic meaning. However, this semantic meaning of landmark points is
often lost in 2D approaches where landmarks are either moved to visible
boundaries or ignored as the pose of the face changes. In order to extract
consistent alignment points across large poses, the 3D structure of the face
must be considered in the alignment step. However, extracting a 3D structure
from a single 2D image usually requires alignment in the first place. We
present our novel approach to simultaneously extract the 3D shape of the face
and the semantically consistent 2D alignment through a 3D Spatial Transformer
Network (3DSTN) to model both the camera projection matrix and the warping
parameters of a 3D model. By utilizing a generic 3D model and a Thin Plate
Spline (TPS) warping function, we are able to generate subject specific 3D
shapes without the need for a large 3D shape basis. In addition, our proposed
network can be trained in an end-to-end framework on entirely synthetic data
from the 300W-LP dataset. Unlike other 3D methods, our approach only requires
one pass through the network resulting in a faster than real-time alignment.
Evaluations of our model on the Annotated Facial Landmarks in the Wild (AFLW)
and AFLW2000-3D datasets show our method achieves state-of-the-art performance
over other 3D approaches to alignment.
Machine learning \& artificial intelligence in the quantum domain
Authors: Vedran Dunjko, Hans J. Briegel
Date: 2017-09-08 00:00:00
Tasks: BIG-bench Machine Learning

Quantum information technologies, and intelligent learning systems, are both
emergent technologies that will likely have a transforming impact on our
society. The respective underlying fields of research -- quantum information
(QI) versus machine learning (ML) and artificial intelligence (AI) -- have
their own specific challenges, which have hitherto been investigated largely
independently. However, in a growing body of recent work, researchers have been
probing the question to what extent these fields can learn and benefit from
each other. QML explores the interaction between quantum computing and ML,
investigating how results and techniques from one field can be used to solve
the problems of the other. Recently, we have witnessed breakthroughs in both
directions of influence. For instance, quantum computing is finding a vital
application in providing speed-ups in ML, critical in our "big data" world.
Conversely, ML already permeates cutting-edge technologies, and may become
instrumental in advanced quantum technologies. Aside from quantum speed-up in
data analysis, or classical ML optimization used in quantum experiments,
quantum enhancements have also been demonstrated for interactive learning,
highlighting the potential of quantum-enhanced learning agents. Finally, works
exploring the use of AI for the very design of quantum experiments, and for
performing parts of genuine research autonomously, have reported their first
successes. Beyond the topics of mutual enhancement, researchers have also
broached the fundamental issue of quantum generalizations of ML/AI concepts.
This deals with questions of the very meaning of learning and intelligence in a
world that is described by quantum mechanics. In this review, we describe the
main ideas, recent developments, and progress in a broad spectrum of research
investigating machine learning and artificial intelligence in the quantum
domain.
Faster independent component analysis by preconditioning with Hessian approximations
Authors: Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort
Date: 2017-06-25 00:00:00

Independent Component Analysis (ICA) is a technique for unsupervised
exploration of multi-channel data that is widely used in observational
sciences. In its classic form, ICA relies on modeling the data as linear
mixtures of non-Gaussian independent sources. The maximization of the
corresponding likelihood is a challenging problem if it has to be completed
quickly and accurately on large sets of real data. We introduce the
Preconditioned ICA for Real Data (Picard) algorithm, which is a relative L-BFGS
algorithm preconditioned with sparse Hessian approximations. Extensive
numerical comparisons to several algorithms of the same class demonstrate the
superior performance of the proposed technique, especially on real data, for
which the ICA model does not necessarily hold.
Vessel Segmentation and Catheter Detection in X-Ray Angiograms Using Superpixels
Authors: Hamid R. Fazlali, Nader Karimi, S. M. Reza Soroushmehr, Shahram Shirani, Brahmajee. K. Nallamothu, Kevin R. Ward, Shadrokh Samavi, Kayvan Najarian
Date: 2017-09-08 00:00:00
Tasks: Segmentation, Superpixels

Coronary artery disease (CAD) is the leading causes of death around the
world. One of the most common imaging methods for diagnosing this disease is
X-ray angiography. Diagnosing using these images is usually challenging due to
non-uniform illumination, low contrast, presence of other body tissues,
presence of catheter etc. These challenges make the diagnoses task of
cardiologists tougher and more prone to misdiagnosis. In this paper we propose
a new automated framework for coronary arteries segmentation, catheter
detection and center-line extraction in x-ray angiography images. Our proposed
segmentation method is based on superpixels. In this method at first three
different superpixel scales are exploited and a measure for vesselness
probability of each superpixel is determined. A majority voting is used for
obtaining an initial segmentation map from these three superpixel scales. This
initial segmentation is refined by finding the orthogonal line on each ridge
pixel of vessel region. In this framework we use our catheter detection and
tracking method which detects the catheter by finding its ridge in the first
frame and traces in other frames by fitting a second order polynomial on it.
Also we use the image ridges for extracting the coronary arteries centerlines.
We evaluated our method qualitatively and quantitatively on two different
challenging datasets and compared it with one of the previous well-known
coronary arteries segmentation methods. Our method could detect the catheter
and reduced the false positive rate in addition to achieving better
segmentation results. The evaluation results prove that our method performs
better in a much shorter time.
Crowdsourcing Predictors of Residential Electric Energy Usage
Authors: Mark D. Wagy, Josh C. Bongard, James P. Bagrow, Paul D. H. Hines
Date: 2017-09-08 00:00:00
Tasks: Astronomy, valid

Crowdsourcing has been successfully applied in many domains including
astronomy, cryptography and biology. In order to test its potential for useful
application in a Smart Grid context, this paper investigates the extent to
which a crowd can contribute predictive hypotheses to a model of residential
electric energy consumption. In this experiment, the crowd generated hypotheses
about factors that make one home different from another in terms of monthly
energy usage. To implement this concept, we deployed a web-based system within
which 627 residential electricity customers posed 632 questions that they
thought predictive of energy usage. While this occurred, the same group
provided 110,573 answers to these questions as they accumulated. Thus users
both suggested the hypotheses that drive a predictive model and provided the
data upon which the model is built. We used the resulting question and answer
data to build a predictive model of monthly electric energy consumption, using
random forest regression. Because of the sparse nature of the answer data,
careful statistical work was needed to ensure that these models are valid. The
results indicate that the crowd can generate useful hypotheses, despite the
sparse nature of the dataset.
Cycles in adversarial regularized learning
Authors: Panayotis Mertikopoulos, Christos Papadimitriou, Georgios Piliouras
Date: 2017-09-08 00:00:00

Regularized learning is a fundamental technique in online optimization,
machine learning and many other fields of computer science. A natural question
that arises in these settings is how regularized learning algorithms behave
when faced against each other. We study a natural formulation of this problem
by coupling regularized learning dynamics in zero-sum games. We show that the
system's behavior is Poincar\'e recurrent, implying that almost every
trajectory revisits any (arbitrarily small) neighborhood of its starting point
infinitely often. This cycling behavior is robust to the agents' choice of
regularization mechanism (each agent could be using a different regularizer),
to positive-affine transformations of the agents' utilities, and it also
persists in the case of networked competition, i.e., for zero-sum polymatrix
games.