pwc-archive/papers-with-abstracts huggingface dataset:
Comparative Analysis of Open Source Frameworks for Machine Learning with Use Case in Single-Threaded and Multi-Threaded Modes
Authors: Yuriy Kochura, Sergii Stirenko, Anis Rojbi, Oleg Alienin, Michail Novotarskiy, Yuri Gordienko
Date: 2017-06-07 00:00:00
Tasks: BIG-bench Machine Learning, CPU, GPU

The basic features of some of the most versatile and popular open source
frameworks for machine learning (TensorFlow, Deep Learning4j, and H2O) are
considered and compared. Their comparative analysis was performed and
conclusions were made as to the advantages and disadvantages of these
platforms. The performance tests for the de facto standard MNIST data set were
carried out on H2O framework for deep learning algorithms designed for CPU and
GPU platforms for single-threaded and multithreaded modes of operation.
Stochastic Global Optimization Algorithms: A Systematic Formal Approach
Authors: Jonatan Gomez
Date: 2017-06-07 00:00:00
Tasks: global-optimization

As we know, some global optimization problems cannot be solved using analytic
methods, so numeric/algorithmic approaches are used to find near to the optimal
solutions for them. A stochastic global optimization algorithm (SGoal) is an
iterative algorithm that generates a new population (a set of candidate
solutions) from a previous population using stochastic operations. Although
some research works have formalized SGoals using Markov kernels, such
formalization is not general and sometimes is blurred. In this paper, we
propose a comprehensive and systematic formal approach for studying SGoals.
First, we present the required theory of probability (\sigma-algebras,
measurable functions, kernel, markov chain, products, convergence and so on)
and prove that some algorithmic functions like swapping and projection can be
represented by kernels. Then, we introduce the notion of join-kernel as a way
of characterizing the combination of stochastic methods. Next, we define the
optimization space, a formal structure (a set with a \sigma-algebra that
contains strict \epsilon-optimal states) for studying SGoals, and we develop
kernels, like sort and permutation, on such structure. Finally, we present some
popular SGoals in terms of the developed theory, we introduce sufficient
conditions for convergence of a SGoal, and we prove convergence of some popular
SGoals.
Insights into Analogy Completion from the Biomedical Domain
Authors: Denis Newman-Griffis, Albert M. Lai, Eric Fosler-Lussier
Date: 2017-06-07 00:00:00
Proceeding: WS 2017 8
Tasks: Word Embeddings

Analogy completion has been a popular task in recent years for evaluating the
semantic properties of word embeddings, but the standard methodology makes a
number of assumptions about analogies that do not always hold, either in recent
benchmark datasets or when expanding into other domains. Through an analysis of
analogies in the biomedical domain, we identify three assumptions: that of a
Single Answer for any given analogy, that the pairs involved describe the Same
Relationship, and that each pair is Informative with respect to the other. We
propose modifying the standard methodology to relax these assumptions by
allowing for multiple correct answers, reporting MAP and MRR in addition to
accuracy, and using multiple example pairs. We further present BMASS, a novel
dataset for evaluating linguistic regularities in biomedical embeddings, and
demonstrate that the relationships described in the dataset pose significant
semantic challenges to current word embedding methods.
Efficient Reinforcement Learning via Initial Pure Exploration
Authors: Sudeep Raja Putta, Theja Tulabandhula
Date: 2017-06-07 00:00:00
Tasks: Multi-Armed Bandits, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In several realistic situations, an interactive learning agent can practice
and refine its strategy before going on to be evaluated. For instance, consider
a student preparing for a series of tests. She would typically take a few
practice tests to know which areas she needs to improve upon. Based of the
scores she obtains in these practice tests, she would formulate a strategy for
maximizing her scores in the actual tests. We treat this scenario in the
context of an agent exploring a fixed-horizon episodic Markov Decision Process
(MDP), where the agent can practice on the MDP for some number of episodes (not
necessarily known in advance) before starting to incur regret for its actions.
  During practice, the agent's goal must be to maximize the probability of
following an optimal policy. This is akin to the problem of Pure Exploration
(PE). We extend the PE problem of Multi Armed Bandits (MAB) to MDPs and propose
a Bayesian algorithm called Posterior Sampling for Pure Exploration (PSPE),
which is similar to its bandit counterpart. We show that the Bayesian simple
regret converges at an optimal exponential rate when using PSPE.
  When the agent starts being evaluated, its goal would be to minimize the
cumulative regret incurred. This is akin to the problem of Reinforcement
Learning (RL). The agent uses the Posterior Sampling for Reinforcement Learning
algorithm (PSRL) initialized with the posteriors of the practice phase. We
hypothesize that this PSPE + PSRL combination is an optimal strategy for
minimizing regret in RL problems with an initial practice phase. We show
empirical results which prove that having a lower simple regret at the end of
the practice phase results in having lower cumulative regret during evaluation.
Gated Recurrent Neural Tensor Network
Authors: Andros Tjandra, Sakriani Sakti, Ruli Manurung, Mirna Adriani, Satoshi Nakamura
Date: 2017-06-07 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling
temporal and sequential data need to capture long-term dependencies on datasets
and represent them in hidden layers with a powerful model to capture more
information from inputs. For modeling long-term dependencies in a dataset, the
gating mechanism concept can help RNNs remember and forget previous
information. Representing the hidden layers of an RNN with more expressive
operations (i.e., tensor products) helps it learn a more complex relationship
between the current input and the previous hidden layer information. These
ideas can generally improve RNN performances. In this paper, we proposed a
novel RNN architecture that combine the concepts of gating mechanism and the
tensor product into a single model. By combining these two concepts into a
single RNN, our proposed models learn long-term dependencies by modeling with
gating units and obtain more expressive and direct interaction between input
and hidden layers using a tensor product on 3-dimensional array (tensor) weight
parameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit
(GRU) RNN and combine them with a tensor product inside their formulations. Our
proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural
Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor
Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the
tensor product. We conducted experiments with our proposed models on word-level
and character-level language modeling tasks and revealed that our proposed
models significantly improved their performance compared to our baseline
models.
Anytime Monte Carlo
Authors: Lawrence M. Murray, Sumeetpal Singh, Anthony Lee
Date: 2016-12-10 00:00:00

Monte Carlo algorithms simulate some prescribed number of samples, taking some random real time to complete the computations necessary. This work considers the converse: to impose a real-time budget on the computation, which results in the number of samples simulated being random. To complicate matters, the real time taken for each simulation may depend on the sample produced, so that the samples themselves are not independent of their number, and a length bias with respect to compute time is apparent. This is especially problematic when a Markov chain Monte Carlo (MCMC) algorithm is used and the final state of the Markov chain -- rather than an average over all states -- is required, which is the case in parallel tempering implementations of MCMC. The length bias does not diminish with the compute budget in this case. It also occurs in sequential Monte Carlo (SMC) algorithms, which is the focus of this paper. We propose an anytime framework to address the concern, using a continuous-time Markov jump process to study the progress of the computation in real time. We first show that for any MCMC algorithm, the length bias of the final state's distribution due to the imposed real-time computing budget can be eliminated by using a multiple chain construction. The utility of this construction is then demonstrated on a large-scale SMC^2 implementation, using four billion particles distributed across a cluster of 128 graphics processing units on the Amazon EC2 service. The anytime framework imposes a real-time budget on the MCMC move steps within the SMC$^{2}$ algorithm, ensuring that all processors are simultaneously ready for the resampling step, demonstrably reducing idleness to due waiting times and providing substantial control over the total compute budget.
Improving Max-Sum through Decimation to Solve Loopy Distributed Constraint Optimization Problems
Authors: Jesús Cerquides, Rémi Emonet, Gauthier Picard, Juan A. Rodríguez-Aguilar
Date: 2017-06-07 00:00:00

In the context of solving large distributed constraint optimization problems
(DCOP), belief-propagation and approximate inference algorithms are candidates
of choice. However, in general, when the factor graph is very loopy (i.e.
cyclic), these solution methods suffer from bad performance, due to
non-convergence and many exchanged messages. As to improve performances of the
Max-Sum inference algorithm when solving loopy constraint optimization
problems, we propose here to take inspiration from the
belief-propagation-guided dec-imation used to solve sparse random graphs
(k-satisfiability). We propose the novel DeciMaxSum method, which is
parameterized in terms of policies to decide when to trigger decimation, which
variables to decimate, and which values to assign to decimated variables. Based
on an empirical evaluation on a classical BP benchmark (the Ising model), some
of these combinations of policies exhibit better performance than
state-of-the-art competitors.
Synthesizing Filamentary Structured Images with GANs
Authors: He Zhao, Huiqi Li, Li Cheng
Date: 2017-06-07 00:00:00
Tasks: Medical Image Analysis, Style Transfer

This paper aims at synthesizing filamentary structured images such as retinal
fundus images and neuronal images, as follows: Given a ground-truth, to
generate multiple realistic looking phantoms. A ground-truth could be a binary
segmentation map containing the filamentary structured morphology, while the
synthesized output image is of the same size as the ground-truth and has
similar visual appearance to what have been presented in the training set. Our
approach is inspired by the recent progresses in generative adversarial nets
(GANs) as well as image style transfer. In particular, it is dedicated to our
problem context with the following properties: Rather than large-scale dataset,
it works well in the presence of as few as 10 training examples, which is
common in medical image analysis; It is capable of synthesizing diverse images
from the same ground-truth; Last and importantly, the synthetic images produced
by our approach are demonstrated to be useful in boosting image analysis
performance. Empirical examination over various benchmarks of fundus and
neuronal images demonstrate the advantages of the proposed approach.
Toward a Better Understanding of Leaderboard
Authors: Wenjie Zheng
Date: 2015-10-12 00:00:00
Tasks: BIG-bench Machine Learning

The leaderboard in machine learning competitions is a tool to show the
performance of various participants and to compare them. However, the
leaderboard quickly becomes no longer accurate, due to hack or overfitting.
This article gives two pieces of advice to prevent easy hack or overfitting. By
following these advice, we reach the conclusion that something like the Ladder
leaderboard introduced in [blum2015ladder] is inevitable. With this
understanding, we naturally simplify Ladder by eliminating its redundant
computation and explain how to choose the parameter and interpret it. We also
prove that the sample complexity is cubic to the desired precision of the
leaderboard.
Time Stretch Inspired Computational Imaging
Authors: Bahram Jalali, Madhuri Suthar, Mohamad Asghari, Ata Mahjoubfar
Date: 2017-06-07 00:00:00

We show that dispersive propagation of light followed by phase detection has
properties that can be exploited for extracting features from the waveforms.
This discovery is spearheading development of a new class of physics-inspired
algorithms for feature extraction from digital images with unique properties
and superior dynamic range compared to conventional algorithms. In certain
cases, these algorithms have the potential to be an energy efficient and
scalable substitute to synthetically fashioned computational techniques in
practice today.
Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition
Authors: Hong Liu, Juanhui Tu, Mengyuan Liu
Date: 2017-05-23 00:00:00
Tasks: 3D Action Recognition, Action Recognition, Skeleton Based Action Recognition, Temporal Action Localization, Vocal Bursts Valence Prediction

It remains a challenge to efficiently extract spatialtemporal information
from skeleton sequences for 3D human action recognition. Although most recent
action recognition methods are based on Recurrent Neural Networks which present
outstanding performance, one of the shortcomings of these methods is the
tendency to overemphasize the temporal information. Since 3D convolutional
neural network(3D CNN) is a powerful tool to simultaneously learn features from
both spatial and temporal dimensions through capturing the correlations between
three dimensional signals, this paper proposes a novel two-stream model using
3D CNN. To our best knowledge, this is the first application of 3D CNN in
skeleton-based action recognition. Our method consists of three stages. First,
skeleton joints are mapped into a 3D coordinate space and then encoding the
spatial and temporal information, respectively. Second, 3D CNN models are
seperately adopted to extract deep features from two streams. Third, to enhance
the ability of deep features to capture global relationships, we extend every
stream into multitemporal version. Extensive experiments on the SmartHome
dataset and the large-scale NTU RGB-D dataset demonstrate that our method
outperforms most of RNN-based methods, which verify the complementary property
between spatial and temporal information and the robustness to noise.
A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized Nonnegative Matrix Factorization
Authors: Renbo Zhao, Vincent Y. F. Tan
Date: 2016-09-04 00:00:00

The multiplicative update (MU) algorithm has been extensively used to
estimate the basis and coefficient matrices in nonnegative matrix factorization
(NMF) problems under a wide range of divergences and regularizers. However,
theoretical convergence guarantees have only been derived for a few special
divergences without regularization. In this work, we provide a conceptually
simple, self-contained, and unified proof for the convergence of the MU
algorithm applied on NMF with a wide range of divergences and regularizers. Our
main result shows the sequence of iterates (i.e., pairs of basis and
coefficient matrices) produced by the MU algorithm converges to the set of
stationary points of the non-convex NMF optimization problem. Our proof
strategy has the potential to open up new avenues for analyzing similar
problems in machine learning and signal processing.
Generative-Discriminative Variational Model for Visual Recognition
Authors: Chih-Kuan Yeh, Yao-Hung Hubert Tsai, Yu-Chiang Frank Wang
Date: 2017-06-07 00:00:00
Tasks: Classification, General Classification, model, Multi-class Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Zero-Shot Learning

The paradigm shift from shallow classifiers with hand-crafted features to
end-to-end trainable deep learning models has shown significant improvements on
supervised learning tasks. Despite the promising power of deep neural networks
(DNN), how to alleviate overfitting during training has been a research topic
of interest. In this paper, we present a Generative-Discriminative Variational
Model (GDVM) for visual classification, in which we introduce a latent variable
inferred from inputs for exhibiting generative abilities towards prediction. In
other words, our GDVM casts the supervised learning task as a generative
learning process, with data discrimination to be jointly exploited for improved
classification. In our experiments, we consider the tasks of multi-class
classification, multi-label classification, and zero-shot learning. We show
that our GDVM performs favorably against the baselines or recent generative DNN
models.
Guided Interaction Exploration in Artifact-centric Process Models
Authors: Maikel L. van Eck, Natalia Sidorova, Wil M. P. van der Aalst
Date: 2017-06-07 00:00:00

Artifact-centric process models aim to describe complex processes as a
collection of interacting artifacts. Recent development in process mining allow
for the discovery of such models. However, the focus is often on the
representation of the individual artifacts rather than their interactions.
Based on event data we can automatically discover composite state machines
representing artifact-centric processes. Moreover, we provide ways of
visualizing and quantifying interactions among different artifacts. For
example, we are able to highlight strongly correlated behaviours in different
artifacts. The approach has been fully implemented as a ProM plug-in; the CSM
Miner provides an interactive artifact-centric process discovery tool focussing
on interactions. The approach has been evaluated using real life data sets,
including the personal loan and overdraft process of a Dutch financial
institution.
Visual Servoing from Deep Neural Networks
Authors: Quentin Bateux, Eric Marchand, Jürgen Leitner, Francois Chaumette, Peter Corke
Date: 2017-05-24 00:00:00

We present a deep neural network-based method to perform high-precision,
robust and real-time 6 DOF visual servoing. The paper describes how to create a
dataset simulating various perturbations (occlusions and lighting conditions)
from a single real-world image of the scene. A convolutional neural network is
fine-tuned using this dataset to estimate the relative pose between two images
of the same scene. The output of the network is then employed in a visual
servoing control scheme. The method converges robustly even in difficult
real-world settings with strong lighting variations and occlusions.A
positioning error of less than one millimeter is obtained in experiments with a
6 DOF robot.
Cascade Ranking for Operational E-commerce Search
Authors: Shichen Liu, Fei Xiao, Wenwu Ou, Luo Si
Date: 2017-06-07 00:00:00
Tasks: CPU

In the 'Big Data' era, many real-world applications like search involve the
ranking problem for a large number of items. It is important to obtain
effective ranking results and at the same time obtain the results efficiently
in a timely manner for providing good user experience and saving computational
costs. Valuable prior research has been conducted for learning to efficiently
rank like the cascade ranking (learning) model, which uses a sequence of
ranking functions to progressively filter some items and rank the remaining
items. However, most existing research of learning to efficiently rank in
search is studied in a relatively small computing environments with simulated
user queries.
  This paper presents novel research and thorough study of designing and
deploying a Cascade model in a Large-scale Operational E-commerce Search
application (CLOES), which deals with hundreds of millions of user queries per
day with hundreds of servers. The challenge of the real-world application
provides new insights for research: 1). Real-world search applications often
involve multiple factors of preferences or constraints with respect to user
experience and computational costs such as search accuracy, search latency,
size of search results and total CPU cost, while most existing search solutions
only address one or two factors; 2). Effectiveness of e-commerce search
involves multiple types of user behaviors such as click and purchase, while
most existing cascade ranking in search only models the click behavior. Based
on these observations, a novel cascade ranking model is designed and deployed
in an operational e-commerce search application. An extensive set of
experiments demonstrate the advantage of the proposed work to address multiple
factors of effectiveness, efficiency and user experience in the real-world
application.
Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction
Authors: Qiuhong Ke, Mohammed Bennamoun, Senjian An, Farid Bossaid, Ferdous Sohel
Date: 2016-08-18 00:00:00
Tasks: Optical Flow Estimation

Predicting an interaction before it is fully executed is very important in
applications such as human-robot interaction and video surveillance. In a
two-human interaction scenario, there often contextual dependency structure
between the global interaction context of the two humans and the local context
of the different body parts of each human. In this paper, we propose to learn
the structure of the interaction contexts, and combine it with the spatial and
temporal information of a video sequence for a better prediction of the
interaction class. The structural models, including the spatial and the
temporal models, are learned with Long Short Term Memory (LSTM) networks to
capture the dependency of the global and local contexts of each RGB frame and
each optical flow image, respectively. LSTM networks are also capable of
detecting the key information from the global and local interaction contexts.
Moreover, to effectively combine the structural models with the spatial and
temporal models for interaction prediction, a ranking score fusion method is
also introduced to automatically compute the optimal weight of each model for
score fusion. Experimental results on the BIT Interaction and the
UT-Interaction datasets clearly demonstrate the benefits of the proposed
method.
Astronomical image reconstruction with convolutional neural networks
Authors: Rémi Flamary
Date: 2016-12-14 00:00:00
Tasks: Astronomy, Image Reconstruction

State of the art methods in astronomical image reconstruction rely on the
resolution of a regularized or constrained optimization problem. Solving this
problem can be computationally intensive and usually leads to a quadratic or at
least superlinear complexity w.r.t. the number of pixels in the image. We
investigate in this work the use of convolutional neural networks for image
reconstruction in astronomy. With neural networks, the computationally
intensive tasks is the training step, but the prediction step has a fixed
complexity per pixel, i.e. a linear complexity. Numerical experiments show that
our approach is both computationally efficient and competitive with other state
of the art methods in addition to being interpretable.