pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for Fatigue Estimation on the Basis of Multimodal Human-Machine Interactions
Authors: Yuri Gordienko, Sergii Stirenko, Yuriy Kochura, Oleg Alienin, Michail Novotarskiy, Nikita Gordienko
Date: 2017-12-30 00:00:00
Tasks: BIG-bench Machine Learning

The new method is proposed to monitor the level of current physical load and
accumulated fatigue by several objective and subjective characteristics. It was
applied to the dataset targeted to estimate the physical load and fatigue by
several statistical and machine learning methods. The data from peripheral
sensors (accelerometer, GPS, gyroscope, magnetometer) and brain-computing
interface (electroencephalography) were collected, integrated, and analyzed by
several statistical and machine learning methods (moment analysis, cluster
analysis, principal component analysis, etc.). The hypothesis 1 was presented
and proved that physical activity can be classified not only by objective
parameters, but by subjective parameters also. The hypothesis 2 (experienced
physical load and subsequent restoration as fatigue level can be estimated
quantitatively and distinctive patterns can be recognized) was presented and
some ways to prove it were demonstrated. Several "physical load" and "fatigue"
metrics were proposed. The results presented allow to extend application of the
machine learning methods for characterization of complex human activity
patterns (for example, to estimate their actual physical load and fatigue, and
give cautions and advice).
Towards co-evolution of fitness predictors and Deep Neural Networks
Authors: Włodzimierz Funika, Paweł Koperek
Date: 2017-12-30 00:00:00

Deep neural networks proved to be a very useful and powerful tool with many
practical applications. They especially excel at learning from large data sets
with labeled samples. However, in order to achieve good learning results, the
network architecture has to be carefully designed. Creating an optimal topology
requires a lot of experience and knowledge. Unfortunately there are no
practically applicable algorithms which could help in this situation. Using an
evolutionary process to develop new network topologies might solve this
problem. The limiting factor in this case is the speed of evaluation of a
single specimen (a single network architecture), which includes learning based
on the whole large dataset. In this paper we propose to overcome this problem
by using a fitness prediction technique: use subsets of the original training
set to conduct the training process and use its results as an approximation of
specimen's fitness. We discuss the feasibility of this approach in context of
the desired fitness predictor features and analyze whether subsets obtained in
an evolutionary process can be used to estimate the fitness of the network
topology. Finally we draw conclusions from our experiments and outline plans
for future work.
Default Logic and Bounded Treewidth
Authors: Johannes K. Fichte, Markus Hecher, Irina Schindler
Date: 2017-06-28 00:00:00

In this paper, we study Reiter's propositional default logic when the
treewidth of a certain graph representation (semi-primal graph) of the input
theory is bounded. We establish a dynamic programming algorithm on tree
decompositions that decides whether a theory has a consistent stable extension
(Ext). Our algorithm can even be used to enumerate all generating defaults
(ExtEnum) that lead to stable extensions.
  We show that our algorithm decides Ext in linear time in the input theory and
triple exponential time in the treewidth (so-called fixed-parameter linear
algorithm).
  Further, our algorithm solves ExtEnum with a pre-computation step that is
linear in the input theory and triple exponential in the treewidth followed by
a linear delay to output solutions.
Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference
Authors: Yi Tay, Luu Anh Tuan, Siu Cheung Hui
Date: 2017-12-30 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Natural Language Inference, Representation Learning

This paper presents a new deep learning architecture for Natural Language
Inference (NLI). Firstly, we introduce a new architecture where alignment pairs
are compared, compressed and then propagated to upper layers for enhanced
representation learning. Secondly, we adopt factorization layers for efficient
and expressive compression of alignment vectors into scalar features, which are
then used to augment the base word representations. The design of our approach
is aimed to be conceptually simple, compact and yet powerful. We conduct
experiments on three popular benchmarks, SNLI, MultiNLI and SciTail, achieving
competitive performance on all. A lightweight parameterization of our model
also enjoys a $\approx 3$ times reduction in parameter size compared to the
existing state-of-the-art models, e.g., ESIM and DIIN, while maintaining
competitive performance. Additionally, visual analysis shows that our
propagated features are highly interpretable.
Modeling Latent Attention Within Neural Networks
Authors: Christopher Grimm, Dilip Arumugam, Siddharth Karamcheti, David Abel, Lawson L. S. Wong, Michael L. Littman
Date: 2017-06-02 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decision Making, Reinforcement Learning

Deep neural networks are able to solve tasks across a variety of domains and
modalities of data. Despite many empirical successes, we lack the ability to
clearly understand and interpret the learned internal mechanisms that
contribute to such effective behaviors or, more critically, failure modes. In
this work, we present a general method for visualizing an arbitrary neural
network's inner mechanisms and their power and limitations. Our dataset-centric
method produces visualizations of how a trained network attends to components
of its inputs. The computed "attention masks" support improved interpretability
by highlighting which input attributes are critical in determining output. We
demonstrate the effectiveness of our framework on a variety of deep neural
network architectures in domains from computer vision, natural language
processing, and reinforcement learning. The primary contribution of our
approach is an interpretable visualization of attention that provides unique
insights into the network's underlying decision-making process irrespective of
the data modality.
Towards automated patient data cleaning using deep learning: A feasibility study on the standardization of organ labeling
Authors: Timothy Rozario, Troy Long, Mingli Chen, Weiguo Lu, Steve Jiang
Date: 2017-12-30 00:00:00

Data cleaning consumes about 80% of the time spent on data analysis for
clinical research projects. This is a much bigger problem in the era of big
data and machine learning in the field of medicine where large volumes of data
are being generated. We report an initial effort towards automated patient data
cleaning using deep learning: the standardization of organ labeling in
radiation therapy. Organs are often labeled inconsistently at different
institutions (sometimes even within the same institution) and at different time
periods, which poses a problem for clinical research, especially for
multi-institutional collaborative clinical research where the acquired patient
data is not being used effectively. We developed a convolutional neural network
(CNN) to automatically identify each organ in the CT image and then label it
with the standardized nomenclature presented at AAPM Task Group 263. We tested
this model on the CT images of 54 patients with prostate and 100 patients with
head and neck cancer who previously received radiation therapy. The model
achieved 100% accuracy in detecting organs and assigning standardized labels
for the patients tested. This work shows the feasibility of using deep learning
in patient data cleaning that enables standardized datasets to be generated for
effective intra- and interinstitutional collaborative clinical research.
Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs
Authors: Xing Di, Vishal M. Patel
Date: 2017-12-30 00:00:00
Tasks: Attribute, Face Generation

Automatic synthesis of faces from visual attributes is an important problem
in computer vision and has wide applications in law enforcement and
entertainment. With the advent of deep generative convolutional neural networks
(CNNs), attempts have been made to synthesize face images from attributes and
text descriptions. In this paper, we take a different approach, where we
formulate the original problem as a stage-wise learning problem. We first
synthesize the facial sketch corresponding to the visual attributes and then we
reconstruct the face image based on the synthesized sketch. The proposed
Attribute2Sketch2Face framework, which is based on a combination of deep
Conditional Variational Autoencoder (CVAE) and Generative Adversarial Networks
(GANs), consists of three stages: (1) Synthesis of facial sketch from
attributes using a CVAE architecture, (2) Enhancement of coarse sketches to
produce sharper sketches using a GAN-based framework, and (3) Synthesis of face
from sketch using another GAN-based network. Extensive experiments and
comparison with recent methods are performed to verify the effectiveness of the
proposed attribute-based three stage face synthesis method.
Deep Convolutional Neural Networks for Anomaly Event Classification on Distributed Systems
Authors: Jiechao Cheng, Rui Ren, Lei Wang, Jianfeng Zhan
Date: 2017-10-25 00:00:00
Tasks: Classification, General Classification

The increasing popularity of server usage has brought a plenty of anomaly log
events, which have threatened a vast collection of machines. Recognizing and
categorizing the anomalous events thereby is a much salient work for our
systems, especially the ones generate the massive amount of data and harness it
for technology value creation and business development. To assist in focusing
on the classification and the prediction of anomaly events, and gaining
critical insights from system event records, we propose a novel log
preprocessing method which is very effective to filter abundant information and
retain critical characteristics. Additionally, a competitive approach for
automated classification of anomalous events detected from the distributed
system logs with the state-of-the-art deep (Convolutional Neural Network)
architectures is proposed in this paper. We measure a series of deep CNN
algorithms with varied hyper-parameter combinations by using standard
evaluation metrics, the results of our study reveals the advantages and
potential capabilities of the proposed deep CNN models for anomaly event
classification tasks on real-world systems. The optimal classification
precision of our approach is 98.14%, which surpasses the popular traditional
machine learning methods.
Attentional Pooling for Action Recognition
Authors: Rohit Girdhar, Deva Ramanan
Date: 2017-11-04 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Action Recognition, Human-Object Interaction Detection, Temporal Action Localization

We introduce a simple yet surprisingly powerful model to incorporate
attention in action recognition and human object interaction tasks. Our
proposed attention module can be trained with or without extra supervision, and
gives a sizable boost in accuracy while keeping the network size and
computational cost nearly the same. It leads to significant improvements over
state of the art base architecture on three standard action recognition
benchmarks across still images and videos, and establishes new state of the art
on MPII dataset with 12.5% relative improvement. We also perform an extensive
analysis of our attention module both empirically and analytically. In terms of
the latter, we introduce a novel derivation of bottom-up and top-down attention
as low-rank approximations of bilinear pooling methods (typically used for
fine-grained classification). From this perspective, our attention formulation
suggests a novel characterization of action recognition as a fine-grained
recognition problem.
Particle Clustering Machine: A Dynamical System Based Approach
Authors: Sambarta Dasgupta, Keivan Ebrahimi, Umesh Vaidya
Date: 2017-12-30 00:00:00
Tasks: Clustering

Identification of the clusters from an unlabeled data set is one of the most
important problems in Unsupervised Machine Learning. The state of the art
clustering algorithms are based on either the statistical properties or the
geometric properties of the data set. In this work, we propose a novel method
to cluster the data points using dynamical systems theory. After constructing a
gradient dynamical system using interaction potential, we prove that the
asymptotic dynamics of this system will determine the cluster centers, when the
dynamical system is initialized at the data points. Most of the existing
heuristic-based clustering techniques suffer from a disadvantage, namely the
stochastic nature of the solution. Whereas, the proposed algorithm is
deterministic, and the outcome would not change over multiple runs of the
proposed algorithm with the same input data. Another advantage of the proposed
method is that the number of clusters, which is difficult to determine in
practice, does not have to be specified in advance. Simulation results with are
presented, and comparisons are made with the existing methods.
Dendritic error backpropagation in deep cortical microcircuits
Authors: João Sacramento, Rui Ponte Costa, Yoshua Bengio, Walter Senn
Date: 2017-12-30 00:00:00
Tasks: Denoising

Animal behaviour depends on learning to associate sensory stimuli with the
desired motor command. Understanding how the brain orchestrates the necessary
synaptic modifications across different brain areas has remained a longstanding
puzzle. Here, we introduce a multi-area neuronal network model in which
synaptic plasticity continuously adapts the network towards a global desired
output. In this model synaptic learning is driven by a local dendritic
prediction error that arises from a failure to predict the top-down input given
the bottom-up activities. Such errors occur at apical dendrites of pyramidal
neurons where both long-range excitatory feedback and local inhibitory
predictions are integrated. When local inhibition fails to match excitatory
feedback an error occurs which triggers plasticity at bottom-up synapses at
basal dendrites of the same pyramidal neurons. We demonstrate the learning
capabilities of the model in a number of tasks and show that it approximates
the classical error backpropagation algorithm. Finally, complementing this
cortical circuit with a disinhibitory mechanism enables attention-like stimulus
denoising and generation. Our framework makes several experimental predictions
on the function of dendritic integration and cortical microcircuits, is
consistent with recent observations of cross-area learning, and suggests a
biological implementation of deep learning.
Characterizing optimal hierarchical policy inference on graphs via non-equilibrium thermodynamics
Authors: Daniel McNamee
Date: 2017-12-29 00:00:00

Hierarchies are of fundamental interest in both stochastic optimal control
and biological control due to their facilitation of a range of desirable
computational traits in a control algorithm and the possibility that they may
form a core principle of sensorimotor and cognitive control systems. However, a
theoretically justified construction of state-space hierarchies over all
spatial resolutions and their evolution through a policy inference process
remains elusive. Here, a formalism for deriving such normative representations
of discrete Markov decision processes is introduced in the context of graphs.
The resulting hierarchies correspond to a hierarchical policy inference
algorithm approximating a discrete gradient flow between state-space trajectory
densities generated by the prior and optimal policies.
Deep Poisson Factorization Machines: factor analysis for mapping behaviors in journalist ecosystem
Authors: Pau Perng-Hwa Kung
Date: 2015-12-18 00:00:00
Tasks: Variational Inference

Newsroom in online ecosystem is difficult to untangle. With prevalence of
social media, interactions between journalists and individuals become visible,
but lack of understanding to inner processing of information feedback loop in
public sphere leave most journalists baffled. Can we provide an organized view
to characterize journalist behaviors on individual level to know better of the
ecosystem? To this end, I propose Poisson Factorization Machine (PFM), a
Bayesian analogue to matrix factorization that assumes Poisson distribution for
generative process. The model generalizes recent studies on Poisson Matrix
Factorization to account temporal interaction which involves tensor-like
structure, and label information. Two inference procedures are designed, one
based on batch variational EM and another stochastic variational inference
scheme that efficiently scales with data size. An important novelty in this
note is that I show how to stack layers of PFM to introduce a deep
architecture. This work discusses some potential results applying the model and
explains how such latent factors may be useful for analyzing latent behaviors
for data exploration.
Optimization and Testing in Linear Non-Gaussian Component Analysis
Authors: Ze Jin, Benjamin B. Risk, David S. Matteson
Date: 2017-12-23 00:00:00

Independent component analysis (ICA) decomposes multivariate data into
mutually independent components (ICs). The ICA model is subject to a constraint
that at most one of these components is Gaussian, which is required for model
identifiability. Linear non-Gaussian component analysis (LNGCA) generalizes the
ICA model to a linear latent factor model with any number of both non-Gaussian
components (signals) and Gaussian components (noise), where observations are
linear combinations of independent components. Although the individual Gaussian
components are not identifiable, the Gaussian subspace is identifiable. We
introduce an estimator along with its optimization approach in which
non-Gaussian and Gaussian components are estimated simultaneously, maximizing
the discrepancy of each non-Gaussian component from Gaussianity while
minimizing the discrepancy of each Gaussian component from Gaussianity. When
the number of non-Gaussian components is unknown, we develop a statistical test
to determine it based on resampling and the discrepancy of estimated
components. Through a variety of simulation studies, we demonstrate the
improvements of our estimator over competing estimators, and we illustrate the
effectiveness of the test to determine the number of non-Gaussian components.
Further, we apply our method to real data examples and demonstrate its
practical value.
A Deep Belief Network Based Machine Learning System for Risky Host Detection
Authors: Wangyan Feng, Shuning Wu, Xiaodan Li, Kevin Kunkle
Date: 2017-12-29 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Intrusion Detection, Management, Network Intrusion Detection

To assure cyber security of an enterprise, typically SIEM (Security
Information and Event Management) system is in place to normalize security
event from different preventive technologies and flag alerts. Analysts in the
security operation center (SOC) investigate the alerts to decide if it is truly
malicious or not. However, generally the number of alerts is overwhelming with
majority of them being false positive and exceeding the SOC's capacity to
handle all alerts. There is a great need to reduce the false positive rate as
much as possible. While most previous research focused on network intrusion
detection, we focus on risk detection and propose an intelligent Deep Belief
Network machine learning system. The system leverages alert information,
various security logs and analysts' investigation results in a real enterprise
environment to flag hosts that have high likelihood of being compromised. Text
mining and graph based method are used to generate targets and create features
for machine learning. In the experiment, Deep Belief Network is compared with
other machine learning algorithms, including multi-layer neural network, random
forest, support vector machine and logistic regression. Results on real
enterprise data indicate that the deep belief network machine learning system
performs better than other algorithms for our problem and is six times more
effective than current rule-based system. We also implement the whole system
from data collection, label creation, feature engineering to host score
generation in a real enterprise production environment.
Methods for Detecting Paraphrase Plagiarism
Authors: Victor Thompson
Date: 2017-12-29 00:00:00

Paraphrase plagiarism is one of the difficult challenges facing plagiarism
detection systems. Paraphrasing occur when texts are lexically or syntactically
altered to look different, but retain their original meaning. Most plagiarism
detection systems (many of which are commercial based) are designed to detect
word co-occurrences and light modifications, but are unable to detect severe
semantic and structural alterations such as what is seen in many academic
documents. Hence many paraphrase plagiarism cases go undetected. In this paper,
we approached the problem of paraphrase plagiarism by proposing methods for
detecting the most common techniques (phenomena) used in paraphrasing texts
(namely; lexical substitution, insertion/deletion and word and phrase
reordering), and combined the methods into a paraphrase detection model. We
evaluated our proposed methods and model on collections containing paraphrase
texts. Experimental results show significant improvement in performance when
the methods were combined (the proposed model) as opposed to running them
individually. The results also show that the proposed paraphrase detection
model outperformed a standard baseline (based on greedy string tilling), and
previous studies.
Classification and Representation via Separable Subspaces: Performance Limits and Algorithms
Authors: Ishan Jindal, Matthew Nokleby
Date: 2017-05-07 00:00:00
Tasks: Classification, Dictionary Learning, Diversity, General Classification

We study the classification performance of Kronecker-structured models in two
asymptotic regimes and developed an algorithm for separable, fast and compact
K-S dictionary learning for better classification and representation of
multidimensional signals by exploiting the structure in the signal. First, we
study the classification performance in terms of diversity order and pairwise
geometry of the subspaces. We derive an exact expression for the diversity
order as a function of the signal and subspace dimensions of a K-S model. Next,
we study the classification capacity, the maximum rate at which the number of
classes can grow as the signal dimension goes to infinity. Then we describe a
fast algorithm for Kronecker-Structured Learning of Discriminative Dictionaries
(K-SLD2). Finally, we evaluate the empirical classification performance of K-S
models for the synthetic data, showing that they agree with the diversity order
analysis. We also evaluate the performance of K-SLD2 on synthetic and
real-world datasets showing that the K-SLD2 balances compact signal
representation and good classification performance.