pwc-archive/papers-with-abstracts huggingface dataset:
Distributed Collaborative Hashing and Its Applications in Ant Financial
Authors: Chaochao Chen, Ziqi Liu, Peilin Zhao, Longfei Li, Jun Zhou, Xiaolong Li
Date: 2018-04-13 00:00:00
Tasks: Collaborative Filtering

Collaborative filtering, especially latent factor model, has been popularly
used in personalized recommendation. Latent factor model aims to learn user and
item latent factors from user-item historic behaviors. To apply it into real
big data scenarios, efficiency becomes the first concern, including offline
model training efficiency and online recommendation efficiency. In this paper,
we propose a Distributed Collaborative Hashing (DCH) model which can
significantly improve both efficiencies. Specifically, we first propose a
distributed learning framework, following the state-of-the-art parameter server
paradigm, to learn the offline collaborative model. Our model can be learnt
efficiently by distributedly computing subgradients in minibatches on workers
and updating model parameters on servers asynchronously. We then adopt hashing
technique to speedup the online recommendation procedure. Recommendation can be
quickly made through exploiting lookup hash tables. We conduct thorough
experiments on two real large-scale datasets. The experimental results
demonstrate that, comparing with the classic and state-of-the-art (distributed)
latent factor models, DCH has comparable performance in terms of recommendation
accuracy but has both fast convergence speed in offline model training
procedure and realtime efficiency in online recommendation procedure.
Furthermore, the encouraging performance of DCH is also shown for several
real-world applications in Ant Financial.
Grow and Prune Compact, Fast, and Accurate LSTMs
Authors: Xiaoliang Dai, Hongxu Yin, Niraj K. Jha
Date: 2018-05-30 00:00:00
Tasks: Image Captioning, speech-recognition, Speech Recognition

Long short-term memory (LSTM) has been widely used for sequential data
modeling. Researchers have increased LSTM depth by stacking LSTM cells to
improve performance. This incurs model redundancy, increases run-time delay,
and makes the LSTMs more prone to overfitting. To address these problems, we
propose a hidden-layer LSTM (H-LSTM) that adds hidden layers to LSTM's original
one level non-linear control gates. H-LSTM increases accuracy while employing
fewer external stacked layers, thus reducing the number of parameters and
run-time latency significantly. We employ grow-and-prune (GP) training to
iteratively adjust the hidden layers through gradient-based growth and
magnitude-based pruning of connections. This learns both the weights and the
compact architecture of H-LSTM control gates. We have GP-trained H-LSTMs for
image captioning and speech recognition applications. For the NeuralTalk
architecture on the MSCOCO dataset, our three models reduce the number of
parameters by 38.7x [floating-point operations (FLOPs) by 45.5x], run-time
latency by 4.5x, and improve the CIDEr score by 2.6. For the DeepSpeech2
architecture on the AN4 dataset, our two models reduce the number of parameters
by 19.4x (FLOPs by 23.5x), run-time latency by 15.7%, and the word error rate
from 12.9% to 8.7%. Thus, GP-trained H-LSTMs can be seen to be compact, fast,
and accurate.
Conformation Clustering of Long MD Protein Dynamics with an Adversarial Autoencoder
Authors: Yunlong Liu, L. Mario Amzel
Date: 2018-05-31 00:00:00
Tasks: Clustering, GPU

Recent developments in specialized computer hardware have greatly accelerated
atomic level Molecular Dynamics (MD) simulations. A single GPU-attached cluster
is capable of producing microsecond-length trajectories in reasonable amounts
of time. Multiple protein states and a large number of microstates associated
with folding and with the function of the protein can be observed as
conformations sampled in the trajectories. Clustering those conformations,
however, is needed for identifying protein states, evaluating transition rates
and understanding protein behavior. In this paper, we propose a novel
data-driven generative conformation clustering method based on the adversarial
autoencoder (AAE) and provide the associated software implementation Cong. The
method was tested using a 208 microseconds MD simulation of the fast-folding
peptide Trp-Cage (20 residues) obtained from the D.E. Shaw Research Group. The
proposed clustering algorithm identifies many of the salient features of the
folding process by grouping a large number of conformations that share common
features not easily identifiable in the trajectory.
Attention-Based LSTM for Psychological Stress Detection from Spoken Language Using Distant Supervision
Authors: Genta Indra Winata, Onno Pepijn Kampman, Pascale Fung
Date: 2018-05-31 00:00:00

We propose a Long Short-Term Memory (LSTM) with attention mechanism to
classify psychological stress from self-conducted interview transcriptions. We
apply distant supervision by automatically labeling tweets based on their
hashtag content, which complements and expands the size of our corpus. This
additional data is used to initialize the model parameters, and which it is
fine-tuned using the interview data. This improves the model's robustness,
especially by expanding the vocabulary size. The bidirectional LSTM model with
attention is found to be the best model in terms of accuracy (74.1%) and
f-score (74.3%). Furthermore, we show that distant supervision fine-tuning
enhances the model's performance by 1.6% accuracy and 2.1% f-score. The
attention mechanism helps the model to select informative words.
Data science is science's second chance to get causal inference right: A classification of data science tasks
Authors: Miguel A. Hernán, John Hsu, Brian Healy
Date: 2018-04-28 00:00:00
Tasks: Causal Inference, counterfactual, Decision Making, General Classification

Causal inference from observational data is the goal of many data analyses in
the health and social sciences. However, academic statistics has often frowned
upon data analyses with a causal objective. The introduction of the term "data
science" provides a historic opportunity to redefine data analysis in such a
way that it naturally accommodates causal inference from observational data.
Like others before, we organize the scientific contributions of data science
into three classes of tasks: Description, prediction, and counterfactual
prediction (which includes causal inference). An explicit classification of
data science tasks is necessary to discuss the data, assumptions, and analytics
required to successfully accomplish each task. We argue that a failure to
adequately describe the role of subject-matter expert knowledge in data
analysis is a source of widespread misunderstandings about data science.
Specifically, causal analyses typically require not only good data and
algorithms, but also domain expert knowledge. We discuss the implications for
the use of data science to guide decision-making in the real world and to train
data scientists.
Adversarial Attacks on Face Detectors using Neural Net based Constrained Optimization
Authors: Avishek Joey Bose, Parham Aarabi
Date: 2018-05-31 00:00:00
Tasks: Adversarial Attack, image-classification, Image Classification, object-detection, Object Detection

Adversarial attacks involve adding, small, often imperceptible, perturbations
to inputs with the goal of getting a machine learning model to misclassifying
them. While many different adversarial attack strategies have been proposed on
image classification models, object detection pipelines have been much harder
to break. In this paper, we propose a novel strategy to craft adversarial
examples by solving a constrained optimization problem using an adversarial
generator network. Our approach is fast and scalable, requiring only a forward
pass through our trained generator network to craft an adversarial sample.
Unlike in many attack strategies, we show that the same trained generator is
capable of attacking new images without explicitly optimizing on them. We
evaluate our attack on a trained Faster R-CNN face detector on the cropped
300-W face dataset where we manage to reduce the number of detected faces to
$0.5\%$ of all originally detected faces. In a different experiment, also on
300-W, we demonstrate the robustness of our attack to a JPEG compression based
defense typical JPEG compression level of $75\%$ reduces the effectiveness of
our attack from only $0.5\%$ of detected faces to a modest $5.0\%$.
Rotation Equivariance and Invariance in Convolutional Neural Networks
Authors: Benjamin Chidester, Minh N. Do, Jian Ma
Date: 2018-05-31 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Performance of neural networks can be significantly improved by encoding
known invariance for particular tasks. Many image classification tasks, such as
those related to cellular imaging, exhibit invariance to rotation. We present a
novel scheme using the magnitude response of the 2D-discrete-Fourier transform
(2D-DFT) to encode rotational invariance in neural networks, along with a new,
efficient convolutional scheme for encoding rotational equivariance throughout
convolutional layers. We implemented this scheme for several image
classification tasks and demonstrated improved performance, in terms of
classification accuracy, time required to train the model, and robustness to
hyperparameter selection, over a standard CNN and another state-of-the-art
method.
Algebraic Expression of Subjective Spatial and Temporal Patterns
Authors: Chuyu Xiong
Date: 2018-05-26 00:00:00
Tasks: BIG-bench Machine Learning

Universal learning machine is a theory trying to study machine learning from
mathematical point of view. The outside world is reflected inside an universal
learning machine according to pattern of incoming data. This is subjective
pattern of learning machine. In [2,4], we discussed subjective spatial pattern,
and established a powerful tool -- X-form, which is an algebraic expression for
subjective spatial pattern. However, as the initial stage of study, there we
only discussed spatial pattern. Here, we will discuss spatial and temporal
patterns, and algebraic expression for them.
Evaluating Reinforcement Learning Algorithms in Observational Health Settings
Authors: Omer Gottesman, Fredrik Johansson, Joshua Meier, Jack Dent, Dong-hun Lee, Srivatsan Srinivasan, Linying Zhang, Yi Ding, David Wihl, Xuefeng Peng, Jiayu Yao, Isaac Lage, Christopher Mosch, Li-wei H. Lehman, Matthieu Komorowski, Aldo Faisal, Leo Anthony Celi, David Sontag, Finale Doshi-Velez
Date: 2018-05-31 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Much attention has been devoted recently to the development of machine
learning algorithms with the goal of improving treatment policies in
healthcare. Reinforcement learning (RL) is a sub-field within machine learning
that is concerned with learning how to make sequences of decisions so as to
optimize long-term effects. Already, RL algorithms have been proposed to
identify decision-making strategies for mechanical ventilation, sepsis
management and treatment of schizophrenia. However, before implementing
treatment policies learned by black-box algorithms in high-stakes clinical
decision problems, special care must be taken in the evaluation of these
policies.
  In this document, our goal is to expose some of the subtleties associated
with evaluating RL algorithms in healthcare. We aim to provide a conceptual
starting point for clinical and computational researchers to ask the right
questions when designing and evaluating algorithms for new ways of treating
patients. In the following, we describe how choices about how to summarize a
history, variance of statistical estimators, and confounders in more ad-hoc
measures can result in unreliable, even misleading estimates of the quality of
a treatment policy. We also provide suggestions for mitigating these
effects---for while there is much promise for mining observational health data
to uncover better treatment policies, evaluation must be performed
thoughtfully.
Root-cause Analysis for Time-series Anomalies via Spatiotemporal Graphical Modeling in Distributed Complex Systems
Authors: Chao Liu, Kin Gwn Lore, Zhanhong Jiang, Soumik Sarkar
Date: 2018-05-31 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis

Performance monitoring, anomaly detection, and root-cause analysis in complex
cyber-physical systems (CPSs) are often highly intractable due to widely
diverse operational modes, disparate data types, and complex fault propagation
mechanisms. This paper presents a new data-driven framework for root-cause
analysis, based on a spatiotemporal graphical modeling approach built on the
concept of symbolic dynamics for discovering and representing causal
interactions among sub-systems of complex CPSs. We formulate the root-cause
analysis problem as a minimization problem via the proposed inference based
metric and present two approximate approaches for root-cause analysis, namely
the sequential state switching ($S^3$, based on free energy concept of a
restricted Boltzmann machine, RBM) and artificial anomaly association ($A^3$, a
classification framework using deep neural networks, DNN). Synthetic data from
cases with failed pattern(s) and anomalous node(s) are simulated to validate
the proposed approaches. Real dataset based on Tennessee Eastman process (TEP)
is also used for comparison with other approaches. The results show that: (1)
$S^3$ and $A^3$ approaches can obtain high accuracy in root-cause analysis
under both pattern-based and node-based fault scenarios, in addition to
successfully handling multiple nominal operating modes, (2) the proposed
tool-chain is shown to be scalable while maintaining high accuracy, and (3) the
proposed framework is robust and adaptive in different fault conditions and
performs better in comparison with the state-of-the-art methods.
Dynamic Advisor-Based Ensemble (dynABE): Case study in stock trend prediction of critical metal companies
Authors: Zhengyang Dong
Date: 2018-05-24 00:00:00
Tasks: Stock Prediction, Stock Trend Prediction, Time Series Analysis

Stock trend prediction is a challenging task due to the market's noise, and
machine learning techniques have recently been successful in coping with this
challenge. In this research, we create a novel framework for stock prediction,
Dynamic Advisor-Based Ensemble (dynABE). dynABE explores domain-specific areas
based on the companies of interest, diversifies the feature set by creating
different "advisors" that each handles a different area, follows an effective
model ensemble procedure for each advisor, and combines the advisors together
in a second-level ensemble through an online update strategy we developed.
dynABE is able to adapt to price pattern changes of the market during the
active trading period robustly, without needing to retrain the entire model. We
test dynABE on three cobalt-related companies, and it achieves the best-case
misclassification error of 31.12% and an annualized absolute return of 359.55%
with zero maximum drawdown. dynABE also consistently outperforms the baseline
models of support vector machine, neural network, and random forest in all case
studies.
Image-Dependent Local Entropy Models for Learned Image Compression
Authors: David Minnen, George Toderici, Saurabh Singh, Sung Jin Hwang, Michele Covell
Date: 2018-05-31 00:00:00
Tasks: Decoder, Image Compression

The leading approach for image compression with artificial neural networks
(ANNs) is to learn a nonlinear transform and a fixed entropy model that are
optimized for rate-distortion performance. We show that this approach can be
significantly improved by incorporating spatially local, image-dependent
entropy models. The key insight is that existing ANN-based methods learn an
entropy model that is shared between the encoder and decoder, but they do not
transmit any side information that would allow the model to adapt to the
structure of a specific image. We present a method for augmenting ANN-based
image coders with image-dependent side information that leads to a 17.8% rate
reduction over a state-of-the-art ANN-based baseline model on a standard
evaluation set, and 70-98% reductions on images with low visual complexity that
are poorly captured by a fixed, global entropy model.
Adversarial Constraint Learning for Structured Prediction
Authors: Hongyu Ren, Russell Stewart, Jiaming Song, Volodymyr Kuleshov, Stefano Ermon
Date: 2018-05-27 00:00:00
Tasks: Pose Estimation, Prediction, Structured Prediction, Time Series, Time Series Analysis, Time Series Prediction, valid

Constraint-based learning reduces the burden of collecting labels by having
users specify general properties of structured outputs, such as constraints
imposed by physical laws. We propose a novel framework for simultaneously
learning these constraints and using them for supervision, bypassing the
difficulty of using domain expertise to manually specify constraints. Learning
requires a black-box simulator of structured outputs, which generates valid
labels, but need not model their corresponding inputs or the input-label
relationship. At training time, we constrain the model to produce outputs that
cannot be distinguished from simulated labels by adversarial training.
Providing our framework with a small number of labeled inputs gives rise to a
new semi-supervised structured prediction model; we evaluate this model on
multiple tasks --- tracking, pose estimation and time series prediction --- and
find that it achieves high accuracy with only a small number of labeled inputs.
In some cases, no labels are required at all.
Empirical Evaluation of Character-Based Model on Neural Named-Entity Recognition in Indonesian Conversational Texts
Authors: Kemal Kurniawan, Samuel Louvan
Date: 2018-05-31 00:00:00
Proceeding: WS 2018 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Despite the long history of named-entity recognition (NER) task in the
natural language processing community, previous work rarely studied the task on
conversational texts. Such texts are challenging because they contain a lot of
word variations which increase the number of out-of-vocabulary (OOV) words. The
high number of OOV words poses a difficulty for word-based neural models.
Meanwhile, there is plenty of evidence to the effectiveness of character-based
neural models in mitigating this OOV problem. We report an empirical evaluation
of neural sequence labeling models with character embedding to tackle NER task
in Indonesian conversational texts. Our experiments show that (1) character
models outperform word embedding-only models by up to 4 $F_1$ points, (2)
character models perform better in OOV cases with an improvement of as high as
15 $F_1$ points, and (3) character models are robust against a very high OOV
rate.
Data-driven root-cause analysis for distributed system anomalies
Authors: Chao Liu, Kin Gwn Lore, Soumik Sarkar
Date: 2016-05-20 00:00:00
Tasks: Multi-class Classification

Modern distributed cyber-physical systems encounter a large variety of
anomalies and in many cases, they are vulnerable to catastrophic fault
propagation scenarios due to strong connectivity among the sub-systems. In this
regard, root-cause analysis becomes highly intractable due to complex fault
propagation mechanisms in combination with diverse operating modes. This paper
presents a new data-driven framework for root-cause analysis for addressing
such issues. The framework is based on a spatiotemporal feature extraction
scheme for distributed cyber-physical systems built on the concept of symbolic
dynamics for discovering and representing causal interactions among subsystems
of a complex system. We present two approaches for root-cause analysis, namely
the sequential state switching ($S^3$, based on free energy concept of a
Restricted Boltzmann Machine, RBM) and artificial anomaly association ($A^3$, a
multi-class classification framework using deep neural networks, DNN).
Synthetic data from cases with failed pattern(s) and anomalous node are
simulated to validate the proposed approaches, then compared with the
performance of vector autoregressive (VAR) model-based root-cause analysis.
Real dataset based on Tennessee Eastman process (TEP) is also used for
validation. The results show that: (1) $S^3$ and $A^3$ approaches can obtain
high accuracy in root-cause analysis and successfully handle multiple nominal
operation modes, and (2) the proposed tool-chain is shown to be scalable while
maintaining high accuracy.
Efficient Traffic-Sign Recognition with Scale-aware CNN
Authors: Yuchen Yang, Shuo Liu, Wei Ma, Qiuyuan Wang, Zheng Liu
Date: 2018-05-31 00:00:00
Tasks: General Classification, Traffic Sign Recognition

The paper presents a Traffic Sign Recognition (TSR) system, which can fast
and accurately recognize traffic signs of different sizes in images. The system
consists of two well-designed Convolutional Neural Networks (CNNs), one for
region proposals of traffic signs and one for classification of each region. In
the proposal CNN, a Fully Convolutional Network (FCN) with a dual multi-scale
architecture is proposed to achieve scale invariant detection. In training the
proposal network, a modified "Online Hard Example Mining" (OHEM) scheme is
adopted to suppress false positives. The classification network fuses
multi-scale features as representation and adopts an "Inception" module for
efficiency. We evaluate the proposed TSR system and its components with
extensive experiments. Our method obtains $99.88\%$ precision and $96.61\%$
recall on the Swedish Traffic Signs Dataset (STSD), higher than
state-of-the-art methods. Besides, our system is faster and more lightweight
than state-of-the-art deep learning networks for traffic sign recognition.
A mixture model for aggregation of multiple pre-trained weak classifiers
Authors: Rudrasis Chakraborty, Chun-Hao Yang, Baba C. Vemuri
Date: 2018-05-31 00:00:00
Tasks: General Classification

Deep networks have gained immense popularity in Computer Vision and other
fields in the past few years due to their remarkable performance on
recognition/classification tasks surpassing the state-of-the art. One of the
keys to their success lies in the richness of the automatically learned
features. In order to get very good accuracy, one popular option is to increase
the depth of the network. Training such a deep network is however infeasible or
impractical with moderate computational resources and budget. The other
alternative to increase the performance is to learn multiple weak classifiers
and boost their performance using a boosting algorithm or a variant thereof.
But, one of the problems with boosting algorithms is that they require a
re-training of the networks based on the misclassified samples. Motivated by
these problems, in this work we propose an aggregation technique which combines
the output of multiple weak classifiers. We formulate the aggregation problem
using a mixture model fitted to the trained classifier outputs. Our model does
not require any re-training of the `weak' networks and is computationally very
fast (takes $<30$ seconds to run in our experiments). Thus, using a less
expensive training stage and without doing any re-training of networks, we
experimentally demonstrate that it is possible to boost the performance by
$12\%$. Furthermore, we present experiments using hand-crafted features and
improved the classification performance using the proposed aggregation
technique. One of the major advantages of our framework is that our framework
allows one to combine features that are very likely to be of distinct
dimensions since they are extracted using different networks/algorithms. Our
experimental results demonstrate a significant performance gain from the use of
our aggregation technique at a very small computational cost.