pwc-archive/papers-with-abstracts huggingface dataset:
Deep Multiscale Model Learning
Authors: Yating Wang, Siu Wun Cheung, Eric T. Chung, Yalchin Efendiev, Min Wang
Date: 2018-06-13 00:00:00
Tasks: Deep Learning, model

The objective of this paper is to design novel multi-layer neural network
architectures for multiscale simulations of flows taking into account the
observed data and physical modeling concepts. Our approaches use deep learning
concepts combined with local multiscale model reduction methodologies to
predict flow dynamics. Using reduced-order model concepts is important for
constructing robust deep learning architectures since the reduced-order models
provide fewer degrees of freedom. Flow dynamics can be thought of as
multi-layer networks. More precisely, the solution (e.g., pressures and
saturations) at the time instant $n+1$ depends on the solution at the time
instant $n$ and input parameters, such as permeability fields, forcing terms,
and initial conditions. One can regard the solution as a multi-layer network,
where each layer, in general, is a nonlinear forward map and the number of
layers relates to the internal time steps. We will rely on rigorous model
reduction concepts to define unknowns and connections for each layer. In each
layer, our reduced-order models will provide a forward map, which will be
modified ("trained") using available data. It is critical to use reduced-order
models for this purpose, which will identify the regions of influence and the
appropriate number of variables. Because of the lack of available data, the
training will be supplemented with computational data as needed and the
interpolation between data-rich and data-deficient models. We will also use
deep learning algorithms to train the elements of the reduced model discrete
system. We will present main ingredients of our approach and numerical results.
Numerical results show that using deep learning and multiscale models, we can
improve the forward models, which are conditioned to the available data.
Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network
Authors: Xue Yang, Hao Sun, Xian Sun, Menglong Yan, Zhi Guo, Kun fu
Date: 2018-06-13 00:00:00
Tasks: Position

Ship detection is of great importance and full of challenges in the field of
remote sensing. The complexity of application scenarios, the redundancy of
detection region, and the difficulty of dense ship detection are all the main
obstacles that limit the successful operation of traditional methods in ship
detection. In this paper, we propose a brand new detection model based on
multitask rotational region convolutional neural network to solve the problems
above. This model is mainly consist of five consecutive parts: Dense Feature
Pyramid Network (DFPN), adaptive region of interest (ROI) Align, rotational
bounding box regression, prow direction prediction and rotational nonmaximum
suppression (R-NMS). First of all, the low-level location information and
high-level semantic information are fully utilized through multiscale feature
networks. Then, we design Adaptive ROI Align to obtain high quality proposals
which remain complete spatial and semantic information. Unlike most previous
approaches, the prediction obtained by our method is the minimum bounding
rectangle of the object with less redundant regions. Therefore, rotational
region detection framework is more suitable to detect the dense object than
traditional detection model. Additionally, we can find the berthing and sailing
direction of ship through prediction. A detailed evaluation based on SRSS for
rotation detection shows that our detection method has a competitive
performance.
Detection of Premature Ventricular Contractions Using Densely Connected Deep Convolutional Neural Network with Spatial Pyramid Pooling Layer
Authors: Jianning Li
Date: 2018-06-12 00:00:00
Tasks: Rhythm, Specificity

Premature ventricular contraction(PVC) is a type of premature ectopic beat originating from the ventricles. Automatic method for accurate and robust detection of PVC is highly clinically desired.Currently, most of these methods are developed and tested using the same database divided into training and testing set and their generalization performance across databases has not been fully validated. In this paper, a method based on densely connected convolutional neural network and spatial pyramid pooling is proposed for PVC detection which can take arbitrarily-sized QRS complexes as input both in training and testing. With a much less complicated and more straightforward architecture,the proposed network achieves comparable results to current state-of-the-art deep learning based method with regard to accuracy,sensitivity and specificity by training and testing using the MIT-BIH arrhythmia database as benchmark.Besides the benchmark database,QRS complexes are extracted from four more open databases namely the St-Petersburg Institute of Cardiological Technics 12-lead Arrhythmia Database,The MIT-BIH Normal Sinus Rhythm Database,The MIT-BIH Long Term Database and European ST-T Database. The extracted QRS complexes are different in length and sampling rate among the five databases.Cross-database training and testing is also experimented.The performance of the network shows an improvement on the benchmark database according to the result demonstrating the advantage of using multiple databases for training over using only a single database.The network also achieves satisfactory scores on the other four databases showing good generalization capability.
Regularized Orthogonal Machine Learning for Nonlinear Semiparametric Models
Authors: Denis Nekipelov, Vira Semenova, Vasilis Syrgkanis
Date: 2018-06-13 00:00:00
Tasks: BIG-bench Machine Learning

This paper proposes a Lasso-type estimator for a high-dimensional sparse parameter identified by a single index conditional moment restriction (CMR). In addition to this parameter, the moment function can also depend on a nuisance function, such as the propensity score or the conditional choice probability, which we estimate by modern machine learning tools. We first adjust the moment function so that the gradient of the future loss function is insensitive (formally, Neyman-orthogonal) with respect to the first-stage regularization bias, preserving the single index property. We then take the loss function to be an indefinite integral of the adjusted moment function with respect to the single index. The proposed Lasso estimator converges at the oracle rate, where the oracle knows the nuisance function and solves only the parametric problem. We demonstrate our method by estimating the short-term heterogeneous impact of Connecticut's Jobs First welfare reform experiment on women's welfare participation decision.
Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data?
Authors: Yi Zhang, Xu sun, Shuming Ma, Yang Yang, Xuancheng Ren
Date: 2017-11-22 00:00:00
Proceeding: COLING 2018 8
Tasks: Chunking, NER, TAG

Existing neural models usually predict the tag of the current token
independent of the neighboring tags. The popular LSTM-CRF model considers the
tag dependencies between every two consecutive tags. However, it is hard for
existing neural models to take longer distance dependencies of tags into
consideration. The scalability is mainly limited by the complex model
structures and the cost of dynamic programming during training. In our work, we
first design a new model called "high order LSTM" to predict multiple tags for
the current token which contains not only the current tag but also the previous
several tags. We call the number of tags in one prediction as "order". Then we
propose a new method called Multi-Order BiLSTM (MO-BiLSTM) which combines low
order and high order LSTMs together. MO-BiLSTM keeps the scalability to high
order models with a pruning technique. We evaluate MO-BiLSTM on all-phrase
chunking and NER datasets. Experiment results show that MO-BiLSTM achieves the
state-of-the-art result in chunking and highly competitive results in two NER
datasets.
Integral Privacy for Sampling
Authors: Hisham Husain, Zac Cranko, Richard Nock
Date: 2018-06-13 00:00:00
Tasks: Density Estimation, Fairness

Differential privacy is a leading protection setting, focused by design on individual privacy. Many applications, in medical / pharmaceutical domains or social networks, rather posit privacy at a group level, a setting we call integral privacy. We aim for the strongest form of privacy: the group size is in particular not known in advance. We study a problem with related applications in domains cited above that have recently met with substantial recent press: sampling. Keeping correct utility levels in such a strong model of statistical indistinguishability looks difficult to be achieved with the usual differential privacy toolbox because it would typically scale in the worst case the sensitivity by the sample size and so the noise variance by up to its square. We introduce a trick specific to sampling that bypasses the sensitivity analysis. Privacy enforces an information theoretic barrier on approximation, and we show how to reach this barrier with guarantees on the approximation of the target non private density. We do so using a recent approach to non private density estimation relying on the original boosting theory, learning the sufficient statistics of an exponential family with classifiers. Approximation guarantees cover the mode capture problem. In the context of learning, the sampling problem is particularly important: because integral privacy enjoys the same closure under post-processing as differential privacy does, any algorithm using integrally privacy sampled data would result in an output equally integrally private. We also show that this brings fairness guarantees on post-processing that would eventually elude classical differential privacy: any decision process has bounded data-dependent bias when the data is integrally privately sampled. Experimental results against private kernel density estimation and private GANs displays the quality of our results.
Disintegration and Bayesian Inversion via String Diagrams
Authors: Kenta Cho, Bart Jacobs
Date: 2017-08-29 00:00:00

The notions of disintegration and Bayesian inversion are fundamental in
conditional probability theory. They produce channels, as conditional
probabilities, from a joint state, or from an already given channel (in
opposite direction). These notions exist in the literature, in concrete
situations, but are presented here in abstract graphical formulations. The
resulting abstract descriptions are used for proving basic results in
conditional probability theory. The existence of disintegration and Bayesian
inversion is discussed for discrete probability, and also for measure-theoretic
probability --- via standard Borel spaces and via likelihoods. Finally, the
usefulness of disintegration and Bayesian inversion is illustrated in several
examples.
A Unified Framework for Generalizable Style Transfer: Style and Content Separation
Authors: Yexun Zhang, Ya zhang, Wenbin Cai
Date: 2018-06-13 00:00:00
Tasks: Decoder, Multi-Task Learning, Style Transfer

Image style transfer has drawn broad attention in recent years. However, most
existing methods aim to explicitly model the transformation between different
styles, and the learned model is thus not generalizable to new styles. We here
propose a unified style transfer framework for both character typeface transfer
and neural style transfer tasks leveraging style and content separation. A key
merit of such framework is its generalizability to new styles and contents. The
overall framework consists of style encoder, content encoder, mixer and
decoder. The style encoder and content encoder are used to extract the style
and content representations from the corresponding reference images. The mixer
integrates the above two representations and feeds it into the decoder to
generate images with the target style and content. During training, the encoder
networks learn to extract styles and contents from limited size of
style/content reference images. This learning framework allows simultaneous
style transfer among multiple styles and can be deemed as a special
`multi-task' learning scenario. The encoders are expected to capture the
underlying features for different styles and contents which is generalizable to
new styles and contents. Under this framework, we design two individual
networks for character typeface transfer and neural style transfer,
respectively. For character typeface transfer, to separate the style features
and content features, we leverage the conditional dependence of styles and
contents given an image. For neural style transfer, we leverage the statistical
information of feature maps in certain layers to represent style. Extensive
experimental results have demonstrated the effectiveness and robustness of the
proposed methods.
Deep Learning for Forecasting Stock Returns in the Cross-Section
Authors: Masaya Abe, Hideki Nakayama
Date: 2018-01-03 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, speech-recognition, Speech Recognition

Many studies have been undertaken by using machine learning techniques,
including neural networks, to predict stock returns. Recently, a method known
as deep learning, which achieves high performance mainly in image recognition
and speech recognition, has attracted attention in the machine learning field.
This paper implements deep learning to predict one-month-ahead stock returns in
the cross-section in the Japanese stock market and investigates the performance
of the method. Our results show that deep neural networks generally outperform
shallow neural networks, and the best networks also outperform representative
machine learning models. These results indicate that deep learning shows
promise as a skillful machine learning method to predict stock returns in the
cross-section.
Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection
Authors: Guansong Pang, Longbing Cao, Ling Chen, Huan Liu
Date: 2018-06-13 00:00:00
Tasks: Anomaly Detection, Disease Prediction, Network Intrusion Detection, Outlier Detection, Representation Learning

Learning expressive low-dimensional representations of ultrahigh-dimensional
data, e.g., data with thousands/millions of features, has been a major way to
enable learning methods to address the curse of dimensionality. However,
existing unsupervised representation learning methods mainly focus on
preserving the data regularity information and learning the representations
independently of subsequent outlier detection methods, which can result in
suboptimal and unstable performance of detecting irregularities (i.e.,
outliers).
  This paper introduces a ranking model-based framework, called RAMODO, to
address this issue. RAMODO unifies representation learning and outlier
detection to learn low-dimensional representations that are tailored for a
state-of-the-art outlier detection approach - the random distance-based
approach. This customized learning yields more optimal and stable
representations for the targeted outlier detectors. Additionally, RAMODO can
leverage little labeled data as prior knowledge to learn more expressive and
application-relevant representations. We instantiate RAMODO to an efficient
method called REPEN to demonstrate the performance of RAMODO.
  Extensive empirical results on eight real-world ultrahigh dimensional data
sets show that REPEN (i) enables a random distance-based detector to obtain
significantly better AUC performance and two orders of magnitude speedup; (ii)
performs substantially better and more stably than four state-of-the-art
representation learning methods; and (iii) leverages less than 1% labeled data
to achieve up to 32% AUC improvement.
BA-Net: Dense Bundle Adjustment Network
Authors: Chengzhou Tang, Ping Tan
Date: 2018-06-13 00:00:00
Tasks: Depth And Camera Motion

This paper introduces a network architecture to solve the structure-from-motion (SfM) problem via feature-metric bundle adjustment (BA), which explicitly enforces multi-view geometry constraints in the form of feature-metric error. The whole pipeline is differentiable so that the network can learn suitable features that make the BA problem more tractable. Furthermore, this work introduces a novel depth parameterization to recover dense per-pixel depth. The network first generates several basis depth maps according to the input image and optimizes the final depth as a linear combination of these basis depth maps via feature-metric BA. The basis depth maps generator is also learned via end-to-end training. The whole system nicely combines domain knowledge (i.e. hard-coded multi-view geometry constraints) and deep learning (i.e. feature learning and basis depth maps learning) to address the challenging dense SfM problem. Experiments on large scale real data prove the success of the proposed method.
Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning
Authors: Kunkun Pang, Mingzhi Dong, Yang Wu, Timothy Hospedales
Date: 2018-06-12 00:00:00
Proceeding: ICLR 2018 1
Tasks: Active Learning, Deep Reinforcement Learning, Meta-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Active learning (AL) aims to enable training high performance classifiers
with low annotation cost by predicting which subset of unlabelled instances
would be most beneficial to label. The importance of AL has motivated extensive
research, proposing a wide variety of manually designed AL algorithms with
diverse theoretical and intuitive motivations. In contrast to this body of
research, we propose to treat active learning algorithm design as a
meta-learning problem and learn the best criterion from data. We model an
active learning algorithm as a deep neural network that inputs the base learner
state and the unlabelled point set and predicts the best point to annotate
next. Training this active query policy network with reinforcement learning,
produces the best non-myopic policy for a given dataset. The key challenge in
achieving a general solution to AL then becomes that of learner generalisation,
particularly across heterogeneous datasets. We propose a multi-task
dataset-embedding approach that allows dataset-agnostic active learners to be
trained. Our evaluation shows that AL algorithms trained in this way can
directly generalise across diverse problems.
Drive2Vec: Multiscale State-Space Embedding of Vehicular Sensor Data
Authors: David Hallac, Suvrat Bhooshan, Michael Chen, Kacem Abida, Rok Sosic, Jure Leskovec
Date: 2018-06-12 00:00:00

With automobiles becoming increasingly reliant on sensors to perform various
driving tasks, it is important to encode the relevant CAN bus sensor data in a
way that captures the general state of the vehicle in a compact form. In this
paper, we develop a deep learning-based method, called Drive2Vec, for embedding
such sensor data in a low-dimensional yet actionable form. Our method is based
on stacked gated recurrent units (GRUs). It accepts a short interval of
automobile sensor data as input and computes a low-dimensional representation
of that data, which can then be used to accurately solve a range of tasks. With
this representation, we (1) predict the exact values of the sensors in the
short term (up to three seconds in the future), (2) forecast the long-term
average values of these same sensors, (3) infer additional contextual
information that is not encoded in the data, including the identity of the
driver behind the wheel, and (4) build a knowledge base that can be used to
auto-label data and identify risky states. We evaluate our approach on a
dataset collected by Audi, which equipped a fleet of test vehicles with data
loggers to store all sensor readings on 2,098 hours of driving on real roads.
We show in several experiments that our method outperforms other baselines by
up to 90%, and we further demonstrate how these embeddings of sensor data can
be used to solve a variety of real-world automotive applications.
A Connectome Based Hexagonal Lattice Convolutional Network Model of the Drosophila Visual System
Authors: Fabian David Tschopp, Michael B. Reiser, Srinivas C. Turaga
Date: 2018-06-12 00:00:00
Tasks: Object Tracking

What can we learn from a connectome? We constructed a simplified model of the
first two stages of the fly visual system, the lamina and medulla. The
resulting hexagonal lattice convolutional network was trained using
backpropagation through time to perform object tracking in natural scene
videos. Networks initialized with weights from connectome reconstructions
automatically discovered well-known orientation and direction selectivity
properties in T4 neurons and their inputs, while networks initialized at random
did not. Our work is the first demonstration, that knowledge of the connectome
can enable in silico predictions of the functional properties of individual
neurons in a circuit, leading to an understanding of circuit function from
structure alone.
ASP:A Fast Adversarial Attack Example Generation Framework based on Adversarial Saliency Prediction
Authors: Fuxun Yu, Qide Dong, Xiang Chen
Date: 2018-02-15 00:00:00
Tasks: Adversarial Attack, image-classification, Image Classification, Saliency Prediction

With the excellent accuracy and feasibility, the Neural Networks have been
widely applied into the novel intelligent applications and systems. However,
with the appearance of the Adversarial Attack, the NN based system performance
becomes extremely vulnerable:the image classification results can be
arbitrarily misled by the adversarial examples, which are crafted images with
human unperceivable pixel-level perturbation. As this raised a significant
system security issue, we implemented a series of investigations on the
adversarial attack in this work: We first identify an image's pixel
vulnerability to the adversarial attack based on the adversarial saliency
analysis. By comparing the analyzed saliency map and the adversarial
perturbation distribution, we proposed a new evaluation scheme to
comprehensively assess the adversarial attack precision and efficiency. Then,
with a novel adversarial saliency prediction method, a fast adversarial example
generation framework, namely "ASP", is proposed with significant attack
efficiency improvement and dramatic computation cost reduction. Compared to the
previous methods, experiments show that ASP has at most 12 times speed-up for
adversarial example generation, 2 times lower perturbation rate, and high
attack success rate of 87% on both MNIST and Cifar10. ASP can be also well
utilized to support the data-hungry NN adversarial training. By reducing the
attack success rate as much as 90%, ASP can quickly and effectively enhance the
defense capability of NN based system to the adversarial attacks.
Global-Locally Self-Attentive Dialogue State Tracker
Authors: Victor Zhong, Caiming Xiong, Richard Socher
Date: 2018-05-19 00:00:00
Tasks: Dialogue State Tracking, Multi-domain Dialogue State Tracking, Task-Oriented Dialogue Systems

Dialogue state tracking, which estimates user goals and requests given the
dialogue context, is an essential part of task-oriented dialogue systems. In
this paper, we propose the Global-Locally Self-Attentive Dialogue State Tracker
(GLAD), which learns representations of the user utterance and previous system
actions with global-local modules. Our model uses global modules to share
parameters between estimators for different types (called slots) of dialogue
states, and uses local modules to learn slot-specific features. We show that
this significantly improves tracking of rare states and achieves
state-of-the-art performance on the WoZ and DSTC2 state tracking tasks. GLAD
obtains 88.1% joint goal accuracy and 97.1% request accuracy on WoZ,
outperforming prior work by 3.7% and 5.5%. On DSTC2, our model obtains 74.5%
joint goal accuracy and 97.5% request accuracy, outperforming prior work by
1.1% and 1.0%.