pwc-archive/papers-with-abstracts huggingface dataset:
PCA-Based Missing Information Imputation for Real-Time Crash Likelihood Prediction Under Imbalanced Data
Authors: Jintao Ke, Shuaichao Zhang, Hai Yang, Xiqun Chen
Date: 2018-02-11 00:00:00
Tasks: Clustering, Imputation, Missing Values

The real-time crash likelihood prediction has been an important research
topic. Various classifiers, such as support vector machine (SVM) and tree-based
boosting algorithms, have been proposed in traffic safety studies. However, few
research focuses on the missing data imputation in real-time crash likelihood
prediction, although missing values are commonly observed due to breakdown of
sensors or external interference. Besides, classifying imbalanced data is also
a difficult problem in real-time crash likelihood prediction, since it is hard
to distinguish crash-prone cases from non-crash cases which compose the
majority of the observed samples. In this paper, principal component analysis
(PCA) based approaches, including LS-PCA, PPCA, and VBPCA, are employed for
imputing missing values, while two kinds of solutions are developed to solve
the problem in imbalanced data. The results show that PPCA and VBPCA not only
outperform LS-PCA and other imputation methods (including mean imputation and
k-means clustering imputation), in terms of the root mean square error (RMSE),
but also help the classifiers achieve better predictive performance. The two
solutions, i.e., cost-sensitive learning and synthetic minority oversampling
technique (SMOTE), help improve the sensitivity by adjusting the classifiers to
pay more attention to the minority class.
Tree-to-tree Neural Networks for Program Translation
Authors: Xinyun Chen, Chang Liu, Dawn Song
Date: 2018-02-11 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder, Translation

Program translation is an important tool to migrate legacy code in one
language into an ecosystem built in a different language. In this work, we are
the first to employ deep neural networks toward tackling this problem. We
observe that program translation is a modular procedure, in which a sub-tree of
the source tree is translated into the corresponding target sub-tree at each
step. To capture this intuition, we design a tree-to-tree neural network to
translate a source tree into a target one. Meanwhile, we develop an attention
mechanism for the tree-to-tree model, so that when the decoder expands one
non-terminal in the target tree, the attention mechanism locates the
corresponding sub-tree in the source tree to guide the expansion of the
decoder. We evaluate the program translation capability of our tree-to-tree
model against several state-of-the-art approaches. Compared against other
neural translation models, we observe that our approach is consistently better
than the baselines with a margin of up to 15 points. Further, our approach can
improve the previous state-of-the-art program translation approaches by a
margin of 20 points on the translation of real-world projects.
A Machine Learning Approach to Adaptive Covariance Localization
Authors: Azam Moosavi, Ahmed Attia, Adrian Sandu
Date: 2018-01-02 00:00:00
Tasks: BIG-bench Machine Learning, Weather Forecasting

Data assimilation plays a key role in large-scale atmospheric weather
forecasting, where the state of the physical system is estimated from model
outputs and observations, and is then used as initial condition to produce
accurate future forecasts. The Ensemble Kalman Filter (EnKF) provides a
practical implementation of the statistical solution of the data assimilation
problem and has gained wide popularity as. This success can be attributed to
its simple formulation and ease of implementation. EnKF is a Monte-Carlo
algorithm that solves the data assimilation problem by sampling the probability
distributions involved in Bayes theorem. Because of this, all flavors of EnKF
are fundamentally prone to sampling errors when the ensemble size is small. In
typical weather forecasting applications, the model state space has dimension
$10^{9}-10^{12}$, while the ensemble size typically ranges between $30-100$
members. Sampling errors manifest themselves as long-range spurious
correlations and have been shown to cause filter divergence. To alleviate this
effect covariance localization dampens spurious correlations between state
variables located at a large distance in the physical space, via an empirical
distance-dependent function. The quality of the resulting analysis and forecast
is greatly influenced by the choice of the localization function parameters,
e.g., the radius of influence. The localization radius is generally tuned
empirically to yield desirable results.This work, proposes two adaptive
algorithms for covariance localization in the EnKF framework, both based on a
machine learning approach. The first algorithm adapts the localization radius
in time, while the second algorithm tunes the localization radius in both time
and space. Numerical experiments carried out with the Lorenz-96 model, and a
quasi-geostrophic model, reveal the potential of the proposed machine learning
approaches.
Dual Control Memory Augmented Neural Networks for Treatment Recommendations
Authors: Hung Le, Truyen Tran, Svetha Venkatesh
Date: 2018-02-11 00:00:00

Machine-assisted treatment recommendations hold a promise to reduce physician
time and decision errors. We formulate the task as a sequence-to-sequence
prediction model that takes the entire time-ordered medical history as input,
and predicts a sequence of future clinical procedures and medications. It is
built on the premise that an effective treatment plan may have long-term
dependencies from previous medical history. We approach the problem by using a
memory-augmented neural network, in particular, by leveraging the recent
differentiable neural computer that consists of a neural controller and an
external memory module. But differing from the original model, we use dual
controllers, one for encoding the history followed by another for decoding the
treatment sequences. In the encoding phase, the memory is updated as new input
is read; at the end of this phase, the memory holds not only the medical
history but also the information about the current illness. During the decoding
phase, the memory is write-protected. The decoding controller generates a
treatment sequence, one treatment option at a time. The resulting dual
controller write-protected memory-augmented neural network is demonstrated on
the MIMIC-III dataset on two tasks: procedure prediction and medication
prescription. The results show improved performance over both traditional
bag-of-words and sequence-to-sequence methods.
On the Rates of Convergence from Surrogate Risk Minimizers to the Bayes Optimal Classifier
Authors: Jingwei Zhang, Tongliang Liu, DaCheng Tao
Date: 2018-02-11 00:00:00

We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer to the Bayes optimal classifier, enabling fair comparisons of the excess risks of different surrogate risk minimizers. The main result of the paper has practical implications including (1) showing that hinge loss is superior to logistic and exponential loss in the sense that its empirical minimizer converges faster to the Bayes optimal classifier and (2) guiding to modify surrogate loss functions to accelerate the convergence to the Bayes optimal classifier.
Dual Memory Neural Computer for Asynchronous Two-view Sequential Learning
Authors: Hung Le, Truyen Tran, Svetha Venkatesh
Date: 2018-02-02 00:00:00
Tasks: Decoder, MULTI-VIEW LEARNING, Vocal Bursts Valence Prediction

One of the core tasks in multi-view learning is to capture relations among
views. For sequential data, the relations not only span across views, but also
extend throughout the view length to form long-term intra-view and inter-view
interactions. In this paper, we present a new memory augmented neural network
model that aims to model these complex interactions between two asynchronous
sequential views. Our model uses two encoders for reading from and writing to
two external memories for encoding input views. The intra-view interactions and
the long-term dependencies are captured by the use of memories during this
encoding process. There are two modes of memory accessing in our system:
late-fusion and early-fusion, corresponding to late and early inter-view
interactions. In the late-fusion mode, the two memories are separated,
containing only view-specific contents. In the early-fusion mode, the two
memories share the same addressing space, allowing cross-memory accessing. In
both cases, the knowledge from the memories will be combined by a decoder to
make predictions over the output space. The resulting dual memory neural
computer is demonstrated on a comprehensive set of experiments, including a
synthetic task of summing two sequences and the tasks of drug prescription and
disease progression in healthcare. The results demonstrate competitive
performance over both traditional algorithms and deep learning methods designed
for multi-view problems.
CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
Authors: Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiao-Feng Wang, Carl A. Gunter
Date: 2018-01-24 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

The popularity of ASR (automatic speech recognition) systems, like Google
Voice, Cortana, brings in security concerns, as demonstrated by recent attacks.
The impacts of such threats, however, are less clear, since they are either
less stealthy (producing noise-like voice commands) or requiring the physical
presence of an attack device (using ultrasound). In this paper, we demonstrate
that not only are more practical and surreptitious attacks feasible but they
can even be automatically constructed. Specifically, we find that the voice
commands can be stealthily embedded into songs, which, when played, can
effectively control the target system through ASR without being noticed. For
this purpose, we developed novel techniques that address a key technical
challenge: integrating the commands into a song in a way that can be
effectively recognized by ASR through the air, in the presence of background
noise, while not being detected by a human listener. Our research shows that
this can be done automatically against real world ASR applications. We also
demonstrate that such CommanderSongs can be spread through Internet (e.g.,
YouTube) and radio, potentially affecting millions of ASR users. We further
present a new mitigation technique that controls this threat.
Understanding Convolutional Networks with APPLE : Automatic Patch Pattern Labeling for Explanation
Authors: Sandeep Konam, Ian Quah, Stephanie Rosenthal, Manuela Veloso
Date: 2018-02-11 00:00:00
Tasks: Classification, General Classification

With the success of deep learning, recent efforts have been focused on
analyzing how learned networks make their classifications. We are interested in
analyzing the network output based on the network structure and information
flow through the network layers. We contribute an algorithm for 1) analyzing a
deep network to find neurons that are 'important' in terms of the network
classification outcome, and 2)automatically labeling the patches of the input
image that activate these important neurons. We propose several measures of
importance for neurons and demonstrate that our technique can be used to gain
insight into, and explain how a network decomposes an image to make its final
classification.
Asynchronous Decentralized Parallel Stochastic Gradient Descent
Authors: Xiangru Lian, Wei zhang, Ce Zhang, Ji Liu
Date: 2017-10-18 00:00:00
Proceeding: ICML 2018 7
Tasks: GPU

Most commonly used distributed machine learning systems are either
synchronous or centralized asynchronous. Synchronous algorithms like
AllReduce-SGD perform poorly in a heterogeneous environment, while asynchronous
algorithms using a parameter server suffer from 1) communication bottleneck at
parameter servers when workers are many, and 2) significantly worse convergence
when the traffic to parameter server is congested. Can we design an algorithm
that is robust in a heterogeneous environment, while being communication
efficient and maintaining the best-possible convergence rate? In this paper, we
propose an asynchronous decentralized stochastic gradient decent algorithm
(AD-PSGD) satisfying all above expectations. Our theoretical analysis shows
AD-PSGD converges at the optimal $O(1/\sqrt{K})$ rate as SGD and has linear
speedup w.r.t. number of workers. Empirically, AD-PSGD outperforms the best of
decentralized parallel SGD (D-PSGD), asynchronous parallel SGD (A-PSGD), and
standard data parallel SGD (AllReduce-SGD), often by orders of magnitude in a
heterogeneous environment. When training ResNet-50 on ImageNet with up to 128
GPUs, AD-PSGD converges (w.r.t epochs) similarly to the AllReduce-SGD, but each
epoch can be up to 4-8X faster than its synchronous counterparts in a
network-sharing HPC environment. To the best of our knowledge, AD-PSGD is the
first asynchronous algorithm that achieves a similar epoch-wise convergence
rate as AllReduce-SGD, at an over 100-GPU scale.
Deep learning with t-exponential Bayesian kitchen sinks
Authors: Harris Partaourides, Sotirios Chatzis
Date: 2018-02-10 00:00:00
Tasks: Deep Learning

Bayesian learning has been recently considered as an effective means of
accounting for uncertainty in trained deep network parameters. This is of
crucial importance when dealing with small or sparse training datasets. On the
other hand, shallow models that compute weighted sums of their inputs, after
passing them through a bank of arbitrary randomized nonlinearities, have been
recently shown to enjoy good test error bounds that depend on the number of
nonlinearities. Inspired from these advances, in this paper we examine novel
deep network architectures, where each layer comprises a bank of arbitrary
nonlinearities, linearly combined using multiple alternative sets of weights.
We effect model training by means of approximate inference based on a
t-divergence measure; this generalizes the Kullback-Leibler divergence in the
context of the t-exponential family of distributions. We adopt the
t-exponential family since it can more flexibly accommodate real-world data,
that entail outliers and distributions with fat tails, compared to conventional
Gaussian model assumptions. We extensively evaluate our approach using several
challenging benchmarks, and provide comparative results to related
state-of-the-art techniques.
Gradually Updated Neural Networks for Large-Scale Image Recognition
Authors: Siyuan Qiao, Zhishuai Zhang, Wei Shen, Bo wang, Alan Yuille
Date: 2017-11-25 00:00:00
Proceeding: ICML 2018 7

Depth is one of the keys that make neural networks succeed in the task of
large-scale image recognition. The state-of-the-art network architectures
usually increase the depths by cascading convolutional layers or building
blocks. In this paper, we present an alternative method to increase the depth.
Our method is by introducing computation orderings to the channels within
convolutional layers or blocks, based on which we gradually compute the outputs
in a channel-wise manner. The added orderings not only increase the depths and
the learning capacities of the networks without any additional computation
costs, but also eliminate the overlap singularities so that the networks are
able to converge faster and perform better. Experiments show that the networks
based on our method achieve the state-of-the-art performances on CIFAR and
ImageNet datasets.
On the Universal Approximability and Complexity Bounds of Quantized ReLU Neural Networks
Authors: Yukun Ding, Jinglan Liu, JinJun Xiong, Yiyu Shi
Date: 2018-02-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Quantization

Compression is a key step to deploy large neural networks on
resource-constrained platforms. As a popular compression technique,
quantization constrains the number of distinct weight values and thus reducing
the number of bits required to represent and store each weight. In this paper,
we study the representation power of quantized neural networks. First, we prove
the universal approximability of quantized ReLU networks on a wide class of
functions. Then we provide upper bounds on the number of weights and the memory
size for a given approximation error bound and the bit-width of weights for
function-independent and function-dependent structures. Our results reveal
that, to attain an approximation error bound of $\epsilon$, the number of
weights needed by a quantized network is no more than
$\mathcal{O}\left(\log^5(1/\epsilon)\right)$ times that of an unquantized
network. This overhead is of much lower order than the lower bound of the
number of weights needed for the error bound, supporting the empirical success
of various quantization techniques. To the best of our knowledge, this is the
first in-depth study on the complexity bounds of quantized neural networks.
Graph Planning with Expected Finite Horizon
Authors: Krishnendu Chatterjee, Laurent Doyen
Date: 2018-02-10 00:00:00
Tasks: Traveling Salesman Problem

Graph planning gives rise to fundamental algorithmic questions such as
shortest path, traveling salesman problem, etc. A classical problem in discrete
planning is to consider a weighted graph and construct a path that maximizes
the sum of weights for a given time horizon $T$. However, in many scenarios,
the time horizon is not fixed, but the stopping time is chosen according to
some distribution such that the expected stopping time is $T$. If the stopping
time distribution is not known, then to ensure robustness, the distribution is
chosen by an adversary, to represent the worst-case scenario.
  A stationary plan for every vertex always chooses the same outgoing edge. For
fixed horizon or fixed stopping-time distribution, stationary plans are not
sufficient for optimality. Quite surprisingly we show that when an adversary
chooses the stopping-time distribution with expected stopping time $T$, then
stationary plans are sufficient. While computing optimal stationary plans for
fixed horizon is NP-complete, we show that computing optimal stationary plans
under adversarial stopping-time distribution can be achieved in polynomial
time. Consequently, our polynomial-time algorithm for adversarial stopping time
also computes an optimal plan among all possible plans.
Handwritten Bangla Character Recognition Using The State-of-Art Deep Convolutional Neural Networks
Authors: Md Zahangir Alom, Peheding Sidike, Mahmudul Hasan, Tark M. Taha, Vijayan K. Asari
Date: 2017-12-28 00:00:00
Tasks: Object Recognition, Translation

In spite of advances in object recognition technology, Handwritten Bangla
Character Recognition (HBCR) remains largely unsolved due to the presence of
many ambiguous handwritten characters and excessively cursive Bangla
handwritings. Even the best existing recognizers do not lead to satisfactory
performance for practical applications related to Bangla character recognition
and have much lower performance than those developed for English alpha-numeric
characters. To improve the performance of HBCR, we herein present the
application of the state-of-the-art Deep Convolutional Neural Networks (DCNN)
including VGG Network, All Convolution Network (All-Conv Net), Network in
Network (NiN), Residual Network, FractalNet, and DenseNet for HBCR. The deep
learning approaches have the advantage of extracting and using feature
information, improving the recognition of 2D shapes with a high degree of
invariance to translation, scaling and other distortions. We systematically
evaluated the performance of DCNN models on publicly available Bangla
handwritten character dataset called CMATERdb and achieved the superior
recognition accuracy when using DCNN models. This improvement would help in
building an automatic HBCR system for practical applications.
Online Classification with Complex Metrics
Authors: Bowei Yan, Oluwasanmi Koyejo, Kai Zhong, Pradeep Ravikumar
Date: 2016-10-23 00:00:00
Tasks: Binary Classification, Classification, General Classification

We present a framework and analysis of consistent binary classification for
complex and non-decomposable performance metrics such as the F-measure and the
Jaccard measure. The proposed framework is general, as it applies to both batch
and online learning, and to both linear and non-linear models. Our work follows
recent results showing that the Bayes optimal classifier for many complex
metrics is given by a thresholding of the conditional probability of the
positive class. This manuscript extends this thresholding characterization --
showing that the utility is strictly locally quasi-concave with respect to the
threshold for a wide range of models and performance metrics. This, in turn,
motivates simple normalized gradient ascent updates for threshold estimation.
We present a finite-sample regret analysis for the resulting procedure. In
particular, the risk for the batch case converges to the Bayes risk at the same
rate as that of the underlying conditional probability estimation, and the risk
of proposed online algorithm converges at a rate that depends on the
conditional probability estimation risk. For instance, in the special case
where the conditional probability model is logistic regression, our procedure
achieves $O(\frac{1}{\sqrt{n}})$ sample complexity, both for batch and online
training. Empirical evaluation shows that the proposed algorithms out-perform
alternatives in practice, with comparable or better prediction performance and
reduced run time for various metrics and datasets.
Optimize transfer learning for lung diseases in bronchoscopy using a new concept: sequential fine-tuning
Authors: Tao Tan, Zhang Li, Haixia Liu, Ping Liu, Wenfang Tang, Hui Li, Yue Sun, Yusheng Yan, Keyu Li, Tao Xu, Shanshan Wan, Ke Lou, Jun Xu, Huiming Ying, Quchang Ouyang, Yuling Tang, Zheyu Hu, Qiang Li
Date: 2018-02-10 00:00:00
Tasks: Transfer Learning

Bronchoscopy inspection as a follow-up procedure from the radiological
imaging plays a key role in lung disease diagnosis and determining treatment
plans for the patients. Doctors needs to make a decision whether to biopsy the
patients timely when performing bronchoscopy. However, the doctors also needs
to be very selective with biopsies as biopsies may cause uncontrollable
bleeding of the lung tissue which is life-threaten. To help doctors to be more
selective on biopsies and provide a second opinion on diagnosis, in this work,
we propose a computer-aided diagnosis (CAD) system for lung diseases including
cancers and tuberculosis (TB). The system is developed based on transfer
learning. We propose a novel transfer learning method: sentential fine-tuning .
Compared to traditional fine-tuning methods, our methods achieves the best
performance. We obtained a overall accuracy of 77.0% a dataset of 81 normal
cases, 76 tuberculosis cases and 277 lung cancer cases while the other
traditional transfer learning methods achieve an accuracy of 73% and 68%. . The
detection accuracy of our method for cancers, TB and normal cases are 87%, 54%
and 91% respectively. This indicates that the CAD system has potential to
improve lung disease diagnosis accuracy in bronchoscopy and it also might be
used to be more selective with biopsies.