pwc-archive/papers-with-abstracts huggingface dataset:
Geometry and Symmetry in Short-and-Sparse Deconvolution
Authors: Han-Wen Kuo, Yenson Lau, Yuqian Zhang, John Wright
Date: 2019-01-02 00:00:00

We study the $\textit{Short-and-Sparse (SaS) deconvolution}$ problem of
recovering a short signal $\mathbf a_0$ and a sparse signal $\mathbf x_0$ from
their convolution. We propose a method based on nonconvex optimization, which
under certain conditions recovers the target short and sparse signals, up to a
signed shift symmetry which is intrinsic to this model. This symmetry plays a
central role in shaping the optimization landscape for deconvolution. We give a
$\textit{regional analysis}$, which characterizes this landscape geometrically,
on a union of subspaces. Our geometric characterization holds when the
length-$p_0$ short signal $\mathbf a_0$ has shift coherence $\mu$, and $\mathbf
x_0$ follows a random sparsity model with sparsity rate $\theta \in
\Bigl[\frac{c_1}{p_0}, \frac{c_2}{p_0\sqrt\mu +
\sqrt{p_0}}\Bigr]\cdot\frac{1}{\log^2p_0}$. Based on this geometry, we give a
provable method that successfully solves SaS deconvolution with high
probability.
An Automatic Interaction Detection Hybrid Model for Bankcard Response Classification
Authors: Yan Wang, Xuelei Sherry Ni, Brian Stone
Date: 2019-01-02 00:00:00
Tasks: Classification, General Classification, regression

In this paper, we propose a hybrid bankcard response model, which integrates
decision tree based chi-square automatic interaction detection (CHAID) into
logistic regression. In the first stage of the hybrid model, CHAID analysis is
used to detect the possibly potential variable interactions. Then in the second
stage, these potential interactions are served as the additional input
variables in logistic regression. The motivation of the proposed hybrid model
is that adding variable interactions may improve the performance of logistic
regression. To demonstrate the effectiveness of the proposed hybrid model, it
is evaluated on a real credit customer response data set. As the results
reveal, by identifying potential interactions among independent variables, the
proposed hybrid approach outperforms the logistic regression without searching
for interactions in terms of classification accuracy, the area under the
receiver operating characteristic curve (ROC), and Kolmogorov-Smirnov (KS)
statistics. Furthermore, CHAID analysis for interaction detection is much more
computationally efficient than the stepwise search mentioned above and some
identified interactions are shown to have statistically significant predictive
power on the target variable. Last but not least, the customer profile created
based on the CHAID tree provides a reasonable interpretation of the
interactions, which is the required by regulations of the credit industry.
Hence, this study provides an alternative for handling bankcard classification
tasks.
A Survey on Multi-output Learning
Authors: Donna Xu, Yaxin Shi, Ivor W. Tsang, Yew-Soon Ong, Chen Gong, Xiaobo Shen
Date: 2019-01-02 00:00:00
Tasks: Decision Making, Survey

Multi-output learning aims to simultaneously predict multiple outputs given an input. It is an important learning problem due to the pressing need for sophisticated decision making in real-world applications. Inspired by big data, the 4Vs characteristics of multi-output imposes a set of challenges to multi-output learning, in terms of the volume, velocity, variety and veracity of the outputs. Increasing number of works in the literature have been devoted to the study of multi-output learning and the development of novel approaches for addressing the challenges encountered. However, it lacks a comprehensive overview on different types of challenges of multi-output learning brought by the characteristics of the multiple outputs and the techniques proposed to overcome the challenges. This paper thus attempts to fill in this gap to provide a comprehensive review on this area. We first introduce different stages of the life cycle of the output labels. Then we present the paradigm on multi-output learning, including its myriads of output structures, definitions of its different sub-problems, model evaluation metrics and popular data repositories used in the study. Subsequently, we review a number of state-of-the-art multi-output learning methods, which are categorized based on the challenges.
Natively Interpretable Machine Learning and Artificial Intelligence: Preliminary Results and Future Directions
Authors: Christopher J. Hazard, Christopher Fusting, Michael Resnick, Michael Auerbach, Michael Meehan, Valeri Korobov
Date: 2019-01-02 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Interpretable Machine Learning, Model Compression, Reinforcement Learning

Machine learning models have become more and more complex in order to better
approximate complex functions. Although fruitful in many domains, the added
complexity has come at the cost of model interpretability. The once popular
k-nearest neighbors (kNN) approach, which finds and uses the most similar data
for reasoning, has received much less attention in recent decades due to
numerous problems when compared to other techniques. We show that many of these
historical problems with kNN can be overcome, and our contribution has
applications not only in machine learning but also in online learning, data
synthesis, anomaly detection, model compression, and reinforcement learning,
without sacrificing interpretability. We introduce a synthesis between kNN and
information theory that we hope will provide a clear path towards models that
are innately interpretable and auditable. Through this work we hope to gather
interest in combining kNN with information theory as a promising path to fully
auditable machine learning and artificial intelligence.
Opportunistic Learning: Budgeted Cost-Sensitive Learning from Data Streams
Authors: Mohammad Kachuee, Orpaz Goldstein, Kimmo Karkkainen, Sajad Darabi, Majid Sarrafzadeh
Date: 2019-01-02 00:00:00
Proceeding: ICLR 2019 5
Tasks: Learning-To-Rank, Reinforcement Learning

In many real-world learning scenarios, features are only acquirable at a cost
constrained under a budget. In this paper, we propose a novel approach for
cost-sensitive feature acquisition at the prediction-time. The suggested method
acquires features incrementally based on a context-aware feature-value
function. We formulate the problem in the reinforcement learning paradigm, and
introduce a reward function based on the utility of each feature. Specifically,
MC dropout sampling is used to measure expected variations of the model
uncertainty which is used as a feature-value function. Furthermore, we suggest
sharing representations between the class predictor and value function
estimator networks. The suggested approach is completely online and is readily
applicable to stream learning setups. The solution is evaluated on three
different datasets including the well-known MNIST dataset as a benchmark as
well as two cost-sensitive datasets: Yahoo Learning to Rank and a dataset in
the medical domain for diabetes classification. According to the results, the
proposed method is able to efficiently acquire features and make accurate
predictions.
Analysis of Contraction Effort Level in EMG-Based Gesture Recognition Using Hyperdimensional Computing
Authors: Ali Moin, Andy Zhou, Simone Benatti, Abbas Rahimi, Luca Benini, Jan M. Rabaey
Date: 2019-01-02 00:00:00
Tasks: General Classification, Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

Varying contraction levels of muscles is a big challenge in electromyography-based gesture recognition. Some use cases require the classifier to be robust against varying force changes, while others demand to distinguish between different effort levels of performing the same gesture. We use brain-inspired hyperdimensional computing paradigm to build classification models that are both robust to these variations and able to recognize multiple contraction levels. Experimental results on 5 subjects performing 9 gestures with 3 effort levels show up to 39.17% accuracy drop when training and testing across different effort levels, with up to 30.35% recovery after applying our algorithm.
Multitask Learning Deep Neural Networks to Combine Revealed and Stated Preference Data
Authors: Shenhao Wang, Qingyi Wang, Jinhua Zhao
Date: 2019-01-02 00:00:00
Tasks: Autonomous Vehicles

It is an enduring question how to combine revealed preference (RP) and stated preference (SP) data to analyze travel behavior. This study presents a framework of multitask learning deep neural networks (MTLDNNs) for this question, and demonstrates that MTLDNNs are more generic than the traditional nested logit (NL) method, due to its capacity of automatic feature learning and soft constraints. About 1,500 MTLDNN models are designed and applied to the survey data that was collected in Singapore and focused on the RP of four current travel modes and the SP with autonomous vehicles (AV) as the one new travel mode in addition to those in RP. We found that MTLDNNs consistently outperform six benchmark models and particularly the classical NL models by about 5% prediction accuracy in both RP and SP datasets. This performance improvement can be mainly attributed to the soft constraints specific to MTLDNNs, including its innovative architectural design and regularization methods, but not much to the generic capacity of automatic feature learning endowed by a standard feedforward DNN architecture. Besides prediction, MTLDNNs are also interpretable. The empirical results show that AV is mainly the substitute of driving and AV alternative-specific variables are more important than the socio-economic variables in determining AV adoption. Overall, this study introduces a new MTLDNN framework to combine RP and SP, and demonstrates its theoretical flexibility and empirical power for prediction and interpretation. Future studies can design new MTLDNN architectures to reflect the speciality of RP and SP and extend this work to other behavioral analysis.
Ancient Painting to Natural Image: A New Solution for Painting Processing
Authors: Tingting Qiao, Weijing Zhang, Miao Zhang, Zixuan Ma, Duanqing Xu
Date: 2019-01-02 00:00:00
Tasks: Style Transfer

Collecting a large-scale and well-annotated dataset for image processing has
become a common practice in computer vision. However, in the ancient painting
area, this task is not practical as the number of paintings is limited and
their style is greatly diverse. We, therefore, propose a novel solution for the
problems that come with ancient painting processing. This is to use domain
transfer to convert ancient paintings to photo-realistic natural images. By
doing so, the ancient painting processing problems become natural image
processing problems and models trained on natural images can be directly
applied to the transferred paintings. Specifically, we focus on Chinese ancient
flower, bird and landscape paintings in this work. A novel Domain Style
Transfer Network (DSTN) is proposed to transfer ancient paintings to natural
images which employ a compound loss to ensure that the transferred paintings
still maintain the color composition and content of the input paintings. The
experiment results show that the transferred paintings generated by the DSTN
have a better performance in both the human perceptual test and other image
processing tasks than other state-of-art methods, indicating the authenticity
of the transferred paintings and the superiority of the proposed method.
Clustering with Distributed Data
Authors: Soummya Kar, Brian Swenson
Date: 2019-01-01 00:00:00
Tasks: Clustering

We consider $K$-means clustering in networked environments (e.g., internet of
things (IoT) and sensor networks) where data is inherently distributed across
nodes and processing power at each node may be limited. We consider a
clustering algorithm referred to as networked $K$-means, or $NK$-means, which
relies only on local neighborhood information exchange. Information exchange is
limited to low-dimensional statistics and not raw data at the agents. The
proposed approach develops a parametric family of multi-agent clustering
objectives (parameterized by $\rho$) and associated distributed $NK$-means
algorithms (also parameterized by $\rho$). The $NK$-means algorithm with
parameter $\rho$ converges to a set of fixed points relative to the associated
multi-agent objective (designated as `generalized minima'). By appropriate
choice of $\rho$, the set of generalized minima may be brought arbitrarily
close to the set of Lloyd's minima. Thus, the $NK$-means algorithm may be used
to compute Lloyd's minima of the collective dataset up to arbitrary accuracy.
A weighted random survival forest
Authors: Lev V. Utkin, Andrei V. Konstantinov, Viacheslav S. Chukanov, Mikhail V. Kots, Mikhail A. Ryabinin, Anna A. Meldo
Date: 2019-01-01 00:00:00

A weighted random survival forest is presented in the paper. It can be
regarded as a modification of the random forest improving its performance. The
main idea underlying the proposed model is to replace the standard procedure of
averaging used for estimation of the random survival forest hazard function by
weighted avaraging where the weights are assigned to every tree and can be
veiwed as training paremeters which are computed in an optimal way by solving a
standard quadratic optimization problem maximizing Harrell's C-index. Numerical
examples with real data illustrate the outperformance of the proposed model in
comparison with the original random survival forest.
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
Authors: Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z. Qureshi, Mehran Ebrahimi
Date: 2019-01-01 00:00:00
Tasks: Image Inpainting

Over the last few years, deep learning techniques have yielded significant
improvements in image inpainting. However, many of these techniques fail to
reconstruct reasonable structures as they are commonly over-smoothed and/or
blurry. This paper develops a new approach for image inpainting that does a
better job of reproducing filled regions exhibiting fine details. We propose a
two-stage adversarial model EdgeConnect that comprises of an edge generator
followed by an image completion network. The edge generator hallucinates edges
of the missing region (both regular and irregular) of the image, and the image
completion network fills in the missing regions using hallucinated edges as a
priori. We evaluate our model end-to-end over the publicly available datasets
CelebA, Places2, and Paris StreetView, and show that it outperforms current
state-of-the-art techniques quantitatively and qualitatively. Code and models
available at: https://github.com/knazeri/edge-connect
Mapping Areas using Computer Vision Algorithms and Drones
Authors: Bashar Alhafni, Saulo Fernando Guedes, Lays Cavalcante Ribeiro, Juhyun Park, Jeongkyu Lee
Date: 2019-01-01 00:00:00

The goal of this paper is to implement a system, titled as Drone Map Creator
(DMC) using Computer Vision techniques. DMC can process visual information from
an HD camera in a drone and automatically create a map by stitching together
visual information captured by a drone. The proposed approach employs the
Speeded up robust features (SURF) method to detect the key points for each
image frame; then the corresponding points between the frames are identified by
maximizing the determinant of a Hessian matrix. Finally, two images are
stitched together by using the identified points. Our results show that despite
some limitations from the external environment, we could have successfully
stitched images together along video sequences.
Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds
Authors: Andrea Zanette, Emma Brunskill
Date: 2019-01-01 00:00:00
Tasks: Learning Theory, Reinforcement Learning, Reinforcement Learning (RL)

Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduce the barrier to using RL algorithms in practice. As a step towards this we derive an algorithm for finite horizon discrete MDPs and associated analysis that both yields state-of-the art worst-case regret bounds in the dominant terms and yields substantially tighter bounds if the RL environment has small environmental norm, which is a function of the variance of the next-state value functions. An important benefit of our algorithmic is that it does not require apriori knowledge of a bound on the environmental norm. As a result of our analysis, we also help address an open learning theory question~\cite{jiang2018open} about episodic MDPs with a constant upper-bound on the sum of rewards, providing a regret bound with no $H$-dependence in the leading term that scales a polynomial function of the number of episodes.
Nasal Patches and Curves for Expression-robust 3D Face Recognition
Authors: Mehryar Emambakhsh, Adrian Evans
Date: 2019-01-01 00:00:00
Tasks: Denoising, Face Recognition

The potential of the nasal region for expression robust 3D face recognition
is thoroughly investigated by a novel five-step algorithm. First, the nose tip
location is coarsely detected and the face is segmented, aligned and the nasal
region cropped. Then, a very accurate and consistent nasal landmarking
algorithm detects seven keypoints on the nasal region. In the third step, a
feature extraction algorithm based on the surface normals of Gabor-wavelet
filtered depth maps is utilised and, then, a set of spherical patches and
curves are localised over the nasal region to provide the feature descriptors.
The last step applies a genetic algorithm-based feature selector to detect the
most stable patches and curves over different facial expressions. The algorithm
provides the highest reported nasal region-based recognition ranks on the FRGC,
Bosphorus and BU-3DFE datasets. The results are comparable with, and in many
cases better than, many state-of-the-art 3D face recognition algorithms, which
use the whole facial domain. The proposed method does not rely on sophisticated
alignment or denoising steps, is very robust when only one sample per subject
is used in the gallery, and does not require a training step for the
landmarking algorithm. https://github.com/mehryaragha/NoseBiometrics
Augmentation Scheme for Dealing with Imbalanced Network Traffic Classification Using Deep Learning
Authors: Ramin Hasibi, Matin Shokri, Mehdi Dehghan
Date: 2019-01-01 00:00:00
Tasks: Data Augmentation, Density Estimation, General Classification, Management, Traffic Classification

One of the most important tasks in network management is identifying
different types of traffic flows. As a result, a type of management service,
called Network Traffic Classifier (NTC), has been introduced. One type of NTCs
that has gained huge attention in recent years applies deep learning on packets
in order to classify flows. Internet is an imbalanced environment i.e., some
classes of applications are a lot more populated than others e.g., HTTP.
Additionally, one of the challenges in deep learning methods is that they do
not perform well in imbalanced environments in terms of evaluation metrics such
as precision, recall, and $\mathrm{F_1}$ measure. In order to solve this
problem, we recommend the use of augmentation methods to balance the dataset.
In this paper, we propose a novel data augmentation approach based on the use
of Long Short Term Memory (LSTM) networks for generating traffic flow patterns
and Kernel Density Estimation (KDE) for replicating the numerical features of
each class. First, we use the LSTM network in order to learn and generate the
sequence of packets in a flow for classes with less population. Then, we
complete the features of the sequence with generating random values based on
the distribution of a certain feature, which will be estimated using KDE.
Finally, we compare the training of a Convolutional Recurrent Neural Network
(CRNN) in large-scale imbalanced, sampled, and augmented datasets. The
contribution of our augmentation scheme is then evaluated on all of the
datasets through measurements of precision, recall, and F1 measure for every
class of application. The results demonstrate that our scheme is well suited
for network traffic flow datasets and improves the performance of deep learning
algorithms when it comes to above-mentioned metrics.
Complementary reinforcement learning towards explainable agents
Authors: Jung Hoon Lee
Date: 2019-01-01 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) algorithms allow agents to learn skills and
strategies to perform complex tasks without detailed instructions or expensive
labelled training examples. That is, RL agents can learn, as we learn. Given
the importance of learning in our intelligence, RL has been thought to be one
of key components to general artificial intelligence, and recent breakthroughs
in deep reinforcement learning suggest that neural networks (NN) are natural
platforms for RL agents. However, despite the efficiency and versatility of
NN-based RL agents, their decision-making remains incomprehensible, reducing
their utilities. To deploy RL into a wider range of applications, it is
imperative to develop explainable NN-based RL agents. Here, we propose a method
to derive a secondary comprehensible agent from a NN-based RL agent, whose
decision-makings are based on simple rules. Our empirical evaluation of this
secondary agent's performance supports the possibility of building a
comprehensible and transparent agent using a NN-based RL agent.
Supervised Multiscale Dimension Reduction for Spatial Interaction Networks
Authors: Shaobo Han, David B. Dunson
Date: 2019-01-01 00:00:00
Tasks: Dimensionality Reduction

We introduce a multiscale supervised dimension reduction method for SPatial Interaction Network (SPIN) data, which consist of a collection of spatially coordinated interactions. This type of predictor arises when the sampling unit of data is composed of a collection of primitive variables, each of them being essentially unique, so that it becomes necessary to group the variables in order to simplify the representation and enhance interpretability. In this paper, we introduce an empirical Bayes approach called spinlets, which first constructs a partitioning tree to guide the reduction over multiple spatial granularities, and then refines the representation of predictors according to the relevance to the response. We consider an inverse Poisson regression model and propose a new multiscale generalized double Pareto prior, which is induced via a tree-structured parameter expansion scheme. Our approach is motivated by an application in soccer analytics, in which we obtain compact vectorial representations and readily interpretable visualizations of the complex network objects, supervised by the response of interest.
Handwritten Indic Character Recognition using Capsule Networks
Authors: Bodhisatwa Mandal, Suvam Dubey, Swarnendu Ghosh, Ritesh Sarkhel, Nibaran Das
Date: 2019-01-01 00:00:00
Proceeding: IEEE Applied Signal Processing Conference 2018 (ASPCON 2018) 2019 1

Convolutional neural networks(CNNs) has become one of the primary algorithms
for various computer vision tasks. Handwritten character recognition is a
typical example of such task that has also attracted attention. CNN
architectures such as LeNet and AlexNet have become very prominent over the
last two decades however the spatial invariance of the different kernels has
been a prominent issue till now. With the introduction of capsule networks,
kernels can work together in consensus with one another with the help of
dynamic routing, that combines individual opinions of multiple groups of
kernels called capsules to employ equivariance among kernels. In the current
work, we have implemented capsule network on handwritten Indic digits and
character datasets to show its superiority over networks like LeNet.
Furthermore, it has also been shown that they can boost the performance of
other networks like LeNet and AlexNet.