pwc-archive/papers-with-abstracts huggingface dataset:
Secure Deep Learning Engineering: A Software Quality Assurance Perspective
Authors: Lei Ma, Felix Juefei-Xu, Minhui Xue, Qiang Hu, Sen Chen, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, Simon See
Date: 2018-10-10 00:00:00
Tasks: Deep Learning

Over the past decades, deep learning (DL) systems have achieved tremendous
success and gained great popularity in various applications, such as
intelligent machines, image processing, speech processing, and medical
diagnostics. Deep neural networks are the key driving force behind its recent
success, but still seem to be a magic black box lacking interpretability and
understanding. This brings up many open safety and security issues with
enormous and urgent demands on rigorous methodologies and engineering practice
for quality enhancement. A plethora of studies have shown that the
state-of-the-art DL systems suffer from defects and vulnerabilities that can
lead to severe loss and tragedies, especially when applied to real-world
safety-critical applications. In this paper, we perform a large-scale study and
construct a paper repository of 223 relevant works to the quality assurance,
security, and interpretation of deep learning. We, from a software quality
assurance perspective, pinpoint challenges and future opportunities towards
universal secure deep learning engineering. We hope this work and the
accompanied paper repository can pave the path for the software engineering
community towards addressing the pressing industrial demand of secure
intelligent applications.
Random matrix-improved estimation of covariance matrix distances
Authors: Romain Couillet, Malik Tiomoko, Steeve Zozor, Eric Moisan
Date: 2018-10-10 00:00:00

Given two sets $x_1^{(1)},\ldots,x_{n_1}^{(1)}$ and
$x_1^{(2)},\ldots,x_{n_2}^{(2)}\in\mathbb{R}^p$ (or $\mathbb{C}^p$) of random
vectors with zero mean and positive definite covariance matrices $C_1$ and
$C_2\in\mathbb{R}^{p\times p}$ (or $\mathbb{C}^{p\times p}$), respectively,
this article provides novel estimators for a wide range of distances between
$C_1$ and $C_2$ (along with divergences between some zero mean and covariance
$C_1$ or $C_2$ probability measures) of the form $\frac1p\sum_{i=1}^n
f(\lambda_i(C_1^{-1}C_2))$ (with $\lambda_i(X)$ the eigenvalues of matrix $X$).
These estimators are derived using recent advances in the field of random
matrix theory and are asymptotically consistent as $n_1,n_2,p\to\infty$ with
non trivial ratios $p/n_1<1$ and $p/n_2<1$ (the case $p/n_2>1$ is also
discussed). A first "generic" estimator, valid for a large set of $f$
functions, is provided under the form of a complex integral. Then, for a
selected set of $f$'s of practical interest (namely, $f(t)=t$, $f(t)=\log(t)$,
$f(t)=\log(1+st)$ and $f(t)=\log^2(t)$), a closed-form expression is provided.
Beside theoretical findings, simulation results suggest an outstanding
performance advantage for the proposed estimators when compared to the
classical "plug-in" estimator $\frac1p\sum_{i=1}^n f(\lambda_i(\hat
C_1^{-1}\hat C_2))$ (with $\hat
C_a=\frac1{n_a}\sum_{i=1}^{n_a}x_i^{(a)}x_i^{(a){\sf T}}$), and this even for
very small values of $n_1,n_2,p$.
Is there Gender bias and stereotype in Portuguese Word Embeddings?
Authors: Brenda Salenave Santana, Vinicius Woloszyn, Leandro Krug Wives
Date: 2018-10-10 00:00:00
Tasks: Word Embeddings

In this work, we propose an analysis of the presence of gender bias
associated with professions in Portuguese word embeddings. The objective of
this work is to study gender implications related to stereotyped professions
for women and men in the context of the Portuguese language.
ET-Lasso: A New Efficient Tuning of Lasso-type Regularization for High-Dimensional Data
Authors: Songshan Yang, Jiawei Wen, Xiang Zhan, Daniel Kifer
Date: 2018-10-10 00:00:00
Tasks: feature selection

The L1 regularization (Lasso) has proven to be a versatile tool to select relevant features and estimate the model coefficients simultaneously and has been widely used in many research areas such as genomes studies, finance, and biomedical imaging. Despite its popularity, it is very challenging to guarantee the feature selection consistency of Lasso especially when the dimension of the data is huge. One way to improve the feature selection consistency is to select an ideal tuning parameter. Traditional tuning criteria mainly focus on minimizing the estimated prediction error or maximizing the posterior model probability, such as cross-validation and BIC, which may either be time-consuming or fail to control the false discovery rate (FDR) when the number of features is extremely large. The other way is to introduce pseudo-features to learn the importance of the original ones. Recently, the Knockoff filter is proposed to control the FDR when performing feature selection. However, its performance is sensitive to the choice of the expected FDR threshold. Motivated by these ideas, we propose a new method using pseudo-features to obtain an ideal tuning parameter. In particular, we present the Efficient Tuning of Lasso (ET-Lasso) to separate active and inactive features by adding permuted features as pseudo-features in linear models. The pseudo-features are constructed to be inactive by nature, which can be used to obtain a cutoff to select the tuning parameter that separates active and inactive features. Experimental studies on both simulations and real-world data applications are provided to show that ET-Lasso can effectively and efficiently select active features under a wide range of scenarios
LIRS: Enabling efficient machine learning on NVM-based storage via a lightweight implementation of random shuffling
Authors: Zhi-Lin Ke, Hsiang-Yun Cheng, Chia-Lin Yang
Date: 2018-10-10 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning algorithms, such as Support Vector Machine (SVM) and Deep
Neural Network (DNN), have gained a lot of interests recently. When training a
machine learning algorithm, randomly shuffle all the training data can improve
the testing accuracy and boost the convergence rate. Nevertheless, realizing
training data random shuffling in a real system is not a straightforward
process due to the slow random accesses in hard disk drive (HDD). To avoid
frequent random disk access, the effect of random shuffling is often limited in
existing approaches. With the emerging non-volatile memory-based storage
device, such as Intel Optane SSD, which provides fast random accesses, we
propose a lightweight implementation of random shuffling (LIRS) to randomly
shuffle the indexes of the entire training dataset, and the selected training
instances are directly accessed from the storage and packed into batches.
Experimental results show that LIRS can reduce the total training time of SVM
and DNN by 49.9% and 43.5% on average, and improve the final testing accuracy
on DNN by 1.01%.
Multi-class Classification Model Inspired by Quantum Detection Theory
Authors: Prayag Tiwari, Massimo Melucci
Date: 2018-10-10 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Multi-class Classification

Machine Learning has become very famous currently which assist in identifying
the patterns from the raw data. Technological advancement has led to
substantial improvement in Machine Learning which, thus helping to improve
prediction. Current Machine Learning models are based on Classical Theory,
which can be replaced by Quantum Theory to improve the effectiveness of the
model. In the previous work, we developed binary classifier inspired by Quantum
Detection Theory. In this extended abstract, our main goal is to develop
multi-class classifier. We generally use the terminology multinomial
classification or multi-class classification when we have a classification
problem for classifying observations or instances into one of three or more
classes.
Domain Confusion with Self Ensembling for Unsupervised Adaptation
Authors: Jiawei Wang, Zhaoshui He, Chengjian Feng, Zhouping Zhu, Qinzhuang Lin, Jun Lv, Shengli Xie
Date: 2018-10-10 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Data collection and annotation are time-consuming in machine learning, expecially for large scale problem. A common approach for this problem is to transfer knowledge from a related labeled domain to a target one. There are two popular ways to achieve this goal: adversarial learning and self training. In this article, we first analyze the training unstablity problem and the mistaken confusion issue in adversarial learning process. Then, inspired by domain confusion and self-ensembling methods, we propose a combined model to learn feature and class jointly invariant representation, namely Domain Confusion with Self Ensembling (DCSE). The experiments verified that our proposed approach can offer better performance than empirical art in a variety of unsupervised domain adaptation benchmarks.
Decentralized Cooperative Stochastic Bandits
Authors: David Martínez-Rubio, Varun Kanade, Patrick Rebeschini
Date: 2018-10-10 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Multi-Armed Bandits

We study a decentralized cooperative stochastic multi-armed bandit problem with $K$ arms on a network of $N$ agents. In our model, the reward distribution of each arm is the same for each agent and rewards are drawn independently across agents and time steps. In each round, each agent chooses an arm to play and subsequently sends a message to her neighbors. The goal is to minimize the overall regret of the entire network. We design a fully decentralized algorithm that uses an accelerated consensus procedure to compute (delayed) estimates of the average of rewards obtained by all the agents for each arm, and then uses an upper confidence bound (UCB) algorithm that accounts for the delay and error of the estimates. We analyze the regret of our algorithm and also provide a lower bound. The regret is bounded by the optimal centralized regret plus a natural and simple term depending on the spectral gap of the communication matrix. Our algorithm is simpler to analyze than those proposed in prior work and it achieves better regret bounds, while requiring less information about the underlying network. It also performs better empirically.
SECaps: A Sequence Enhanced Capsule Model for Charge Prediction
Authors: Congqing He, Li Peng, Yuquan Le, JiaWei He, Xiangyu Zhu
Date: 2018-10-10 00:00:00
Tasks: Prediction

Automatic charge prediction aims to predict appropriate final charges according to the fact descriptions for a given criminal case. Automatic charge prediction plays a critical role in assisting judges and lawyers to improve the efficiency of legal decisions, and thus has received much attention. Nevertheless, most existing works on automatic charge prediction perform adequately on high-frequency charges but are not yet capable of predicting few-shot charges with limited cases. In this paper, we propose a Sequence Enhanced Capsule model, dubbed as SECaps model, to relieve this problem. Specifically, following the work of capsule networks, we propose the seq-caps layer, which considers sequence information and spatial information of legal texts simultaneously. Then we design a attention residual unit, which provides auxiliary information for charge prediction. In addition, our SECaps model introduces focal loss, which relieves the problem of imbalanced charges. Comparing the state-of-the-art methods, our SECaps model obtains 4.5% and 6.4% absolutely considerable improvements under Macro F1 in Criminal-S and Criminal-L respectively. The experimental results consistently demonstrate the superiorities and competitiveness of our proposed model.
Let's take a Walk on Superpixels Graphs: Deformable Linear Objects Segmentation and Model Estimation
Authors: Daniele De Gregorio, Gianluca Palli, Luigi Di Stefano
Date: 2018-10-10 00:00:00
Tasks: Pose Estimation, Semantic Segmentation, Superpixels

While robotic manipulation of rigid objects is quite straightforward, coping
with deformable objects is an open issue. More specifically, tasks like tying a
knot, wiring a connector or even surgical suturing deal with the domain of
Deformable Linear Objects (DLOs). In particular the detection of a DLO is a
non-trivial problem especially under clutter and occlusions (as well as
self-occlusions). The pose estimation of a DLO results into the identification
of its parameters related to a designed model, e.g. a basis spline. It follows
that the stand-alone segmentation of a DLO might not be sufficient to conduct a
full manipulation task. This is why we propose a novel framework able to
perform both a semantic segmentation and b-spline modeling of multiple
deformable linear objects simultaneously without strict requirements about
environment (i.e. the background). The core algorithm is based on biased random
walks over the Region Adiacency Graph built on a superpixel oversegmentation of
the source image. The algorithm is initialized by a Convolutional Neural
Networks that detects the DLO's endcaps. An open source implementation of the
proposed approach is also provided to easy the reproduction of the whole
detection pipeline along with a novel cables dataset in order to encourage
further experiments.
Invariance Analysis of Saliency Models versus Human Gaze During Scene Free Viewing
Authors: Zhaohui Che, Ali Borji, Guangtao Zhai, Xiongkuo Min
Date: 2018-10-10 00:00:00
Tasks: Data Augmentation

Most of current studies on human gaze and saliency modeling have used
high-quality stimuli. In real world, however, captured images undergo various
types of distortions during the whole acquisition, transmission, and displaying
chain. Some distortion types include motion blur, lighting variations and
rotation. Despite few efforts, influences of ubiquitous distortions on visual
attention and saliency models have not been systematically investigated. In
this paper, we first create a large-scale database including eye movements of
10 observers over 1900 images degraded by 19 types of distortions. Second, by
analyzing eye movements and saliency models, we find that: a) observers look at
different locations over distorted versus original images, and b) performances
of saliency models are drastically hindered over distorted images, with the
maximum performance drop belonging to Rotation and Shearing distortions.
Finally, we investigate the effectiveness of different distortions when serving
as data augmentation transformations. Experimental results verify that some
useful data augmentation transformations which preserve human gaze of reference
images can improve deep saliency models against distortions, while some invalid
transformations which severely change human gaze will degrade the performance.
AI Learns to Recognize Bengali Handwritten Digits: Bengali.AI Computer Vision Challenge 2018
Authors: Sharif Amit Kamran, Ahmed Imtiaz Humayun, Samiul Alam, Rashed Mohammad Doha, Manash Kumar Mandal, Tahsin Reasat, Fuad Rahman
Date: 2018-10-10 00:00:00
Tasks: BIG-bench Machine Learning

Solving problems with Artificial intelligence in a competitive manner has
long been absent in Bangladesh and Bengali-speaking community. On the other
hand, there has not been a well structured database for Bengali Handwritten
digits for mass public use. To bring out the best minds working in machine
learning and use their expertise to create a model which can easily recognize
Bengali Handwritten digits, we organized Bengali.AI Computer Vision
Challenge.The challenge saw both local and international teams participating
with unprecedented efforts.
Learning to Communicate Implicitly By Actions
Authors: Zheng Tian, Shihao Zou, Ian Davies, Tim Warr, Lisheng Wu, Haitham Bou Ammar, Jun Wang
Date: 2018-10-10 00:00:00

In situations where explicit communication is limited, human collaborators act by learning to: (i) infer meaning behind their partner's actions, and (ii) convey private information about the state to their partner implicitly through actions. The first component of this learning process has been well-studied in multi-agent systems, whereas the second --- which is equally crucial for successful collaboration --- has not. To mimic both components mentioned above, thereby completing the learning process, we introduce a novel algorithm: Policy Belief Learning (PBL). PBL uses a belief module to model the other agent's private information and a policy module to form a distribution over actions informed by the belief module. Furthermore, to encourage communication by actions, we propose a novel auxiliary reward which incentivizes one agent to help its partner to make correct inferences about its private information. The auxiliary reward for communication is integrated into the learning of the policy module. We evaluate our approach on a set of environments including a matrix game, particle environment and the non-competitive bidding problem from contract bridge. We show empirically that this auxiliary reward is effective and easy to generalize. These results demonstrate that our PBL algorithm can produce strong pairs of agents in collaborative games where explicit communication is disabled.
New Vistas to study Bhartrhari: Cognitive NLP
Authors: Jayashree Gajjam, Diptesh Kanojia, Malhar Kulkarni
Date: 2018-10-10 00:00:00
Tasks: Sentence

The Sanskrit grammatical tradition which has commenced with Panini's
Astadhyayi mostly as a Padasastra has culminated as a Vakyasastra, at the hands
of Bhartrhari. The grammarian-philosopher Bhartrhari and his authoritative work
'Vakyapadiya' have been a matter of study for modern scholars, at least for
more than 50 years, since Ashok Aklujkar submitted his Ph.D. dissertation at
Harvard University. The notions of a sentence and a word as a meaningful
linguistic unit in the language have been a subject matter for the discussion
in many works that followed later on. While some scholars have applied
philological techniques to critically establish the text of the works of
Bhartrhari, some others have devoted themselves to exploring philosophical
insights from them. Some others have studied his works from the point of view
of modern linguistics, and psychology. Few others have tried to justify the
views by logical discussions.
  In this paper, we present a fresh view to study Bhartrhari, and his works,
especially the 'Vakyapadiya'. This view is from the field of Natural Language
Processing (NLP), more specifically, what is called as Cognitive NLP. We have
studied the definitions of a sentence given by Bhartrhari at the beginning of
the second chapter of 'Vakyapadiya'. We have researched one of these
definitions by conducting an experiment and following the methodology of
silent-reading of Sanskrit paragraphs. We collect the Gaze-behavior data of
participants and analyze it to understand the underlying comprehension
procedure in the human mind and present our results. We evaluate the
statistical significance of our results using T-test, and discuss the caveats
of our work. We also present some general remarks on this experiment and
usefulness of this method for gaining more insights in the work of Bhartrhari.
Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
Authors: Rika Antonova, Mia Kokic, Johannes A. Stork, Danica Kragic
Date: 2018-10-10 00:00:00
Tasks: Bayesian Optimization

We develop an approach that benefits from large simulated datasets and takes
full advantage of the limited online data that is most relevant. We propose a
variant of Bayesian optimization that alternates between using informed and
uninformed kernels. With this Bernoulli Alternation Kernel we ensure that
discrepancies between simulation and reality do not hinder adapting robot
control policies online. The proposed approach is applied to a challenging
real-world problem of task-oriented grasping with novel objects. Our further
contribution is a neural network architecture and training pipeline that use
experience from grasping objects in simulation to learn grasp stability scores.
We learn task scores from a labeled dataset with a convolutional network, which
is used to construct an informed kernel for our variant of Bayesian
optimization. Experiments on an ABB Yumi robot with real sensor data
demonstrate success of our approach, despite the challenge of fulfilling task
requirements and high uncertainty over physical properties of objects.
Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists
Authors: Giancarlo D. Salton, John D. Kelleher
Date: 2018-10-10 00:00:00

Language Models (LMs) are important components in several Natural Language
Processing systems. Recurrent Neural Network LMs composed of LSTM units,
especially those augmented with an external memory, have achieved
state-of-the-art results. However, these models still struggle to process long
sequences which are more likely to contain long-distance dependencies because
of information fading and a bias towards more recent information. In this paper
we demonstrate an effective mechanism for retrieving information in a memory
augmented LSTM LM based on attending to information in memory in proportion to
the number of timesteps the LSTM gating mechanism persisted the information.
Lazy-CFR: fast and near optimal regret minimization for extensive games with imperfect information
Authors: Yichi Zhou, Tongzheng Ren, Jialian Li, Dong Yan, Jun Zhu
Date: 2018-10-10 00:00:00
Tasks: counterfactual

Counterfactual regret minimization (CFR) is the most popular algorithm on
solving two-player zero-sum extensive games with imperfect information and
achieves state-of-the-art performance in practice. However, the performance of
CFR is not fully understood, since empirical results on the regret are much
better than the upper bound proved in \cite{zinkevich2008regret}. Another issue
is that CFR has to traverse the whole game tree in each round, which is
time-consuming in large scale games. In this paper, we present a novel
technique, lazy update, which can avoid traversing the whole game tree in CFR,
as well as a novel analysis on the regret of CFR with lazy update. Our analysis
can also be applied to the vanilla CFR, resulting in a much tighter regret
bound than that in \cite{zinkevich2008regret}. Inspired by lazy update, we
further present a novel CFR variant, named Lazy-CFR. Compared to traversing
$O(|\mathcal{I}|)$ information sets in vanilla CFR, Lazy-CFR needs only to
traverse $O(\sqrt{|\mathcal{I}|})$ information sets per round while keeping the
regret bound almost the same, where $\mathcal{I}$ is the class of all
information sets. As a result, Lazy-CFR shows better convergence result
compared with vanilla CFR. Experimental results consistently show that Lazy-CFR
outperforms the vanilla CFR significantly.
Improving Neural Text Simplification Model with Simplified Corpora
Authors: Jipeng Qiang
Date: 2018-10-10 00:00:00
Tasks: Decoder, Lexical Simplification, Machine Translation, model, Sentence, Text Simplification, Translation

Text simplification (TS) can be viewed as monolingual translation task,
translating between text variations within a single language. Recent neural TS
models draw on insights from neural machine translation to learn lexical
simplification and content reduction using encoder-decoder model. But different
from neural machine translation, we cannot obtain enough ordinary and
simplified sentence pairs for TS, which are expensive and time-consuming to
build. Target-side simplified sentences plays an important role in boosting
fluency for statistical TS, and we investigate the use of simplified sentences
to train, with no changes to the network architecture. We propose to pair
simple training sentence with a synthetic ordinary sentence via
back-translation, and treating this synthetic data as additional training data.
We train encoder-decoder model using synthetic sentence pairs and original
sentence pairs, which can obtain substantial improvements on the available
WikiLarge data and WikiSmall data compared with the state-of-the-art methods.