pwc-archive/papers-with-abstracts huggingface dataset:
Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection
Authors: Yisroel Mirsky, Tomer Doitshman, Yuval Elovici, Asaf Shabtai
Date: 2018-02-25 00:00:00
Tasks: Intrusion Detection, Network Intrusion Detection

Neural networks have become an increasingly popular solution for network
intrusion detection systems (NIDS). Their capability of learning complex
patterns and behaviors make them a suitable solution for differentiating
between normal traffic and network attacks. However, a drawback of neural
networks is the amount of resources needed to train them. Many network gateways
and routers devices, which could potentially host an NIDS, simply do not have
the memory or processing power to train and sometimes even execute such models.
More importantly, the existing neural network solutions are trained in a
supervised manner. Meaning that an expert must label the network traffic and
update the model manually from time to time.
  In this paper, we present Kitsune: a plug and play NIDS which can learn to
detect attacks on the local network, without supervision, and in an efficient
online manner. Kitsune's core algorithm (KitNET) uses an ensemble of neural
networks called autoencoders to collectively differentiate between normal and
abnormal traffic patterns. KitNET is supported by a feature extraction
framework which efficiently tracks the patterns of every network channel. Our
evaluations show that Kitsune can detect various attacks with a performance
comparable to offline anomaly detectors, even on a Raspberry PI. This
demonstrates that Kitsune can be a practical and economic NIDS.
Entropy based Independent Learning in Anonymous Multi-Agent Settings
Authors: Tanvi Verma, Pradeep Varakantham, Hoong Chuin Lau
Date: 2018-03-27 00:00:00
Tasks: Fairness, Multi-agent Reinforcement Learning

Efficient sequential matching of supply and demand is a problem of interest in many online to offline services. For instance, Uber, Lyft, Grab for matching taxis to customers; Ubereats, Deliveroo, FoodPanda etc for matching restaurants to customers. In these online to offline service problems, individuals who are responsible for supply (e.g., taxi drivers, delivery bikes or delivery van drivers) earn more by being at the "right" place at the "right" time. We are interested in developing approaches that learn to guide individuals to be in the "right" place at the "right" time (to maximize revenue) in the presence of other similar "learning" individuals and only local aggregated observation of other agents states (e.g., only number of other taxis in same zone as current agent). A key characteristic of the domains of interest is that the interactions between individuals are anonymous, i.e., the outcome of an interaction (competing for demand) is dependent only on the number and not on the identity of the agents. We model these problems using the Anonymous MARL (AyMARL) model. The key contribution of this paper is in employing principle of maximum entropy to provide a general framework of independent learning that is both empirically effective (even with only local aggregated information of agent population distribution) and theoretically justified. Finally, our approaches provide a significant improvement with respect to joint and individual revenue on a generic simulator for online to offline services and a real world taxi problem over existing approaches. More importantly, this is achieved while having the least variance in revenues earned by the learning individuals, an indicator of fairness.
Semantic Explanations of Predictions
Authors: Freddy Lecue, Jiewen Wu
Date: 2018-05-27 00:00:00
Tasks: General Classification

The main objective of explanations is to transmit knowledge to humans. This
work proposes to construct informative explanations for predictions made from
machine learning models. Motivated by the observations from social sciences,
our approach selects data points from the training sample that exhibit special
characteristics crucial for explanation, for instance, ones contrastive to the
classification prediction and ones representative of the models. Subsequently,
semantic concepts are derived from the selected data points through the use of
domain ontologies. These concepts are filtered and ranked to produce
informative explanations that improves human understanding. The main features
of our approach are that (1) knowledge about explanations is captured in the
form of ontological concepts, (2) explanations include contrastive evidences in
addition to normal evidences, and (3) explanations are user relevant.
Online Learning for Non-Stationary A/B Tests
Authors: Andrés Muñoz Medina, Sergei Vassilvitskii, Dong Yin
Date: 2018-02-14 00:00:00

The rollout of new versions of a feature in modern applications is a manual
multi-stage process, as the feature is released to ever larger groups of users,
while its performance is carefully monitored. This kind of A/B testing is
ubiquitous, but suboptimal, as the monitoring requires heavy human
intervention, is not guaranteed to capture consistent, but short-term
fluctuations in performance, and is inefficient, as better versions take a long
time to reach the full population.
  In this work we formulate this question as that of expert learning, and give
a new algorithm Follow-The-Best-Interval, FTBI, that works in dynamic,
non-stationary environments. Our approach is practical, simple, and efficient,
and has rigorous guarantees on its performance. Finally, we perform a thorough
evaluation on synthetic and real world datasets and show that our approach
outperforms current state-of-the-art methods.
Breaking the Activation Function Bottleneck through Adaptive Parameterization
Authors: Sebastian Flennerhag, Hujun Yin, John Keane, Mark Elliot
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12

Standard neural network architectures are non-linear only by virtue of a
simple element-wise activation function, making them both brittle and
excessively large. In this paper, we consider methods for making the
feed-forward layer more flexible while preserving its basic structure. We
develop simple drop-in replacements that learn to adapt their parameterization
conditional on the input, thereby increasing statistical efficiency
significantly. We present an adaptive LSTM that advances the state of the art
for the Penn Treebank and WikiText-2 word-modeling tasks while using fewer
parameters and converging in less than half as many iterations.
Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings
Authors: Dat Quoc Nguyen, Karin Verspoor
Date: 2018-05-27 00:00:00
Proceeding: WS 2018 7
Tasks: Relation, Relation Extraction, Word Embeddings

We investigate the incorporation of character-based word representations into
a standard CNN-based relation extraction model. We experiment with two common
neural architectures, CNN and LSTM, to learn word vector representations from
character embeddings. Through a task on the BioCreative-V CDR corpus,
extracting relationships between chemicals and diseases, we show that models
exploiting the character-based word representations improve on models that do
not use this information, obtaining state-of-the-art result relative to
previous neural approaches.
Dual Swap Disentangling
Authors: Zunlei Feng, Xinchao Wang, Chenglong Ke, An-Xiang Zeng, DaCheng Tao, Mingli Song
Date: 2018-05-27 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Attribute

Learning interpretable disentangled representations is a crucial yet challenging task. In this paper, we propose a weakly semi-supervised method, termed as Dual Swap Disentangling (DSD), for disentangling using both labeled and unlabeled data. Unlike conventional weakly supervised methods that rely on full annotations on the group of samples, we require only limited annotations on paired samples that indicate their shared attribute like the color. Our model takes the form of a dual autoencoder structure. To achieve disentangling using the labeled pairs, we follow a "encoding-swap-decoding" process, where we first swap the parts of their encodings corresponding to the shared attribute and then decode the obtained hybrid codes to reconstruct the original input pairs. For unlabeled pairs, we follow the "encoding-swap-decoding" process twice on designated encoding parts and enforce the final outputs to approximate the input pairs. By isolating parts of the encoding and swapping them back and forth, we impose the dimension-wise modularity and portability of the encodings of the unlabeled samples, which implicitly encourages disentangling under the guidance of labeled pairs. This dual swap mechanism, tailored for semi-supervised setting, turns out to be very effective. Experiments on image datasets from a wide domain show that our model yields state-of-the-art disentangling performances.
Metric-Optimized Example Weights
Authors: Sen Zhao, Mahdi Milani Fard, Harikrishna Narasimhan, Maya Gupta
Date: 2018-05-27 00:00:00
Proceeding: ICLR 2019 5

Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard loss, where the weights on the training examples are learned to optimize the test metric on a validation set. These metric-optimized example weights can be learned for any test metric, including black box and customized ones for specific applications. We illustrate the performance of the proposed method on diverse public benchmark datasets and real-world applications. We also provide a generalization bound for the method.
Convex Sparse Spectral Clustering: Single-view to Multi-view
Authors: Canyi Lu, Shuicheng Yan, Zhouchen Lin
Date: 2015-11-21 00:00:00
Tasks: Clustering

Spectral Clustering (SC) is one of the most widely used methods for data
clustering. It first finds a low-dimensonal embedding $U$ of data by computing
the eigenvectors of the normalized Laplacian matrix, and then performs k-means
on $U^\top$ to get the final clustering result. In this work, we observe that,
in the ideal case, $UU^\top$ should be block diagonal and thus sparse.
Therefore we propose the Sparse Spectral Clustering (SSC) method which extends
SC with sparse regularization on $UU^\top$. To address the computational issue
of the nonconvex SSC model, we propose a novel convex relaxation of SSC based
on the convex hull of the fixed rank projection matrices. Then the convex SSC
model can be efficiently solved by the Alternating Direction Method of
\canyi{Multipliers} (ADMM). Furthermore, we propose the Pairwise Sparse
Spectral Clustering (PSSC) which extends SSC to boost the clustering
performance by using the multi-view information of data. Experimental
comparisons with several baselines on real-world datasets testify to the
efficacy of our proposed methods.
Generalized Singular Value Thresholding
Authors: Canyi Lu, Changbo Zhu, Chunyan Xu, Shuicheng Yan, Zhouchen Lin
Date: 2014-12-06 00:00:00

This work studies the Generalized Singular Value Thresholding (GSVT) operator
${\text{Prox}}_{g}^{{\sigma}}(\cdot)$, \begin{equation*}
  {\text{Prox}}_{g}^{{\sigma}}(B)=\arg\min\limits_{X}\sum_{i=1}^{m}g(\sigma_{i}(X))
+ \frac{1}{2}||X-B||_{F}^{2}, \end{equation*} associated with a nonconvex
function $g$ defined on the singular values of $X$. We prove that GSVT can be
obtained by performing the proximal operator of $g$ (denoted as
$\text{Prox}_g(\cdot)$) on the singular values since $\text{Prox}_g(\cdot)$ is
monotone when $g$ is lower bounded. If the nonconvex $g$ satisfies some
conditions (many popular nonconvex surrogate functions, e.g., $\ell_p$-norm,
$0<p<1$, of $\ell_0$-norm are special cases), a general solver to find
$\text{Prox}_g(b)$ is proposed for any $b\geq0$. GSVT greatly generalizes the
known Singular Value Thresholding (SVT) which is a basic subroutine in many
convex low rank minimization methods. We are able to solve the nonconvex low
rank minimization problem by using GSVT in place of SVT.
Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers
Authors: Yonatan Geifman, Guy Uziel, Ran El-Yaniv
Date: 2018-05-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: Classification, General Classification

We consider the problem of uncertainty estimation in the context of
(non-Bayesian) deep neural classification. In this context, all known methods
are based on extracting uncertainty signals from a trained network optimized to
solve the classification problem at hand. We demonstrate that such techniques
tend to introduce biased estimates for instances whose predictions are supposed
to be highly confident. We argue that this deficiency is an artifact of the
dynamics of training with SGD-like optimizers, and it has some properties
similar to overfitting. Based on this observation, we develop an uncertainty
estimation algorithm that selectively estimates the uncertainty of highly
confident points, using earlier snapshots of the trained model, before their
estimates are jittered (and way before they are ready for actual
classification). We present extensive experiments indicating that the proposed
algorithm provides uncertainty estimates that are consistently better than all
known methods.
BRITS: Bidirectional Recurrent Imputation for Time Series
Authors: Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei LI, Yitan Li
Date: 2018-05-27 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: General Classification, Imputation, Missing Values, Multivariate Time Series Forecasting, Multivariate Time Series Imputation, regression, Time Series, Time Series Analysis, Traffic Data Imputation

Time series are widely used as signals in many classification/regression
tasks. It is ubiquitous that time series contains many missing values. Given
multiple correlated time series data, how to fill in missing values and to
predict their class labels? Existing imputation methods often impose strong
assumptions of the underlying data generating process, such as linear dynamics
in the state space. In this paper, we propose BRITS, a novel method based on
recurrent neural networks for missing value imputation in time series data. Our
proposed method directly learns the missing values in a bidirectional recurrent
dynamical system, without any specific assumption. The imputed values are
treated as variables of RNN graph and can be effectively updated during the
backpropagation.BRITS has three advantages: (a) it can handle multiple
correlated missing values in time series; (b) it generalizes to time series
with nonlinear dynamics underlying; (c) it provides a data-driven imputation
procedure and applies to general settings with missing data.We evaluate our
model on three real-world datasets, including an air quality dataset, a
health-care data, and a localization data for human activity. Experiments show
that our model outperforms the state-of-the-art methods in both imputation and
classification/regression accuracies.
Generating Fine-Grained Open Vocabulary Entity Type Descriptions
Authors: Rajarshi Bhowmik, Gerard de Melo
Date: 2018-05-27 00:00:00
Proceeding: ACL 2018 7
Tasks: Knowledge Graphs, Vocal Bursts Type Prediction

While large-scale knowledge graphs provide vast amounts of structured facts
about entities, a short textual description can often be useful to succinctly
characterize an entity and its type. Unfortunately, many knowledge graph
entities lack such textual descriptions. In this paper, we introduce a dynamic
memory-based network that generates a short open vocabulary description of an
entity by jointly leveraging induced fact embeddings as well as the dynamic
context of the generated sequence of words. We demonstrate the ability of our
architecture to discern relevant information for more accurate generation of
type description by pitting the system against several strong baselines.
Prediction with a Short Memory
Authors: Vatsal Sharan, Sham Kakade, Percy Liang, Gregory Valiant
Date: 2016-12-08 00:00:00
Tasks: Prediction

We consider the problem of predicting the next observation given a sequence
of past observations, and consider the extent to which accurate prediction
requires complex algorithms that explicitly leverage long-range dependencies.
Perhaps surprisingly, our positive results show that for a broad class of
sequences, there is an algorithm that predicts well on average, and bases its
predictions only on the most recent few observation together with a set of
simple summary statistics of the past observations. Specifically, we show that
for any distribution over observations, if the mutual information between past
observations and future observations is upper bounded by $I$, then a simple
Markov model over the most recent $I/\epsilon$ observations obtains expected KL
error $\epsilon$---and hence $\ell_1$ error $\sqrt{\epsilon}$---with respect to
the optimal predictor that has access to the entire past and knows the data
generating distribution. For a Hidden Markov Model with $n$ hidden states, $I$
is bounded by $\log n$, a quantity that does not depend on the mixing time, and
we show that the trivial prediction algorithm based on the empirical
frequencies of length $O(\log n/\epsilon)$ windows of observations achieves
this error, provided the length of the sequence is $d^{\Omega(\log
n/\epsilon)}$, where $d$ is the size of the observation alphabet.
  We also establish that this result cannot be improved upon, even for the
class of HMMs, in the following two senses: First, for HMMs with $n$ hidden
states, a window length of $\log n/\epsilon$ is information-theoretically
necessary to achieve expected $\ell_1$ error $\sqrt{\epsilon}$. Second, the
$d^{\Theta(\log n/\epsilon)}$ samples required to estimate the Markov model for
an observation alphabet of size $d$ is necessary for any computationally
tractable learning algorithm, assuming the hardness of strongly refuting a
certain class of CSPs.
cpSGD: Communication-efficient and differentially-private distributed SGD
Authors: Naman Agarwal, Ananda Theertha Suresh, Felix Yu, Sanjiv Kumar, H. Brendan McMahan
Date: 2018-05-27 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Privacy Preserving

Distributed stochastic gradient descent is an important subroutine in
distributed learning. A setting of particular interest is when the clients are
mobile devices, where two important concerns are communication efficiency and
the privacy of the clients. Several recent works have focused on reducing the
communication cost or introducing privacy guarantees, but none of the proposed
communication efficient methods are known to be privacy preserving and none of
the known privacy mechanisms are known to be communication efficient. To this
end, we study algorithms that achieve both communication efficiency and
differential privacy. For $d$ variables and $n \approx d$ clients, the proposed
method uses $O(\log \log(nd))$ bits of communication per client per coordinate
and ensures constant privacy.
  We also extend and improve previous analysis of the \emph{Binomial mechanism}
showing that it achieves nearly the same utility as the Gaussian mechanism,
while requiring fewer representation bits, which can be of independent
interest.
DPW-SDNet: Dual Pixel-Wavelet Domain Deep CNNs for Soft Decoding of JPEG-Compressed Images
Authors: Honggang Chen, Xiaohai He, Linbo Qing, Shuhua Xiong, Truong Q. Nguyen
Date: 2018-05-27 00:00:00
Tasks: Blocking, JPEG Artifact Correction

JPEG is one of the widely used lossy compression methods. JPEG-compressed
images usually suffer from compression artifacts including blocking and
blurring, especially at low bit-rates. Soft decoding is an effective solution
to improve the quality of compressed images without changing codec or
introducing extra coding bits. Inspired by the excellent performance of the
deep convolutional neural networks (CNNs) on both low-level and high-level
computer vision problems, we develop a dual pixel-wavelet domain deep
CNNs-based soft decoding network for JPEG-compressed images, namely DPW-SDNet.
The pixel domain deep network takes the four downsampled versions of the
compressed image to form a 4-channel input and outputs a pixel domain
prediction, while the wavelet domain deep network uses the 1-level discrete
wavelet transformation (DWT) coefficients to form a 4-channel input to produce
a DWT domain prediction. The pixel domain and wavelet domain estimates are
combined to generate the final soft decoded result. Experimental results
demonstrate the superiority of the proposed DPW-SDNet over several
state-of-the-art compression artifacts reduction algorithms.
Hierarchical Representation Learning for Kinship Verification
Authors: Naman Kohli, Mayank Vatsa, Richa Singh, Afzel Noore, Angshul Majumdar
Date: 2018-05-27 00:00:00
Tasks: Face Verification, Kinship Verification, Representation Learning

Kinship verification has a number of applications such as organizing large
collections of images and recognizing resemblances among humans. In this
research, first, a human study is conducted to understand the capabilities of
human mind and to identify the discriminatory areas of a face that facilitate
kinship-cues. Utilizing the information obtained from the human study, a
hierarchical Kinship Verification via Representation Learning (KVRL) framework
is utilized to learn the representation of different face regions in an
unsupervised manner. We propose a novel approach for feature representation
termed as filtered contractive deep belief networks (fcDBN). The proposed
feature representation encodes relational information present in images using
filters and contractive regularization penalty. A compact representation of
facial images of kin is extracted as an output from the learned model and a
multi-layer neural network is utilized to verify the kin accurately. A new WVU
Kinship Database is created which consists of multiple images per subject to
facilitate kinship verification. The results show that the proposed deep
learning framework (KVRL-fcDBN) yields stateof-the-art kinship verification
accuracy on the WVU Kinship database and on four existing benchmark datasets.
Further, kinship information is used as a soft biometric modality to boost the
performance of face verification via product of likelihood ratio and support
vector machine based approaches. Using the proposed KVRL-fcDBN framework, an
improvement of over 20% is observed in the performance of face verification.
Deep Watershed Detector for Music Object Recognition
Authors: Lukas Tuggener, Ismail Elezi, Jurgen Schmidhuber, Thilo Stadelmann
Date: 2018-05-26 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Novel Object Detection, Object, object-detection, Object Detection, Object Recognition, Retrieval

Optical Music Recognition (OMR) is an important and challenging area within
music information retrieval, the accurate detection of music symbols in digital
images is a core functionality of any OMR pipeline. In this paper, we introduce
a novel object detection method, based on synthetic energy maps and the
watershed transform, called Deep Watershed Detector (DWD). Our method is
specifically tailored to deal with high resolution images that contain a large
number of very small objects and is therefore able to process full pages of
written music. We present state-of-the-art detection results of common music
symbols and show DWD's ability to work with synthetic scores equally well as on
handwritten music.