pwc-archive/papers-with-abstracts huggingface dataset:
Probabilistic Recurrent State-Space Models
Authors: Andreas Doerr, Christian Daniel, Martin Schiegg, Duy Nguyen-Tuong, Stefan Schaal, Marc Toussaint, Sebastian Trimpe
Date: 2018-01-31 00:00:00
Proceeding: ICML 2018 7
Tasks: Gaussian Processes, State Space Models, Time Series, Time Series Analysis, Variational Inference

State-space models (SSMs) are a highly expressive model class for learning
patterns in time series data and for system identification. Deterministic
versions of SSMs (e.g. LSTMs) proved extremely successful in modeling complex
time series data. Fully probabilistic SSMs, however, are often found hard to
train, even for smaller problems. To overcome this limitation, we propose a
novel model formulation and a scalable training algorithm based on doubly
stochastic variational inference and Gaussian processes. In contrast to
existing work, the proposed variational approximation allows one to fully
capture the latent state temporal correlations. These correlations are the key
to robust training. The effectiveness of the proposed PR-SSM is evaluated on a
set of real-world benchmark datasets in comparison to state-of-the-art
probabilistic model learning methods. Scalability and robustness are
demonstrated on a high dimensional problem.
MARGIN: Uncovering Deep Neural Networks using Graph Signal Analysis
Authors: Rushil Anirudh, Jayaraman J. Thiagarajan, Rahul Sridhar, Peer-Timo Bremer
Date: 2017-11-15 00:00:00

Interpretability has emerged as a crucial aspect of building trust in machine learning systems, aimed at providing insights into the working of complex neural networks that are otherwise opaque to a user. There are a plethora of existing solutions addressing various aspects of interpretability ranging from identifying prototypical samples in a dataset to explaining image predictions or explaining mis-classifications. While all of these diverse techniques address seemingly different aspects of interpretability, we hypothesize that a large family of interepretability tasks are variants of the same central problem which is identifying \emph{relative} change in a model's prediction. This paper introduces MARGIN, a simple yet general approach to address a large set of interpretability tasks MARGIN exploits ideas rooted in graph signal analysis to determine influential nodes in a graph, which are defined as those nodes that maximally describe a function defined on the graph. By carefully defining task-specific graphs and functions, we demonstrate that MARGIN outperforms existing approaches in a number of disparate interpretability challenges.
MOEA/D with Angle-based Constrained Dominance Principle for Constrained Multi-objective Optimization Problems
Authors: Zhun Fan, Yi Fang, Wenji Li, Xinye Cai, Caimin Wei, Erik Goodman
Date: 2018-02-10 00:00:00
Tasks: Diversity

This paper proposes a novel constraint-handling mechanism named angle-based
constrained dominance principle (ACDP) embedded in a decomposition-based
multi-objective evolutionary algorithm (MOEA/D) to solve constrained
multi-objective optimization problems (CMOPs). To maintain the diversity of the
working population, ACDP utilizes the information of the angle of solutions to
adjust the dominance relation of solutions during the evolutionary process.
This paper uses 14 benchmark instances to evaluate the performance of the
MOEA/D with ACDP (MOEA/D-ACDP). Additionally, an engineering optimization
problem (which is I-beam optimization problem) is optimized. The proposed
MOEA/D-ACDP, and four other decomposition-based CMOEAs, including C-MOEA/D,
MOEA/D-CDP, MOEA/D-Epsilon and MOEA/D-SR are tested by the above benchmarks and
the engineering application. The experimental results manifest that MOEA/D-ACDP
is significantly better than the other four CMOEAs on these test instances and
the real-world case, which indicates that ACDP is more effective for solving
CMOPs.
Distributed NLP
Authors: Galip Aydin, Ibrahim Riza Hallac
Date: 2018-02-10 00:00:00

In this paper we present the performance of parallel text processing with Map
Reduce on a cloud platform. Scientific papers in Turkish language are processed
using Zemberek NLP library. Experiments were run on a Hadoop cluster and
compared with the single machines performance.
Online Learning via the Differential Privacy Lens
Authors: Jacob Abernethy, Young Hun Jung, Chansoo Lee, Audra McMillan, Ambuj Tewari
Date: 2017-11-27 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Multi-Armed Bandits

In this paper, we use differential privacy as a lens to examine online learning in both full and partial information settings. The differential privacy framework is, at heart, less about privacy and more about algorithmic stability, and thus has found application in domains well beyond those where information security is central. Here we develop an algorithmic property called one-step differential stability which facilitates a more refined regret analysis for online learning methods. We show that tools from the differential privacy literature can yield regret bounds for many interesting online learning problems including online convex optimization and online linear optimization. Our stability notion is particularly well-suited for deriving first-order regret bounds for follow-the-perturbed-leader algorithms, something that all previous analyses have struggled to achieve. We also generalize the standard max-divergence to obtain a broader class called Tsallis max-divergences. These define stronger notions of stability that are useful in deriving bounds in partial information settings such as multi-armed bandits and bandits with experts.
Feature-Distributed SVRG for High-Dimensional Linear Classification
Authors: Gong-Duo Zhang, Shen-Yi Zhao, Hao Gao, Wu-Jun Li
Date: 2018-02-10 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification, Vocal Bursts Intensity Prediction

Linear classification has been widely used in many high-dimensional
applications like text classification. To perform linear classification for
large-scale tasks, we often need to design distributed learning methods on a
cluster of multiple machines. In this paper, we propose a new distributed
learning method, called feature-distributed stochastic variance reduced
gradient (FD-SVRG) for high-dimensional linear classification. Unlike most
existing distributed learning methods which are instance-distributed, FD-SVRG
is feature-distributed. FD-SVRG has lower communication cost than other
instance-distributed methods when the data dimensionality is larger than the
number of data instances. Experimental results on real data demonstrate that
FD-SVRG can outperform other state-of-the-art distributed methods for
high-dimensional linear classification in terms of both communication cost and
wall-clock time, when the dimensionality is larger than the number of instances
in training data.
Deep Meta-Learning: Learning to Learn in the Concept Space
Authors: Fengwei Zhou, Bin Wu, Zhenguo Li
Date: 2018-02-10 00:00:00
Tasks: Few-Shot Learning, Meta-Learning

Few-shot learning remains challenging for meta-learning that learns a
learning algorithm (meta-learner) from many related tasks. In this work, we
argue that this is due to the lack of a good representation for meta-learning,
and propose deep meta-learning to integrate the representation power of deep
learning into meta-learning. The framework is composed of three modules, a
concept generator, a meta-learner, and a concept discriminator, which are
learned jointly. The concept generator, e.g. a deep residual net, extracts a
representation for each instance that captures its high-level concept, on which
the meta-learner performs few-shot learning, and the concept discriminator
recognizes the concepts. By learning to learn in the concept space rather than
in the complicated instance space, deep meta-learning can substantially improve
vanilla meta-learning, which is demonstrated on various few-shot image
recognition problems. For example, on 5-way-1-shot image recognition on
CIFAR-100 and CUB-200, it improves Matching Nets from 50.53% and 56.53% to
58.18% and 63.47%, improves MAML from 49.28% and 50.45% to 56.65% and 64.63%,
and improves Meta-SGD from 53.83% and 53.34% to 61.62% and 66.95%,
respectively.
Online Learning for Effort Reduction in Interactive Neural Machine Translation
Authors: Álvaro Peris, Francisco Casacuberta
Date: 2018-02-10 00:00:00
Tasks: Machine Translation, Translation

Neural machine translation systems require large amounts of training data and
resources. Even with this, the quality of the translations may be insufficient
for some users or domains. In such cases, the output of the system must be
revised by a human agent. This can be done in a post-editing stage or following
an interactive machine translation protocol.
  We explore the incremental update of neural machine translation systems
during the post-editing or interactive translation processes. Such
modifications aim to incorporate the new knowledge, from the edited sentences,
into the translation system. Updates to the model are performed on-the-fly, as
sentences are corrected, via online learning techniques. In addition, we
implement a novel interactive, adaptive system, able to react to
single-character interactions. This system greatly reduces the human effort
required for obtaining high-quality translations.
  In order to stress our proposals, we conduct exhaustive experiments varying
the amount and type of data available for training. Results show that online
learning effectively achieves the objective of reducing the human effort
required during the post-editing or the interactive machine translation stages.
Moreover, these adaptive systems also perform well in scenarios with scarce
resources. We show that a neural machine translation system can be rapidly
adapted to a specific domain, exclusively by means of online learning
techniques.
Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction
Authors: Botong Wu, Zhen Zhou, Jianwei Wang, Yizhou Wang
Date: 2018-02-10 00:00:00
Tasks: Attribute, Computed Tomography (CT), General Classification, Lung Nodule Classification, Lung Nodule Segmentation, Multi-Task Learning

Refer to the literature of lung nodule classification, many studies adopt
Convolutional Neural Networks (CNN) to directly predict the malignancy of lung
nodules with original thoracic Computed Tomography (CT) and nodule location.
However, these studies cannot tell how the CNN works in terms of predicting the
malignancy of the given nodule, e.g., it's hard to conclude that whether the
region within the nodule or the contextual information matters according to the
output of the CNN. In this paper, we propose an interpretable and multi-task
learning CNN -- Joint learning for \textbf{P}ulmonary \textbf{N}odule
\textbf{S}egmentation \textbf{A}ttributes and \textbf{M}alignancy
\textbf{P}rediction (PN-SAMP). It is able to not only accurately predict the
malignancy of lung nodules, but also provide semantic high-level attributes as
well as the areas of detected nodules. Moreover, the combination of nodule
segmentation, attributes and malignancy prediction is helpful to improve the
performance of each single task. In addition, inspired by the fact that
radiologists often change window widths and window centers to help to make
decision on uncertain nodules, PN-SAMP mixes multiple WW/WC together to gain
information for the raw CT input images. To verify the effectiveness of the
proposed method, the evaluation is implemented on the public LIDC-IDRI dataset,
which is one of the largest dataset for lung nodule malignancy prediction.
Experiments indicate that the proposed PN-SAMP achieves significant improvement
with respect to lung nodule classification, and promising performance on lung
nodule segmentation and attribute learning, compared with the-state-of-the-art
methods.
Distributed One-class Learning
Authors: Ali Shahin Shamsabadi, Hamed Haddadi, Andrea Cavallaro
Date: 2018-02-10 00:00:00
Tasks: Blocking, One-class classifier

We propose a cloud-based filter trained to block third parties from uploading
privacy-sensitive images of others to online social media. The proposed filter
uses Distributed One-Class Learning, which decomposes the cloud-based filter
into multiple one-class classifiers. Each one-class classifier captures the
properties of a class of privacy-sensitive images with an autoencoder. The
multi-class filter is then reconstructed by combining the parameters of the
one-class autoencoders. The training takes place on edge devices (e.g.
smartphones) and therefore users do not need to upload their private and/or
sensitive images to the cloud. A major advantage of the proposed filter over
existing distributed learning approaches is that users cannot access, even
indirectly, the parameters of other users. Moreover, the filter can cope with
the imbalanced and complex distribution of the image content and the
independent probability of addition of new users. We evaluate the performance
of the proposed distributed filter using the exemplar task of blocking a user
from sharing privacy-sensitive images of other users. In particular, we
validate the behavior of the proposed multi-class filter with
non-privacy-sensitive images, the accuracy when the number of classes
increases, and the robustness to attacks when an adversary user has access to
privacy-sensitive images of other users.
2-gram-based Phonetic Feature Generation for Convolutional Neural Network in Assessment of Trademark Similarity
Authors: Kyung Pyo Ko, Kwang Hee Lee, Mi So Jang, Gun Hong Park
Date: 2018-02-10 00:00:00

A trademark is a mark used to identify various commodities. If same or
similar trademark is registered for the same or similar commodity, the
purchaser of the goods may be confused. Therefore, in the process of trademark
registration examination, the examiner judges whether the trademark is the same
or similar to the other applied or registered trademarks. The confusion in
trademarks is based on the visual, phonetic or conceptual similarity of the
marks. In this paper, we focus specifically on the phonetic similarity between
trademarks. We propose a method to generate 2D phonetic feature for
convolutional neural network in assessment of trademark similarity. This
proposed algorithm is tested with 12,553 trademark phonetic similar pairs and
34,020 trademark phonetic non-similar pairs from 2010 to 2016. As a result, we
have obtained approximately 92% judgment accuracy.
Tips, guidelines and tools for managing multi-label datasets: the mldr.datasets R package and the Cometa data repository
Authors: Francisco Charte, Antonio J. Rivera, David Charte, María J. del Jesus, Francisco Herrera
Date: 2018-02-10 00:00:00
Tasks: Management, Multi-Label Learning

New proposals in the field of multi-label learning algorithms have been
growing in number steadily over the last few years. The experimentation
associated with each of them always goes through the same phases: selection of
datasets, partitioning, training, analysis of results and, finally, comparison
with existing methods. This last step is often hampered since it involves using
exactly the same datasets, partitioned in the same way and using the same
validation strategy. In this paper we present a set of tools whose objective is
to facilitate the management of multi-label datasets, aiming to standardize the
experimentation procedure. The two main tools are an R package, mldr.datasets,
and a web repository with datasets, Cometa. Together, these tools will simplify
the collection of datasets, their partitioning, documentation and export to
multiple formats, among other functions. Some tips, recommendations and
guidelines for a good experimental analysis of multi-label methods are also
presented.
The Importance of Norm Regularization in Linear Graph Embedding: Theoretical Analysis and Empirical Demonstration
Authors: Yihan Gao, Chao Zhang, Jian Peng, Aditya Parameswaran
Date: 2018-02-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Graph Embedding

Learning distributed representations for nodes in graphs is a crucial
primitive in network analysis with a wide spectrum of applications. Linear
graph embedding methods learn such representations by optimizing the likelihood
of both positive and negative edges while constraining the dimension of the
embedding vectors. We argue that the generalization performance of these
methods is not due to the dimensionality constraint as commonly believed, but
rather the small norm of embedding vectors. Both theoretical and empirical
evidence are provided to support this argument: (a) we prove that the
generalization error of these methods can be bounded by limiting the norm of
vectors, regardless of the embedding dimension; (b) we show that the
generalization performance of linear graph embedding methods is correlated with
the norm of embedding vectors, which is small due to the early stopping of SGD
and the vanishing gradients. We performed extensive experiments to validate our
analysis and showcased the importance of proper norm regularization in
practice.
Good Arm Identification via Bandit Feedback
Authors: Hideaki Kano, Junya Honda, Kentaro Sakamaki, Kentaro Matsuura, Atsuyoshi Nakamura, Masashi Sugiyama
Date: 2017-10-17 00:00:00

We consider a novel stochastic multi-armed bandit problem called {\em good
arm identification} (GAI), where a good arm is defined as an arm with expected
reward greater than or equal to a given threshold. GAI is a pure-exploration
problem that a single agent repeats a process of outputting an arm as soon as
it is identified as a good one before confirming the other arms are actually
not good. The objective of GAI is to minimize the number of samples for each
process. We find that GAI faces a new kind of dilemma, the {\em
exploration-exploitation dilemma of confidence}, which is different difficulty
from the best arm identification. As a result, an efficient design of
algorithms for GAI is quite different from that for the best arm
identification. We derive a lower bound on the sample complexity of GAI that is
tight up to the logarithmic factor $\mathrm{O}(\log \frac{1}{\delta})$ for
acceptance error rate $\delta$. We also develop an algorithm whose sample
complexity almost matches the lower bound. We also confirm experimentally that
our proposed algorithm outperforms naive algorithms in synthetic settings based
on a conventional bandit problem and clinical trial researches for rheumatoid
arthritis.
Tubule segmentation of fluorescence microscopy images based on convolutional neural networks with inhomogeneity correction
Authors: Soonam Lee, Chichen Fu, Paul Salama, Kenneth W. Dunn, Edward J. Delp
Date: 2018-02-10 00:00:00
Tasks: Data Augmentation, Segmentation

Fluorescence microscopy has become a widely used tool for studying various
biological structures of in vivo tissue or cells. However, quantitative
analysis of these biological structures remains a challenge due to their
complexity which is exacerbated by distortions caused by lens aberrations and
light scattering. Moreover, manual quantification of such image volumes is an
intractable and error-prone process, making the need for automated image
analysis methods crucial. This paper describes a segmentation method for
tubular structures in fluorescence microscopy images using convolutional neural
networks with data augmentation and inhomogeneity correction. The segmentation
results of the proposed method are visually and numerically compared with other
microscopy segmentation methods. Experimental results indicate that the
proposed method has better performance with correctly segmenting and
identifying multiple tubular structures compared to other methods.
Collaborative Learning for Weakly Supervised Object Detection
Authors: Jiajie Wang, Jiangchao Yao, Ya zhang, Rui Zhang
Date: 2018-02-10 00:00:00
Tasks: Object, object-detection, Object Detection, Weakly Supervised Object Detection

Weakly supervised object detection has recently received much attention,
since it only requires image-level labels instead of the bounding-box labels
consumed in strongly supervised learning. Nevertheless, the save in labeling
expense is usually at the cost of model accuracy. In this paper, we propose a
simple but effective weakly supervised collaborative learning framework to
resolve this problem, which trains a weakly supervised learner and a strongly
supervised learner jointly by enforcing partial feature sharing and prediction
consistency. For object detection, taking WSDDN-like architecture as weakly
supervised detector sub-network and Faster-RCNN-like architecture as strongly
supervised detector sub-network, we propose an end-to-end Weakly Supervised
Collaborative Detection Network. As there is no strong supervision available to
train the Faster-RCNN-like sub-network, a new prediction consistency loss is
defined to enforce consistency of predictions between the two sub-networks as
well as within the Faster-RCNN-like sub-networks. At the same time, the two
detectors are designed to partially share features to further guarantee the
model consistency at perceptual level. Extensive experiments on PASCAL VOC 2007
and 2012 data sets have demonstrated the effectiveness of the proposed
framework.
LEARN: Learned Experts' Assessment-based Reconstruction Network for Sparse-data CT
Authors: Hu Chen, Yi Zhang, Yunjin Chen, Junfeng Zhang, Weihua Zhang, Huaiqiaing Sun, Yang Lv, Peixi Liao, Jiliu Zhou, Ge Wang
Date: 2017-07-30 00:00:00
Tasks: Compressive Sensing

Compressive sensing (CS) has proved effective for tomographic reconstruction
from sparsely collected data or under-sampled measurements, which are
practically important for few-view CT, tomosynthesis, interior tomography, and
so on. To perform sparse-data CT, the iterative reconstruction commonly use
regularizers in the CS framework. Currently, how to choose the parameters
adaptively for regularization is a major open problem. In this paper, inspired
by the idea of machine learning especially deep learning, we unfold a
state-of-the-art "fields of experts" based iterative reconstruction scheme up
to a number of iterations for data-driven training, construct a Learned
Experts' Assessment-based Reconstruction Network ("LEARN") for sparse-data CT,
and demonstrate the feasibility and merits of our LEARN network. The
experimental results with our proposed LEARN network produces a competitive
performance with the well-known Mayo Clinic Low-Dose Challenge Dataset relative
to several state-of-the-art methods, in terms of artifact reduction, feature
preservation, and computational speed. This is consistent to our insight that
because all the regularization terms and parameters used in the iterative
reconstruction are now learned from the training data, our LEARN network
utilizes application-oriented knowledge more effectively and recovers
underlying images more favorably than competing algorithms. Also, the number of
layers in the LEARN network is only 12, reducing the computational complexity
of typical iterative algorithms by orders of magnitude.
Coverless information hiding based on Generative Model
Authors: Xintao Duan, Haoxian Song
Date: 2018-02-10 00:00:00
Tasks: Image Steganography, model, Steganalysis

A new coverless image information hiding method based on generative model is
proposed, we feed the secret image to the generative model database, and
generate a meaning-normal and independent image different from the secret
image, then, the generated image is transmitted to the receiver and is fed to
the generative model database to generate another image visually the same as
the secret image. So we only need to transmit the meaning-normal image which is
not related to the secret image, and we can achieve the same effect as the
transmission of the secret image. This is the first time to propose the
coverless image information hiding method based on generative model, compared
with the traditional image steganography, the transmitted image does not embed
any information of the secret image in this method, therefore, can effectively
resist steganalysis tools. Experimental results show that our method has high
capacity, safety and reliability.