pwc-archive/papers-with-abstracts huggingface dataset:
jMetalPy: a Python Framework for Multi-Objective Optimization with Metaheuristics
Authors: Antonio Benitez-Hidalgo, Antonio J. Nebro, Jose Garcia-Nieto, Izaskun Oregi, Javier Del Ser
Date: 2019-03-07 00:00:00
Tasks: Data Visualization

This paper describes jMetalPy, an object-oriented Python-based framework for
multi-objective optimization with metaheuristic techniques. Building upon our
experiences with the well-known jMetal framework, we have developed a new
multi-objective optimization software platform aiming not only at replicating
the former one in a different programming language, but also at taking
advantage of the full feature set of Python, including its facilities for fast
prototyping and the large amount of available libraries for data processing,
data analysis, data visualization, and high-performance computing. As a result,
jMetalPy provides an environment for solving multi-objective optimization
problems focused not only on traditional metaheuristics, but also on techniques
supporting preference articulation and dynamic problems, along with a rich set
of features related to the automatic generation of statistical data from the
results generated, as well as the real-time and interactive visualization of
the Pareto front approximations produced by the algorithms. jMetalPy offers
additionally support for parallel computing in multicore and cluster systems.
We include some use cases to explore the main features of jMetalPy and to
illustrate how to work with it.
Only sparsity based loss function for learning representations
Authors: Vivek Bakaraju, Kishore Reddy Konda
Date: 2019-03-07 00:00:00

We study the emergence of sparse representations in neural networks. We show
that in unsupervised models with regularization, the emergence of sparsity is
the result of the input data samples being distributed along highly non-linear
or discontinuous manifold. We also derive a similar argument for
discriminatively trained networks and present experiments to support this
hypothesis. Based on our study of sparsity, we introduce a new loss function
which can be used as regularization term for models like autoencoders and MLPs.
Further, the same loss function can also be used as a cost function for an
unsupervised single-layered neural network model for learning efficient
representations.
Robust and Communication-Efficient Federated Learning from Non-IID Data
Authors: Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, Wojciech Samek
Date: 2019-03-07 00:00:00
Tasks: Federated Learning, Privacy Preserving

Federated Learning allows multiple parties to jointly train a deep learning
model on their combined data, without any of the participants having to reveal
their local data to a centralized server. This form of privacy-preserving
collaborative learning however comes at the cost of a significant communication
overhead during training. To address this problem, several compression methods
have been proposed in the distributed training literature that can reduce the
amount of required communication by up to three orders of magnitude. These
existing methods however are only of limited utility in the Federated Learning
setting, as they either only compress the upstream communication from the
clients to the server (leaving the downstream communication uncompressed) or
only perform well under idealized conditions such as iid distribution of the
client data, which typically can not be found in Federated Learning. In this
work, we propose Sparse Ternary Compression (STC), a new compression framework
that is specifically designed to meet the requirements of the Federated
Learning environment. Our experiments on four different learning tasks
demonstrate that STC distinctively outperforms Federated Averaging in common
Federated Learning scenarios where clients either a) hold non-iid data, b) use
small batch sizes during training, or where c) the number of clients is large
and the participation rate in every communication round is low. We furthermore
show that even if the clients hold iid data and use medium sized batches for
training, STC still behaves pareto-superior to Federated Averaging in the sense
that it achieves fixed target accuracies on our benchmarks within both fewer
training iterations and a smaller communication budget.
Deep Learning for UL/DL Channel Calibration in Generic Massive MIMO Systems
Authors: Chongwen Huang, George C. Alexandropoulos, Alessio Zappone, Chau Yuen, Mérouane Debbah
Date: 2019-03-07 00:00:00

One of the fundamental challenges to realize massive Multiple-Input Multiple-Output (MIMO) communications is the accurate acquisition of channel state information for a plurality of users at the base station. This is usually accomplished in the UpLink (UL) direction profiting from the time division duplexing mode. In practical base station transceivers, there exist inevitably nonlinear hardware components, like signal amplifiers and various analog filters, which complicates the calibration task. To deal with this challenge, we design a deep neural network for channel calibration between the UL and DownLink (DL) directions. During the initial training phase, the deep neural network is trained from both UL and DL channel measurements. We then leverage the trained deep neural network with the instantaneously estimated UL channel to calibrate the DL one, which is not observable during the UL transmission phase. Our numerical results confirm the merits of the proposed approach, and show that it can achieve performance comparable to conventional approaches, like the Agros method and methods based on least squares, that however assume linear hardware behavior models. More importantly, considering generic nonlinear relationships between the UL and DL channels, it is demonstrated that our deep neural network approach exhibits robust performance, even when the number of training sequences is limited.
Exploit fully automatic low-level segmented PET data for training high-level deep learning algorithms for the corresponding CT data
Authors: Christina Gsaxner, Peter M. Roth, Jürgen Wallner, Jan Egger
Date: 2019-03-07 00:00:00
Tasks: Bladder Segmentation, Data Augmentation, Image Segmentation, Medical Image Analysis, Medical Image Segmentation, Segmentation, Semantic Segmentation

We present an approach for fully automatic urinary bladder segmentation in CT
images with artificial neural networks in this study. Automatic medical image
analysis has become an invaluable tool in the different treatment stages of
diseases. Especially medical image segmentation plays a vital role, since
segmentation is often the initial step in an image analysis pipeline. Since
deep neural networks have made a large impact on the field of image processing
in the past years, we use two different deep learning architectures to segment
the urinary bladder. Both of these architectures are based on pre-trained
classification networks that are adapted to perform semantic segmentation.
Since deep neural networks require a large amount of training data,
specifically images and corresponding ground truth labels, we furthermore
propose a method to generate such a suitable training data set from Positron
Emission Tomography/Computed Tomography image data. This is done by applying
thresholding to the Positron Emission Tomography data for obtaining a ground
truth and by utilizing data augmentation to enlarge the dataset. In this study,
we discuss the influence of data augmentation on the segmentation results, and
compare and evaluate the proposed architectures in terms of qualitative and
quantitative segmentation performance. The results presented in this study
allow concluding that deep neural networks can be considered a promising
approach to segment the urinary bladder in CT images.
Convergence of Multi-Agent Learning with a Finite Step Size in General-Sum Games
Authors: Xinliang Song, Tonghan Wang, Chongjie Zhang
Date: 2019-03-07 00:00:00

Learning in a multi-agent system is challenging because agents are
simultaneously learning and the environment is not stationary, undermining
convergence guarantees. To address this challenge, this paper presents a new
gradient-based learning algorithm, called Gradient Ascent with Shrinking Policy
Prediction (GA-SPP), which augments the basic gradient ascent approach with the
concept of shrinking policy prediction. The key idea behind this algorithm is
that an agent adjusts its strategy in response to the forecasted strategy of
the other agent, instead of its current one. GA-SPP is shown formally to have
Nash convergence in larger settings than existing gradient-based multi-agent
learning methods. Furthermore, unlike existing gradient-based methods, GA-SPP's
theoretical guarantees do not assume the learning rate to be infinitesimal.
Small-world networks for summarization of biomedical articles
Authors: Milad Moradi
Date: 2019-03-07 00:00:00
Tasks: Articles, Informativeness, Sentence, Text Summarization

In recent years, many methods have been developed to identify important
portions of text documents. Summarization tools can utilize these methods to
extract summaries from large volumes of textual information. However, to
identify concepts representing central ideas within a text document and to
extract the most informative sentences that best convey those concepts still
remain two crucial tasks in summarization methods. In this paper, we introduce
a graph-based method to address these two challenges in the context of
biomedical text summarization. We show that how a summarizer can discover
meaningful concepts within a biomedical text document using the Helmholtz
principle. The summarizer considers the meaningful concepts as the main topics
and constructs a graph based on the topics that the sentences share. The
summarizer can produce an informative summary by extracting those sentences
having higher values of the degree. We assess the performance of our method for
summarization of biomedical articles using the Recall-Oriented Understudy for
Gisting Evaluation (ROUGE) toolkit. The results show that the degree can be a
useful centrality measure to identify important sentences in this type of
graph-based modelling. Our method can improve the performance of biomedical
text summarization compared to some state-of-the-art and publicly available
summarizers. Combining a concept-based modelling strategy and a graph-based
approach to sentence extraction, our summarizer can produce summaries with the
highest scores of informativeness among the comparison methods. This research
work can be regarded as a start point to the study of small-world networks in
summarization of biomedical texts.
Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models
Authors: Thomas Drugman, Janne Pylkkonen, Reinhard Kneser
Date: 2019-03-07 00:00:00
Tasks: Active Learning, speech-recognition, Speech Recognition

The goal of this paper is to simulate the benefits of jointly applying active
learning (AL) and semi-supervised training (SST) in a new speech recognition
application. Our data selection approach relies on confidence filtering, and
its impact on both the acoustic and language models (AM and LM) is studied.
While AL is known to be beneficial to AM training, we show that it also carries
out substantial improvements to the LM when combined with SST. Sophisticated
confidence models, on the other hand, did not prove to yield any data selection
gain. Our results indicate that, while SST is crucial at the beginning of the
labeling process, its gains degrade rapidly as AL is set in place. The final
simulation reports that AL allows a transcription cost reduction of about 70%
over random selection. Alternatively, for a fixed transcription budget, the
proposed approach improves the word error rate by about 12.5% relative.
Machine learning method for single trajectory characterization
Authors: Gorka Muñoz-Gil, Miguel Angel Garcia-March, Carlo Manzo, José D. Martín-Guerrero, Maciej Lewenstein
Date: 2019-03-07 00:00:00
Tasks: BIG-bench Machine Learning, Transfer Learning

In order to study transport in complex environments, it is extremely important to determine the physical mechanism underlying diffusion, and precisely characterize its nature and parameters. Often, this task is strongly impacted by data consisting of trajectories with short length and limited localization precision. In this paper, we propose a machine learning method based on a random forest architecture, which is able to associate even very short trajectories to the underlying diffusion mechanism with a high accuracy. In addition, the method is able to classify the motion according to normal or anomalous diffusion, and determine its anomalous exponent with a small error. The method provides highly accurate outputs even when working with very short trajectories and in the presence of experimental noise. We further demonstrate the application of transfer learning to experimental and simulated data not included in the training/testing dataset. This allows for a full, high-accuracy characterization of experimental trajectories without the need of any prior information.
Quantum hardness of learning shallow classical circuits
Authors: Srinivasan Arunachalam, Alex B. Grilo, Aarthi Sundaram
Date: 2019-03-07 00:00:00
Tasks: PAC learning

In this paper we study the quantum learnability of constant-depth classical circuits under the uniform distribution and in the distribution-independent framework of PAC learning. In order to attain our results, we establish connections between quantum learning and quantum-secure cryptosystems. We then achieve the following results. 1) Hardness of learning AC$^0$ and TC$^0$ under the uniform distribution. Our first result concerns the concept class TC$^0$ (resp. AC$^0$), the class of constant-depth and polynomial-sized circuits with unbounded fan-in majority gates (resp. AND, OR, NOT gates). We show that if there exists no quantum polynomial-time (resp. strong sub-exponential time) algorithm to solve the Ring Learning with Errors (RLWE) problem, then there exists no polynomial-time quantum learning algorithm for TC$^0$ (resp. AC$^0$) under the uniform distribution (even with access to quantum membership queries). The main technique in this result uses explicit pseudo-random functions that are believed to be quantum-secure to construct concept classes that are hard to learn quantumly under the uniform distribution. 2) Hardness of learning TC$^0_2$ in the PAC setting. Our second result shows that if there exists no quantum polynomial time algorithm for the LWE problem, then there exists no polynomial time quantum PAC learning algorithm for the class TC$^0_2$, i.e., depth-2 TC$^0$ circuits. The main technique in this result is to establish a connection between the quantum security of public-key cryptosystems and the learnability of a concept class that consists of decryption functions of the cryptosystem. This gives a strong (conditional) negative answer to one of the "Ten Semi-Grand Challenges for Quantum Computing Theory" raised by Aaronson [Aar05], who asked if AC$^0$ and TC$^0$ can be PAC-learned in quantum polynomial time.
The Privacy Blanket of the Shuffle Model
Authors: Borja Balle, James Bell, Adria Gascon, Kobbi Nissim
Date: 2019-03-07 00:00:00
Tasks: model

This work studies differential privacy in the context of the recently proposed shuffle model. Unlike in the local model, where the server collecting privatized data from users can track back an input to a specific user, in the shuffle model users submit their privatized inputs to a server anonymously. This setup yields a trust model which sits in between the classical curator and local models for differential privacy. The shuffle model is the core idea in the Encode, Shuffle, Analyze (ESA) model introduced by Bittau et al. (SOPS 2017). Recent work by Cheu et al. (EUROCRYPT 2019) analyzes the differential privacy properties of the shuffle model and shows that in some cases shuffled protocols provide strictly better accuracy than local protocols. Additionally, Erlingsson et al. (SODA 2019) provide a privacy amplification bound quantifying the level of curator differential privacy achieved by the shuffle model in terms of the local differential privacy of the randomizer used by each user. In this context, we make three contributions. First, we provide an optimal single message protocol for summation of real numbers in the shuffle model. Our protocol is very simple and has better accuracy and communication than the protocols for this same problem proposed by Cheu et al. Optimality of this protocol follows from our second contribution, a new lower bound for the accuracy of private protocols for summation of real numbers in the shuffle model. The third contribution is a new amplification bound for analyzing the privacy of protocols in the shuffle model in terms of the privacy provided by the corresponding local randomizer. Our amplification bound generalizes the results by Erlingsson et al. to a wider range of parameters, and provides a whole family of methods to analyze privacy amplification in the shuffle model.
Active Scene Learning
Authors: Erelcan Yanik, Tevfik Metin Sezgin
Date: 2019-03-07 00:00:00
Tasks: Active Learning, Informativeness, Sketch Recognition

Sketch recognition allows natural and efficient interaction in pen-based
interfaces. A key obstacle to building accurate sketch recognizers has been the
difficulty of creating large amounts of annotated training data. Several
authors have attempted to address this issue by creating synthetic data, and by
building tools that support efficient annotation. Two prominent sets of
approaches stand out from the rest of the crowd. They use interim classifiers
trained with a small set of labeled data to aid the labeling of the remainder
of the data. The first set of approaches uses a classifier trained with a
partially labeled dataset to automatically label unlabeled instances. The
others, based on active learning, save annotation effort by giving priority to
labeling informative data instances. The former is sub-optimal since it doesn't
prioritize the order of labeling to favor informative instances, while the
latter makes the strong assumption that unlabeled data comes in an already
segmented form (i.e. the ink in the training data is already assembled into
groups forming isolated object instances). In this paper, we propose an active
learning framework that combines the strengths of these methods, while
addressing their weaknesses. In particular, we propose two methods for deciding
how batches of unsegmented sketch scenes should be labeled. The first method,
scene-wise selection, assesses the informativeness of each drawing (sketch
scene) as a whole, and asks the user to annotate all objects in the drawing.
The latter, segment-wise selection, attempts more precise targeting to locate
informative fragments of drawings for user labeling. We show that both
selection schemes outperform random selection. Furthermore, we demonstrate that
precise targeting yields superior performance. Overall, our approach allows
reaching top accuracy figures with up to 30% savings in annotation cost.
Predicting Research Trends From Arxiv
Authors: Steffen Eger, Chao Li, Florian Netzer, Iryna Gurevych
Date: 2019-03-07 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

We perform trend detection on two datasets of Arxiv papers, derived from its
machine learning (cs.LG) and natural language processing (cs.CL) categories.
Our approach is bottom-up: we first rank papers by their normalized citation
counts, then group top-ranked papers into different categories based on the
tasks that they pursue and the methods they use. We then analyze these
resulting topics. We find that the dominating paradigm in cs.CL revolves around
natural language generation problems and those in cs.LG revolve around
reinforcement learning and adversarial principles. By extrapolation, we predict
that these topics will remain lead problems/approaches in their fields in the
short- and mid-term.
Limiting Network Size within Finite Bounds for Optimization
Authors: Linu Pinto, Dr. Sasi Gopalan
Date: 2019-03-07 00:00:00
Tasks: Attribute, Binary Classification, General Classification

Largest theoretical contribution to Neural Networks comes from VC Dimension
which characterizes the sample complexity of classification model in a
probabilistic view and are widely used to study the generalization error. So
far in the literature the VC Dimension has only been used to approximate the
generalization error bounds on different Neural Network architectures. VC
Dimension has not yet been implicitly or explicitly stated to fix the network
size which is important as the wrong configuration could lead to high
computation effort in training and leads to over fitting. So there is a need to
bound these units so that task can be computed with only sufficient number of
parameters. For binary classification tasks shallow networks are used as they
have universal approximation property and it is enough to size the hidden layer
width for such networks. The paper brings out a theoretical justification on
required attribute size and its corresponding hidden layer dimension for a
given sample set that gives an optimal binary classification results with
minimum training complexity in a single layered feed forward network framework.
The paper also establishes proof on the existence of bounds on the width of the
hidden layer and its range subjected to certain conditions. Findings in this
paper are experimentally analyzed on three different dataset using Mathlab 2018
(b) software.
Multi-output Bus Travel Time Prediction with Convolutional LSTM Neural Network
Authors: Niklas Christoffer Petersen, Filipe Rodrigues, Francisco Camara Pereira
Date: 2019-03-07 00:00:00

Accurate and reliable travel time predictions in public transport networks
are essential for delivering an attractive service that is able to compete with
other modes of transport in urban areas. The traditional application of this
information, where arrival and departure predictions are displayed on digital
boards, is highly visible in the city landscape of most modern metropolises.
More recently, the same information has become critical as input for
smart-phone trip planners in order to alert passengers about unreachable
connections, alternative route choices and prolonged travel times. More
sophisticated Intelligent Transport Systems (ITS) include the predictions of
connection assurance, i.e. to hold back services in case a connecting service
is delayed. In order to operate such systems, and to ensure the confidence of
passengers in the systems, the information provided must be accurate and
reliable. Traditional methods have trouble with this as congestion, and thus
travel time variability, increases in cities, consequently making travel time
predictions in urban areas a non-trivial task. This paper presents a system for
bus travel time prediction that leverages the non-static spatio-temporal
correlations present in urban bus networks, allowing the discovery of complex
patterns not captured by traditional methods. The underlying model is a
multi-output, multi-time-step, deep neural network that uses a combination of
convolutional and long short-term memory (LSTM) layers. The method is
empirically evaluated and compared to other popular approaches for link travel
time prediction and currently available services, including the currently
deployed model in Copenhagen, Denmark. We find that the proposed model
significantly outperforms all the other methods we compare with, and is able to
detect small irregular peaks in bus travel times very quickly.