pwc-archive/papers-with-abstracts huggingface dataset:
Low-Power Neuromorphic Hardware for Signal Processing Applications
Authors: Bipin Rajendran, Abu Sebastian, Michael Schmuker, Narayan Srinivasa, Evangelos Eleftheriou
Date: 2019-01-11 00:00:00
Tasks: BIG-bench Machine Learning, Reinforcement Learning

Machine learning has emerged as the dominant tool for implementing complex cognitive tasks that require supervised, unsupervised, and reinforcement learning. While the resulting machines have demonstrated in some cases even super-human performance, their energy consumption has often proved to be prohibitive in the absence of costly super-computers. Most state-of-the-art machine learning solutions are based on memory-less models of neurons. This is unlike the neurons in the human brain, which encode and process information using temporal information in spike events. The different computing principles underlying biological neurons and how they combine together to efficiently process information is believed to be a key factor behind their superior efficiency compared to current machine learning systems. Inspired by the time-encoding mechanism used by the brain, third generation spiking neural networks (SNNs) are being studied for building a new class of information processing engines. Modern computing systems based on the von Neumann architecture, however, are ill-suited for efficiently implementing SNNs, since their performance is limited by the need to constantly shuttle data between physically separated logic and memory units. Hence, novel computational architectures that address the von Neumann bottleneck are necessary in order to build systems that can implement SNNs with low energy budgets. In this paper, we review some of the architectural and system level design aspects involved in developing a new class of brain-inspired information processing engines that mimic the time-based information encoding and processing aspects of the brain.
Multi-Level Batch Normalization In Deep Networks For Invasive Ductal Carcinoma Cell Discrimination In Histopathology Images
Authors: Francisco Perdigon Romero, An Tang, Samuel Kadoury
Date: 2019-01-11 00:00:00

Breast cancer is the most diagnosed cancer and the most predominant cause of
death in women worldwide. Imaging techniques such as the breast cancer
pathology helps in the diagnosis and monitoring of the disease. However
identification of malignant cells can be challenging given the high
heterogeneity in tissue absorbotion from staining agents. In this work, we
present a novel approach for Invasive Ductal Carcinoma (IDC) cells
discrimination in histopathology slides. We propose a model derived from the
Inception architecture, proposing a multi-level batch normalization module
between each convolutional steps. This module was used as a base block for the
feature extraction in a CNN architecture. We used the open IDC dataset in which
we obtained a balanced accuracy of 0.89 and an F1 score of 0.90, thus
surpassing recent state of the art classification algorithms tested on this
public dataset.
Machine Learning Automation Toolbox (MLaut)
Authors: Viktor Kazakov, Franz J. Király
Date: 2019-01-11 00:00:00
Tasks: Benchmarking, BIG-bench Machine Learning

In this paper we present MLaut (Machine Learning AUtomation Toolbox) for the
python data science ecosystem. MLaut automates large-scale evaluation and
benchmarking of machine learning algorithms on a large number of datasets.
MLaut provides a high-level workflow interface to machine algorithm algorithms,
implements a local back-end to a database of dataset collections, trained
algorithms, and experimental results, and provides easy-to-use interfaces to
the scikit-learn and keras modelling libraries. Experiments are easy to set up
with default settings in a few lines of code, while remaining fully
customizable to the level of hyper-parameter tuning, pipeline composition, or
deep learning architecture.
  As a principal test case for MLaut, we conducted a large-scale supervised
classification study in order to benchmark the performance of a number of
machine learning algorithms - to our knowledge also the first larger-scale
study on standard supervised learning data sets to include deep learning
algorithms. While corroborating a number of previous findings in literature, we
found (within the limitations of our study) that deep neural networks do not
perform well on basic supervised learning, i.e., outside the more specialized,
image-, audio-, or text-based tasks.
On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator
Authors: Qi Cai, Mingyi Hong, Yongxin Chen, Zhaoran Wang
Date: 2019-01-11 00:00:00
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study the global convergence of generative adversarial imitation learning
for linear quadratic regulators, which is posed as minimax optimization. To
address the challenges arising from non-convex-concave geometry, we analyze the
alternating gradient algorithm and establish its Q-linear rate of convergence
to a unique saddle point, which simultaneously recovers the globally optimal
policy and reward function. We hope our results may serve as a small step
towards understanding and taming the instability in imitation learning as well
as in more general non-convex-concave alternating minimax optimization that
arises from reinforcement learning and generative adversarial learning.
ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation
Authors: J. Edward Hu, Rachel Rudinger, Matt Post, Benjamin Van Durme
Date: 2019-01-11 00:00:00
Tasks: Diversity, Machine Translation, NMT, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence ReWriting, Translation

We present ParaBank, a large-scale English paraphrase dataset that surpasses
prior work in both quantity and quality. Following the approach of ParaNMT, we
train a Czech-English neural machine translation (NMT) system to generate novel
paraphrases of English reference sentences. By adding lexical constraints to
the NMT decoding procedure, however, we are able to produce multiple
high-quality sentential paraphrases per source sentence, yielding an English
paraphrase resource with more than 4 billion generated tokens and exhibiting
greater lexical diversity. Using human judgments, we also demonstrate that
ParaBank's paraphrases improve over ParaNMT on both semantic similarity and
fluency. Finally, we use ParaBank to train a monolingual NMT model with the
same support for lexically-constrained decoding for sentence rewriting tasks.
A General Optimization-based Framework for Global Pose Estimation with Multiple Sensors
Authors: Tong Qin, Shaozu Cao, Jie Pan, Shaojie Shen
Date: 2019-01-11 00:00:00
Tasks: Pose Estimation, Sensor Fusion, State Estimation

Accurate state estimation is a fundamental problem for autonomous robots. To
achieve locally accurate and globally drift-free state estimation, multiple
sensors with complementary properties are usually fused together. Local sensors
(camera, IMU, LiDAR, etc) provide precise pose within a small region, while
global sensors (GPS, magnetometer, barometer, etc) supply noisy but globally
drift-free localization in a large-scale environment. In this paper, we propose
a sensor fusion framework to fuse local states with global sensors, which
achieves locally accurate and globally drift-free pose estimation. Local
estimations, produced by existing VO/VIO approaches, are fused with global
sensors in a pose graph optimization. Within the graph optimization, local
estimations are aligned into a global coordinate. Meanwhile, the accumulated
drifts are eliminated. We evaluate the performance of our system on public
datasets and with real-world experiments. Results are compared against other
state-of-the-art algorithms. We highlight that our system is a general
framework, which can easily fuse various global sensors in a unified pose graph
optimization. Our implementations are open
source\footnote{https://github.com/HKUST-Aerial-Robotics/VINS-Fusion}.
A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
Authors: Tong Qin, Jie Pan, Shaozu Cao, Shaojie Shen
Date: 2019-01-11 00:00:00
Tasks: State Estimation, Visual Odometry

Nowadays, more and more sensors are equipped on robots to increase robustness
and autonomous ability. We have seen various sensor suites equipped on
different platforms, such as stereo cameras on ground vehicles, a monocular
camera with an IMU (Inertial Measurement Unit) on mobile phones, and stereo
cameras with an IMU on aerial robots. Although many algorithms for state
estimation have been proposed in the past, they are usually applied to a single
sensor or a specific sensor suite. Few of them can be employed with multiple
sensor choices. In this paper, we proposed a general optimization-based
framework for odometry estimation, which supports multiple sensor sets. Every
sensor is treated as a general factor in our framework. Factors which share
common state variables are summed together to build the optimization problem.
We further demonstrate the generality with visual and inertial sensors, which
form three sensor suites (stereo cameras, a monocular camera with an IMU, and
stereo cameras with an IMU). We validate the performance of our system on
public datasets and through real-world experiments with multiple sensors.
Results are compared against other state-of-the-art algorithms. We highlight
that our system is a general framework, which can easily fuse various sensors
in a pose graph optimization. Our implementations are open
source\footnote{https://github.com/HKUST-Aerial-Robotics/VINS-Fusion}.
Variation Network: Learning High-level Attributes for Controlled Input Manipulation
Authors: Gaëtan Hadjeres, Frank Nielsen
Date: 2019-01-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: Vocal Bursts Intensity Prediction

This paper presents the Variation Network (VarNet), a generative model providing means to manipulate the high-level attributes of a given input. The originality of our approach is that VarNet is not only capable of handling pre-defined attributes but can also learn the relevant attributes of the dataset by itself. These two settings can also be easily considered at the same time, which makes this model applicable to a wide variety of tasks. Further, VarNet has a sound information-theoretic interpretation which grants us with interpretable means to control how these high-level attributes are learned. We demonstrate experimentally that this model is capable of performing interesting input manipulation and that the learned attributes are relevant and meaningful.
Image Disentanglement and Uncooperative Re-Entanglement for High-Fidelity Image-to-Image Translation
Authors: Adam W. Harley, Shih-En Wei, Jason Saragih, Katerina Fragkiadaki
Date: 2019-01-11 00:00:00
Tasks: Disentanglement, Image-to-Image Translation, Translation

Cross-domain image-to-image translation should satisfy two requirements: (1) preserve the information that is common to both domains, and (2) generate convincing images covering variations that appear in the target domain. This is challenging, especially when there are no example translations available as supervision. Adversarial cycle consistency was recently proposed as a solution, with beautiful and creative results, yielding much follow-up work. However, augmented reality applications cannot readily use such techniques to provide users with compelling translations of real scenes, because the translations do not have high-fidelity constraints. In other words, current models are liable to change details that should be preserved: while re-texturing a face, they may alter the face's expression in an unpredictable way. In this paper, we introduce the problem of high-fidelity image-to-image translation, and present a method for solving it. Our main insight is that low-fidelity translations typically escape a cycle-consistency penalty, because the back-translator learns to compensate for the forward-translator's errors. We therefore introduce an optimization technique that prevents the networks from cooperating: simply train each network only when its input data is real. Prior works, in comparison, train each network with a mix of real and generated data. Experimental results show that our method accurately disentangles the factors that separate the domains, and converges to semantics-preserving translations that prior methods miss.
The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Authors: Devansh Arpit, Yoshua Bengio
Date: 2019-01-11 00:00:00

It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we prove some desirable theoretical properties at initialization which may be enjoyed by ReLU networks. Specifically, it is known that He initialization in deep ReLU networks asymptotically preserves variance of activations in the forward pass and variance of gradients in the backward pass for infinitely wide networks, thus preserving the flow of information in both directions. Our paper goes beyond these results and shows novel properties that hold under He initialization: i) the norm of hidden activation of each layer is equal to the norm of the input, and, ii) the norm of weight gradient of each layer is equal to the product of norm of the input vector and the error at output layer. These results are derived using the PAC analysis framework, and hold true for finitely sized datasets such that the width of the ReLU network only needs to be larger than a certain finite lower bound. As we show, this lower bound depends on the depth of the network and the number of samples, and by the virtue of being a lower bound, over-parameterized ReLU networks are endowed with these desirable properties. For the aforementioned hidden activation norm property under He initialization, we further extend our theory and show that this property holds for a finite width network even when the number of data samples is infinite. Thus we overcome several limitations of existing papers, and show new properties of deep ReLU networks at initialization.
Advanced Rich Transcription System for Estonian Speech
Authors: Tanel Alumäe, Ottokar Tilk, Asadullah
Date: 2019-01-11 00:00:00
Tasks: Speaker Identification

This paper describes the current TT\"U speech transcription system for
Estonian speech. The system is designed to handle semi-spontaneous speech, such
as broadcast conversations, lecture recordings and interviews recorded in
diverse acoustic conditions. The system is based on the Kaldi toolkit.
Multi-condition training using background noise profiles extracted
automatically from untranscribed data is used to improve the robustness of the
system. Out-of-vocabulary words are recovered using a phoneme n-gram based
decoding subgraph and a FST-based phoneme-to-grapheme model. The system
achieves a word error rate of 8.1% on a test set of broadcast conversations.
The system also performs punctuation recovery and speaker identification.
Speaker identification models are trained using a recently proposed weakly
supervised training method.
CT-GAN: Malicious Tampering of 3D Medical Imagery using Deep Learning
Authors: Yisroel Mirsky, Tom Mahler, Ilan Shelef, Yuval Elovici
Date: 2019-01-11 00:00:00

In 2018, clinics and hospitals were hit with numerous attacks leading to significant data breaches and interruptions in medical services. An attacker with access to medical records can do much more than hold the data for ransom or sell it on the black market. In this paper, we show how an attacker can use deep-learning to add or remove evidence of medical conditions from volumetric (3D) medical scans. An attacker may perform this act in order to stop a political candidate, sabotage research, commit insurance fraud, perform an act of terrorism, or even commit murder. We implement the attack using a 3D conditional GAN and show how the framework (CT-GAN) can be automated. Although the body is complex and 3D medical scans are very large, CT-GAN achieves realistic results which can be executed in milliseconds. To evaluate the attack, we focused on injecting and removing lung cancer from CT scans. We show how three expert radiologists and a state-of-the-art deep learning AI are highly susceptible to the attack. We also explore the attack surface of a modern radiology network and demonstrate one attack vector: we intercepted and manipulated CT scans in an active hospital network with a covert penetration test. Demo video: https://youtu.be/_mkRAArj-x0 Source code: https://github.com/ymirsky/CT-GAN
Background Subtraction in Real Applications: Challenges, Current Models and Future Directions
Authors: T. Bouwmans, B. Garcia-Garcia
Date: 2019-01-11 00:00:00

Computer vision applications based on videos often require the detection of
moving objects in their first step. Background subtraction is then applied in
order to separate the background and the foreground. In literature, background
subtraction is surely among the most investigated field in computer vision
providing a big amount of publications. Most of them concern the application of
mathematical and machine learning models to be more robust to the challenges
met in videos. However, the ultimate goal is that the background subtraction
methods developed in research could be employed in real applications like
traffic surveillance. But looking at the literature, we can remark that there
is often a gap between the current methods used in real applications and the
current methods in fundamental research. In addition, the videos evaluated in
large-scale datasets are not exhaustive in the way that they only covered a
part of the complete spectrum of the challenges met in real applications. In
this context, we attempt to provide the most exhaustive survey as possible on
real applications that used background subtraction in order to identify the
real challenges met in practice, the current used background models and to
provide future directions. Thus, challenges are investigated in terms of
camera, foreground objects and environments. In addition, we identify the
background models that are effectively used in these applications in order to
find potential usable recent background models in terms of robustness, time and
memory requirements.
Life is Random, Time is Not: Markov Decision Processes with Window Objectives
Authors: Thomas Brihaye, Florent Delgrange, Youssouf Oualhadj, Mickael Randour
Date: 2019-01-11 00:00:00

The window mechanism was introduced by Chatterjee et al. to strengthen classical game objectives with time bounds. It permits to synthesize system controllers that exhibit acceptable behaviors within a configurable time frame, all along their infinite execution, in contrast to the traditional objectives that only require correctness of behaviors in the limit. The window concept has proved its interest in a variety of two-player zero-sum games because it enables reasoning about such time bounds in system specifications, but also thanks to the increased tractability that it usually yields. In this work, we extend the window framework to stochastic environments by considering Markov decision processes. A fundamental problem in this context is the threshold probability problem: given an objective it aims to synthesize strategies that guarantee satisfying runs with a given probability. We solve it for the usual variants of window objectives, where either the time frame is set as a parameter, or we ask if such a time frame exists. We develop a generic approach for window-based objectives and instantiate it for the classical mean-payoff and parity objectives, already considered in games. Our work paves the way to a wide use of the window mechanism in stochastic models.
An investigation of model-free planning
Authors: Arthur Guez, Mehdi Mirza, Karol Gregor, Rishabh Kabra, Sébastien Racanière, Théophane Weber, David Raposo, Adam Santoro, Laurent Orseau, Tom Eccles, Greg Wayne, David Silver, Timothy Lillicrap
Date: 2019-01-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: Inductive Bias, model, Reinforcement Learning, Reinforcement Learning (RL), Sokoban

The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that it can plan effectively. Prior work has typically utilized an explicit model of the environment, combined with a specific planning algorithm (such as tree search). More recently, a new family of methods have been proposed that learn how to plan, by providing the structure for planning via an inductive bias in the function approximator (such as a tree structured neural network), trained end-to-end by a model-free RL algorithm. In this paper, we go even further, and demonstrate empirically that an entirely model-free approach, without special structure beyond standard neural network components such as convolutional networks and LSTMs, can learn to exhibit many of the characteristics typically associated with a model-based planner. We measure our agent's effectiveness at planning in terms of its ability to generalize across a combinatorial and irreversible state space, its data efficiency, and its ability to utilize additional thinking time. We find that our agent has many of the characteristics that one might expect to find in a planning algorithm. Furthermore, it exceeds the state-of-the-art in challenging combinatorial domains such as Sokoban and outperforms other model-free approaches that utilize strong inductive biases toward planning.
CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
Authors: Kishan Babu Kancharagunta, Shiv Ram Dubey
Date: 2019-01-11 00:00:00
Tasks: Image Generation, SSIM

The primary motivation of Image-to-Image Transformation is to convert an
image of one domain to another domain. Most of the research has been focused on
the task of image transformation for a set of pre-defined domains. Very few
works are reported that actually developed a common framework for
image-to-image transformation for different domains. With the introduction of
Generative Adversarial Networks (GANs) as a general framework for the image
generation problem, there is a tremendous growth in the area of image-to-image
transformation. Most of the research focuses over the suitable objective
function for image-to-image transformation. In this paper, we propose a new
Cyclic-Synthesized Generative Adversarial Networks (CSGAN) for image-to-image
transformation. The proposed CSGAN uses a new objective function (loss) called
Cyclic-Synthesized Loss (CS) between the synthesized image of one domain and
cycled image of another domain. The performance of the proposed CSGAN is
evaluated on two benchmark image-to-image transformation datasets, including
CUHK Face dataset and CMP Facades dataset. The results are computed using the
widely used evaluation metrics such as MSE, SSIM, PSNR, and LPIPS. The
experimental results of the proposed CSGAN approach are compared with the
latest state-of-the-art approaches such as GAN, Pix2Pix, DualGAN, CycleGAN and
PS2GAN. The proposed CSGAN technique outperforms all the methods over CUHK
dataset and exhibits the promising and comparable performance over Facades
dataset in terms of both qualitative and quantitative measures. The code is
available at https://github.com/KishanKancharagunta/CSGAN.