pwc-archive/papers-with-abstracts huggingface dataset:
There are No Bit Parts for Sign Bits in Black-Box Attacks
Authors: Abdullah Al-Dujaili, Una-May O'Reilly
Date: 2019-02-19 00:00:00
Tasks: Adversarial Attack

We present a black-box adversarial attack algorithm which sets new
state-of-the-art model evasion rates for query efficiency in the $\ell_\infty$
and $\ell_2$ metrics, where only loss-oracle access to the model is available.
On two public black-box attack challenges, the algorithm achieves the highest
evasion rate, surpassing all of the submitted attacks. Similar performance is
observed on a model that is secure against substitute-model attacks. For
standard models trained on the MNIST, CIFAR10, and IMAGENET datasets, averaged
over the datasets and metrics, the algorithm is 3.8x less failure-prone, and
spends in total 2.5x fewer queries than the current state-of-the-art attacks
combined given a budget of 10, 000 queries per attack attempt. Notably, it
requires no hyperparameter tuning or any data/time-dependent prior. The
algorithm exploits a new approach, namely sign-based rather than
magnitude-based gradient estimation. This shifts the estimation from continuous
to binary black-box optimization. With three properties of the directional
derivative, we examine three approaches to adversarial attacks. This yields a
superior algorithm breaking a standard MNIST model using just 12 queries on
average!
Probabilistic Modeling with Matrix Product States
Authors: James Stokes, John Terilla
Date: 2019-02-19 00:00:00
Tasks: Inductive Bias

Inspired by the possibility that generative models based on quantum circuits
can provide a useful inductive bias for sequence modeling tasks, we propose an
efficient training algorithm for a subset of classically simulable quantum
circuit models. The gradient-free algorithm, presented as a sequence of exactly
solvable effective models, is a modification of the density matrix
renormalization group procedure adapted for learning a probability
distribution. The conclusion that circuit-based models offer a useful inductive
bias for classical datasets is supported by experimental results on the parity
learning problem.
Proper-Composite Loss Functions in Arbitrary Dimensions
Authors: Zac Cranko, Robert C. Williamson, Richard Nock
Date: 2019-02-19 00:00:00
Tasks: Density Estimation, scoring rule

The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probability distributions which are then scored via a scoring rule. However, recent research so far has primarily been concerned with analysing the (typically) finite-dimensional conditional risk problem on the output space, leaving aside the larger total risk minimisation. We generalise a number of these results to an infinite dimensional setting and in doing so we are able to exploit the familial resemblance of density and conditional density estimation to provide a simple characterisation of the canonical link.
Reconstruction of 3-D Atomic Distortions from Electron Microscopy with Deep Learning
Authors: Nouamane Laanait, Qian He, Albina Y. Borisevich
Date: 2019-02-19 00:00:00
Tasks: Deep Learning

Deep learning has demonstrated superb efficacy in processing imaging data,
yet its suitability in solving challenging inverse problems in scientific
imaging has not been fully explored. Of immense interest is the determination
of local material properties from atomically-resolved imaging, such as electron
microscopy, where such information is encoded in subtle and complex data
signatures, and whose recovery and interpretation necessitate intensive
numerical simulations subject to the requirement of near-perfect knowledge of
the experimental setup. We demonstrate that an end-to-end deep learning model
can successfully recover 3-dimensional atomic distortions of a variety of oxide
perovskite materials from a single 2-dimensional experimental scanning
transmission electron (STEM) micrograph, in the process resolving a
longstanding question in the recovery of 3-D atomic distortions from STEM
experiments. Our results indicate that deep learning is a promising approach to
efficiently address unsolved inverse problems in scientific imaging and to
underpin novel material investigations at atomic resolution.
Predicting city safety perception based on visual image content
Authors: Sergio Acosta, Jorge E. Camargo
Date: 2019-02-19 00:00:00
Proceeding: arXiv 2019 2
Tasks: BIG-bench Machine Learning

Safety perception measurement has been a subject of interest in many cities
of the world. This is due to its social relevance, and to its effect on some
local economic activities. Even though people safety perception is a subjective
topic, sometimes it is possible to find out common patterns given a restricted
geographical and sociocultural context. This paper presents an approach that
makes use of image processing and machine learning techniques to detect with
high accuracy urban environment patterns that could affect citizen's safety
perception.
Hyperbolic Discounting and Learning over Multiple Horizons
Authors: William Fedus, Carles Gelada, Yoshua Bengio, Marc G. Bellemare, Hugo Larochelle
Date: 2019-02-19 00:00:00
Proceeding: ICLR 2020 1
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) typically defines a discount factor as part of
the Markov Decision Process. The discount factor values future rewards by an
exponential scheme that leads to theoretical convergence guarantees of the
Bellman equation. However, evidence from psychology, economics and neuroscience
suggests that humans and animals instead have hyperbolic time-preferences. In
this work we revisit the fundamentals of discounting in RL and bridge this
disconnect by implementing an RL agent that acts via hyperbolic discounting. We
demonstrate that a simple approach approximates hyperbolic discount functions
while still using familiar temporal-difference learning techniques in RL.
Additionally, and independent of hyperbolic discounting, we make a surprising
discovery that simultaneously learning value functions over multiple
time-horizons is an effective auxiliary task which often improves over a strong
value-based RL agent, Rainbow.
Sufficiently Accurate Model Learning
Authors: Clark Zhang, Arbaaz Khan, Santiago Paternain, Alejandro Ribeiro
Date: 2019-02-19 00:00:00
Tasks: model

Modeling how a robot interacts with the environment around it is an important prerequisite for designing control and planning algorithms. In fact, the performance of controllers and planners is highly dependent on the quality of the model. One popular approach is to learn data driven models in order to compensate for inaccurate physical measurements and to adapt to systems that evolve over time. In this paper, we investigate a method to regularize model learning techniques to provide better error characteristics for traditional control and planning algorithms. This work proposes learning "Sufficiently Accurate" models of dynamics using a primal-dual method that can explicitly enforce constraints on the error in pre-defined parts of the state-space. The result of this method is that the error characteristics of the learned model is more predictable and can be better utilized by planning and control algorithms. The characteristics of Sufficiently Accurate models are analyzed through experiments on a simulated ball paddle system.
Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions
Authors: Dogancan Temel, Tariq Alshawi, Min-Hung Chen, Ghassan AlRegib
Date: 2019-02-19 00:00:00
Tasks: Traffic Sign Detection

State-of-the-art algorithms successfully localize and recognize traffic signs over existing datasets, which are limited in terms of challenging condition type and severity. Therefore, it is not possible to estimate the performance of traffic sign detection algorithms under overlooked challenging conditions. Another shortcoming of existing datasets is the limited utilization of temporal information and the unavailability of consecutive frames and annotations. To overcome these shortcomings, we generated the CURE-TSD video dataset and hosted the first IEEE Video and Image Processing (VIP) Cup within the IEEE Signal Processing Society. In this paper, we provide a detailed description of the CURE-TSD dataset, analyze the characteristics of the top performing algorithms, and provide a performance benchmark. Moreover, we investigate the robustness of the benchmarked algorithms with respect to sign size, challenge type and severity. Benchmarked algorithms are based on state-of-the-art and custom convolutional neural networks that achieved a precision of 0.55 and a recall of 0.32, F0.5 score of 0.48 and F2 score of 0.35. Experimental results show that benchmarked algorithms are highly sensitive to tested challenging conditions, which result in an average performance drop of 0.17 in terms of precision and a performance drop of 0.28 in recall under severe conditions. The dataset is publicly available at https://github.com/olivesgatech/CURE-TSD.
On the Impact of the Activation Function on Deep Neural Networks Training
Authors: Soufiane Hayou, Arnaud Doucet, Judith Rousseau
Date: 2019-02-19 00:00:00

The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Samuel et al (2017) who showed that for deep feedforward neural networks only a specific choice of hyperparameters known as the `Edge of Chaos' can lead to good performance. While the work by Samuel et al (2017) discuss trainability issues, we focus here on training acceleration and overall performance. We give a comprehensive theoretical analysis of the Edge of Chaos and show that we can indeed tune the initialization parameters and the activation function in order to accelerate the training and improve the performance.
Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
Authors: Amir Hossein Yazdavar, Mohammad Saeid Mahdavinejad, Goonmeet Bajaj, William Romine, Amirhassan Monadjemi, Krishnaprasad Thirunarayan, Amit Sheth, Jyotishman Pathak
Date: 2019-02-19 00:00:00

With ubiquity of social media platforms, millions of people are sharing their
online persona by expressing their thoughts, moods, emotions, feelings, and
even their daily struggles with mental health issues voluntarily and publicly
on social media. Unlike the most existing efforts which study depression by
analyzing textual content, we examine and exploit multimodal big data to
discern depressive behavior using a wide variety of features including
individual-level demographics. By developing a multimodal framework and
employing statistical techniques for fusing heterogeneous sets of features
obtained by processing visual, textual and user interaction data, we
significantly enhance the current state-of-the-art approaches for identifying
depressed individuals on Twitter (improving the average F1-Score by 5 percent)
as well as facilitate demographic inference from social media for broader
applications. Besides providing insights into the relationship between
demographics and mental health, our research assists in the design of a new
breed of demographic-aware health interventions.
An Adaptive Deep Learning Algorithm Based Autoencoder for Interference Channels
Authors: Dehao Wu, Maziar Nekovee, Yue Wang
Date: 2019-02-18 00:00:00
Tasks: Deep Learning

Deep learning (DL) based autoencoder has shown great potential to significantly enhance the physical layer performance. In this paper, we present a DL based autoencoder for interference channel. Based on a characterization of a k-user Gaussian interference channel, where the interferences are classified as different levels from weak to very strong interferences based on a coupling parameter {\alpha}, a DL neural network (NN) based autoencoder is designed to train the data set and decode the received signals. The performance such a DL autoencoder for different interference scenarios are studied, with {\alpha} known or partially known, where we assume that {\alpha} is predictable but with a varying up to 10\% at the training stage. The results demonstrate that DL based approach has a significant capability to mitigate the effect induced by a poor signal-to-noise ratio (SNR) and a high interference-to-noise ratio (INR). However, the enhancement depends on the knowledge of {\alpha} as well as the interference levels. The proposed DL approach performs well with {\alpha} up to 10\% offset for weak interference level. For strong and very strong interference channel, the offset of {\alpha} needs to be constrained to less than 5\% and 2\%, respectively, to maintain similar performance as {\alpha} is known.
SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color
Authors: Youngjoo Jo, Jongyoul Park
Date: 2019-02-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: Facial Inpainting, Form, Generative Adversarial Network, Image Inpainting

We present a novel image editing system that generates images as the user
provides free-form mask, sketch and color as an input. Our system consist of a
end-to-end trainable convolutional network. Contrary to the existing methods,
our system wholly utilizes free-form user input with color and shape. This
allows the system to respond to the user's sketch and color input, using it as
a guideline to generate an image. In our particular work, we trained network
with additional style loss which made it possible to generate realistic
results, despite large portions of the image being removed. Our proposed
network architecture SC-FEGAN is well suited to generate high quality synthetic
image using intuitive user inputs.
Graph Dynamical Networks for Unsupervised Learning of Atomic Scale Dynamics in Materials
Authors: Tian Xie, Arthur France-Lanord, Yanming Wang, Yang Shao-Horn, Jeffrey C. Grossman
Date: 2019-02-18 00:00:00

Understanding the dynamical processes that govern the performance of functional materials is essential for the design of next generation materials to tackle global energy and environmental challenges. Many of these processes involve the dynamics of individual atoms or small molecules in condensed phases, e.g. lithium ions in electrolytes, water molecules in membranes, molten atoms at interfaces, etc., which are difficult to understand due to the complexity of local environments. In this work, we develop graph dynamical networks, an unsupervised learning approach for understanding atomic scale dynamics in arbitrary phases and environments from molecular dynamics simulations. We show that important dynamical information can be learned for various multi-component amorphous material systems, which is difficult to obtain otherwise. With the large amounts of molecular dynamics data generated everyday in nearly every aspect of materials design, this approach provides a broadly useful, automated tool to understand atomic scale dynamics in material systems.
Commodity RGB-D Sensors: Data Acquisition
Authors: Michael Zollhöfer
Date: 2019-02-18 00:00:00
Tasks: Dynamic Reconstruction

Over the past ten years we have seen a democratization of range sensing
technology. While previously range sensors have been highly expensive and only
accessible to a few domain experts, such sensors are nowadays ubiquitous and
can even be found in the latest generation of mobile devices, e.g., current
smartphones. This democratization of range sensing technology was started with
the release of the Microsoft Kinect, and since then many different commodity
range sensors followed its lead, such as the Primesense Carmine, Asus Xtion
Pro, and the Structure Sensor from Occipital. The availability of cheap range
sensing technology led to a big leap in research, especially in the context of
more powerful static and dynamic reconstruction techniques, starting from 3D
scanning applications, such as KinectFusion, to highly accurate face and body
tracking approaches. In this chapter, we have a detailed look into the
different types of existing range sensors. We discuss the two fundamental types
of commodity range sensing techniques in detail, namely passive and active
sensing, and we explore the principles these technologies are based on. Our
focus is on modern active commodity range sensors based on time-of-flight and
structured light. We conclude by discussing the noise characteristics, working
ranges, and types of errors made by the different sensing modalities.
Learned In Speech Recognition: Contextual Acoustic Word Embeddings
Authors: Shruti Palaskar, Vikas Raunak, Florian Metze
Date: 2019-02-18 00:00:00
Tasks: Sentence, speech-recognition, Speech Recognition, Spoken Language Understanding, Word Embeddings

End-to-end acoustic-to-word speech recognition models have recently gained
popularity because they are easy to train, scale well to large amounts of
training data, and do not require a lexicon. In addition, word models may also
be easier to integrate with downstream tasks such as spoken language
understanding, because inference (search) is much simplified compared to
phoneme, character or any other sort of sub-word units. In this paper, we
describe methods to construct contextual acoustic word embeddings directly from
a supervised sequence-to-sequence acoustic-to-word speech recognition model
using the learned attention distribution. On a suite of 16 standard sentence
evaluation tasks, our embeddings show competitive performance against a
word2vec model trained on the speech transcriptions. In addition, we evaluate
these embeddings on a spoken language understanding task, and observe that our
embeddings match the performance of text-based embeddings in a pipeline of
first performing speech recognition and then constructing word embeddings from
transcriptions.
Evolutionary Neural AutoML for Deep Learning
Authors: Jason Liang, Elliot Meyerson, Babak Hodjat, Dan Fink, Karl Mutch, Risto Miikkulainen
Date: 2019-02-18 00:00:00
Tasks: AutoML, Deep Learning, Distributed Computing, Evolutionary Algorithms, Hyperparameter Optimization, image-classification, Image Classification, Medical Image Classification, Neural Architecture Search

Deep neural networks (DNNs) have produced state-of-the-art results in many
benchmarks and problem domains. However, the success of DNNs depends on the
proper configuration of its architecture and hyperparameters. Such a
configuration is difficult and as a result, DNNs are often not used to their
full potential. In addition, DNNs in commercial applications often need to
satisfy real-world design constraints such as size or number of parameters. To
make configuration easier, automatic machine learning (AutoML) systems for deep
learning have been developed, focusing mostly on optimization of
hyperparameters.
  This paper takes AutoML a step further. It introduces an evolutionary AutoML
framework called LEAF that not only optimizes hyperparameters but also network
architectures and the size of the network. LEAF makes use of both
state-of-the-art evolutionary algorithms (EAs) and distributed computing
frameworks. Experimental results on medical image classification and natural
language analysis show that the framework can be used to achieve
state-of-the-art performance. In particular, LEAF demonstrates that
architecture optimization provides a significant boost over hyperparameter
optimization, and that networks can be minimized at the same time with little
drop in performance. LEAF therefore forms a foundation for democratizing and
improving AI, as well as making AI practical in future applications.
Autonomous Airline Revenue Management: A Deep Reinforcement Learning Approach to Seat Inventory Control and Overbooking
Authors: Syed Arbab Mohd Shihab, Caleb Logemann, Deepak-George Thomas, Peng Wei
Date: 2019-02-18 00:00:00
Tasks: Deep Reinforcement Learning, Management, Q-Learning, Reinforcement Learning

Revenue management can enable airline corporations to maximize the revenue generated from each scheduled flight departing in their transportation network by means of finding the optimal policies for differential pricing, seat inventory control and overbooking. As different demand segments in the market have different Willingness-To-Pay (WTP), airlines use differential pricing, booking restrictions, and service amenities to determine different fare classes or products targeted at each of these demand segments. Because seats are limited for each flight, airlines also need to allocate seats for each of these fare classes to prevent lower fare class passengers from displacing higher fare class ones and set overbooking limits in anticipation of cancellations and no-shows such that revenue is maximized. Previous work addresses these problems using optimization techniques or classical Reinforcement Learning methods. This paper focuses on the latter problem - the seat inventory control problem - casting it as a Markov Decision Process to be able to find the optimal policy. Multiple fare classes, concurrent continuous arrival of passengers of different fare classes, overbooking and random cancellations that are independent of class have been considered in the model. We have addressed this problem using Deep Q-Learning with the goal of maximizing the reward for each flight departure. The implementation of this technique allows us to employ large continuous state space but also presents the potential opportunity to test on real time airline data. To generate data and train the agent, a basic air-travel market simulator was developed. The performance of the agent in different simulated market scenarios was compared against theoretically optimal solutions and was found to be nearly close to the expected optimal revenue.
Low-bit Quantization of Neural Networks for Efficient Inference
Authors: Yoni Choukroun, Eli Kravchik, Fan Yang, Pavel Kisilev
Date: 2019-02-18 00:00:00
Tasks: Quantization

Recent machine learning methods use increasingly large deep neural networks
to achieve state of the art results in various tasks. The gains in performance
come at the cost of a substantial increase in computation and storage
requirements. This makes real-time implementations on limited resources
hardware a challenging task. One popular approach to address this challenge is
to perform low-bit precision computations via neural network quantization.
However, aggressive quantization generally entails a severe penalty in terms of
accuracy, and often requires retraining of the network, or resorting to higher
bit precision quantization. In this paper, we formalize the linear quantization
task as a Minimum Mean Squared Error (MMSE) problem for both weights and
activations, allowing low-bit precision inference without the need for full
network retraining. The main contributions of our approach are the
optimizations of the constrained MSE problem at each layer of the network, the
hardware aware partitioning of the network parameters, and the use of multiple
low precision quantized tensors for poorly approximated layers. The proposed
approach allows 4 bits integer (INT4) quantization for deployment of pretrained
models on limited hardware resources. Multiple experiments on various network
architectures show that the suggested method yields state of the art results
with minimal loss of tasks accuracy.