pwc-archive/papers-with-abstracts huggingface dataset:
Contextual Online False Discovery Rate Control
Authors: Shiyun Chen, Shiva Kasiviswanathan
Date: 2019-02-07 00:00:00
Tasks: Two-sample testing

Multiple hypothesis testing, a situation when we wish to consider many
hypotheses, is a core problem in statistical inference that arises in almost
every scientific field. In this setting, controlling the false discovery rate
(FDR), which is the expected proportion of type I error, is an important
challenge for making meaningful inferences. In this paper, we consider the
problem of controlling FDR in an online manner. Concretely, we consider an
ordered, possibly infinite, sequence of hypotheses, arriving one at each
timestep, and for each hypothesis we observe a p-value along with a set of
features specific to that hypothesis. The decision whether or not to reject the
current hypothesis must be made immediately at each timestep, before the next
hypothesis is observed. The model of multi-dimensional feature set provides a
very general way of leveraging the auxiliary information in the data which
helps in maximizing the number of discoveries.
  We propose a new class of powerful online testing procedures, where the
rejections thresholds (significance levels) are learnt sequentially by
incorporating contextual information and previous results. We prove that any
rule in this class controls online FDR under some standard assumptions. We then
focus on a subclass of these procedures, based on weighting significance
levels, to derive a practical algorithm that learns a parametric weight
function in an online fashion to gain more discoveries. We also theoretically
prove, in a stylized setting, that our proposed procedures would lead to an
increase in the achieved statistical power over a popular online testing
procedure proposed by Javanmard & Montanari (2018). Finally, we demonstrate the
favorable performance of our procedure, by comparing it to state-of-the-art
online multiple testing procedures, on both synthetic data and real data
generated from different applications.
Combining learning rate decay and weight decay with complexity gradient descent - Part I
Authors: Pierre H. Richemond, Yike Guo
Date: 2019-02-07 00:00:00

The role of $L^2$ regularization, in the specific case of deep neural
networks rather than more traditional machine learning models, is still not
fully elucidated. We hypothesize that this complex interplay is due to the
combination of overparameterization and high dimensional phenomena that take
place during training and make it unamenable to standard convex optimization
methods. Using insights from statistical physics and random fields theory, we
introduce a parameter factoring in both the level of the loss function and its
remaining nonconvexity: the \emph{complexity}. We proceed to show that it is
desirable to proceed with \emph{complexity gradient descent}. We then show how
to use this intuition to derive novel and efficient annealing schemes for the
strength of $L^2$ regularization when performing standard stochastic gradient
descent in deep neural networks.
Mean Field Limit of the Learning Dynamics of Multilayer Neural Networks
Authors: Phan-Minh Nguyen
Date: 2019-02-07 00:00:00

Can multilayer neural networks -- typically constructed as highly complex
structures with many nonlinearly activated neurons across layers -- behave in a
non-trivial way that yet simplifies away a major part of their complexities? In
this work, we uncover a phenomenon in which the behavior of these complex
networks -- under suitable scalings and stochastic gradient descent dynamics --
becomes independent of the number of neurons as this number grows sufficiently
large. We develop a formalism in which this many-neurons limiting behavior is
captured by a set of equations, thereby exposing a previously unknown operating
regime of these networks. While the current pursuit is mathematically
non-rigorous, it is complemented with several experiments that validate the
existence of this behavior.
Deep execution monitor for robot assistive tasks
Authors: Lorenzo Mauro, Edoardo Alati, Marta Sanzari, Valsamis Ntouskos, Gianluca Massimiani, Fiora Pirri
Date: 2019-02-07 00:00:00

We consider a novel approach to high-level robot task execution for a robot
assistive task. In this work we explore the problem of learning to predict the
next subtask by introducing a deep model for both sequencing goals and for
visually evaluating the state of a task. We show that deep learning for
monitoring robot tasks execution very well supports the interconnection between
task-level planning and robot operations. These solutions can also cope with
the natural non-determinism of the execution monitor. We show that a deep
execution monitor leverages robot performance. We measure the improvement
taking into account some robot helping tasks performed at a warehouse.
Visual search and recognition for robot task execution and monitoring
Authors: Lorenzo Mauro, Francesco Puja, Simone Grazioso, Valsamis Ntouskos, Marta Sanzari, Edoardo Alati, Fiora Pirri
Date: 2019-02-07 00:00:00
Tasks: Common Sense Reasoning, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL)

Visual search of relevant targets in the environment is a crucial robot
skill. We propose a preliminary framework for the execution monitor of a robot
task, taking care of the robot attitude to visually searching the environment
for targets involved in the task. Visual search is also relevant to recover
from a failure. The framework exploits deep reinforcement learning to acquire a
"common sense" scene structure and it takes advantage of a deep convolutional
network to detect objects and relevant relations holding between them. The
framework builds on these methods to introduce a vision-based execution
monitoring, which uses classical planning as a backbone for task execution.
Experiments show that with the proposed vision-based execution monitor the
robot can complete simple tasks and can recover from failures in autonomy.
Crop Yield Prediction Using Deep Neural Networks
Authors: Saeed Khaki, Lizhi Wang
Date: 2019-02-07 00:00:00
Tasks: Crop Yield Prediction, feature selection, Prediction

Crop yield is a highly complex trait determined by multiple factors such as genotype, environment, and their interactions. Accurate yield prediction requires fundamental understanding of the functional relationship between yield and these interactive factors, and to reveal such relationship requires both comprehensive datasets and powerful algorithms. In the 2018 Syngenta Crop Challenge, Syngenta released several large datasets that recorded the genotype and yield performances of 2,267 maize hybrids planted in 2,247 locations between 2008 and 2016 and asked participants to predict the yield performance in 2017. As one of the winning teams, we designed a deep neural network (DNN) approach that took advantage of state-of-the-art modeling and solution techniques. Our model was found to have a superior prediction accuracy, with a root-mean-square-error (RMSE) being 12% of the average yield and 50% of the standard deviation for the validation dataset using predicted weather data. With perfect weather data, the RMSE would be reduced to 11% of the average yield and 46% of the standard deviation. We also performed feature selection based on the trained DNN model, which successfully decreased the dimension of the input space without significant drop in the prediction accuracy. Our computational results suggested that this model significantly outperformed other popular methods such as Lasso, shallow neural networks (SNN), and regression tree (RT). The results also revealed that environmental factors had a greater effect on the crop yield than genotype.
FaceSpoof Buster: a Presentation Attack Detector Based on Intrinsic Image Properties and Deep Learning
Authors: Rodrigo Bresan, Allan Pinto, Anderson Rocha, Carlos Beluzo, Tiago Carvalho
Date: 2019-02-07 00:00:00
Tasks: Face Recognition, Meta-Learning

Nowadays, the adoption of face recognition for biometric authentication
systems is usual, mainly because this is one of the most accessible biometric
modalities. Techniques that rely on trespassing these kind of systems by using
a forged biometric sample, such as a printed paper or a recorded video of a
genuine access, are known as presentation attacks, but may be also referred in
the literature as face spoofing. Presentation attack detection is a crucial
step for preventing this kind of unauthorized accesses into restricted areas
and/or devices. In this paper, we propose a novel approach which relies in a
combination between intrinsic image properties and deep neural networks to
detect presentation attack attempts. Our method explores depth, salience and
illumination maps, associated with a pre-trained Convolutional Neural Network
in order to produce robust and discriminant features. Each one of these
properties are individually classified and, in the end of the process, they are
combined by a meta learning classifier, which achieves outstanding results on
the most popular datasets for PAD. Results show that proposed method is able to
overpass state-of-the-art results in an inter-dataset protocol, which is
defined as the most challenging in the literature.
3D Human Pose Estimation from Deep Multi-View 2D Pose
Authors: Steven Schwarcz, Thomas Pollard
Date: 2019-02-07 00:00:00
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation

Human pose estimation - the process of recognizing a human's limb positions
and orientations in a video - has many important applications including
surveillance, diagnosis of movement disorders, and computer animation. While
deep learning has lead to great advances in 2D and 3D pose estimation from
single video sources, the problem of estimating 3D human pose from multiple
video sensors with overlapping fields of view has received less attention. When
the application allows use of multiple cameras, 3D human pose estimates may be
greatly improved through fusion of multi-view pose estimates and observation of
limbs that are fully or partially occluded in some views. Past approaches to
multi-view 3D pose estimation have used probabilistic graphical models to
reason over constraints, including per-image pose estimates, temporal
smoothness, and limb length. In this paper, we present a pipeline for
multi-view 3D pose estimation of multiple individuals which combines a
state-of-art 2D pose detector with a factor graph of 3D limb constraints
optimized with belief propagation. We evaluate our results on the TUM-Campus
and Shelf datasets for multi-person 3D pose estimation and show that our system
significantly out-performs the previous state-of-the-art with a simpler model
of limb dependency.
The Long and the Short of It: Summarising Event Sequences with Serial Episodes
Authors: Nikolaj Tatti, Jilles Vreeken
Date: 2019-02-07 00:00:00

An ideal outcome of pattern mining is a small set of informative patterns,
containing no redundancy or noise, that identifies the key structure of the
data at hand. Standard frequent pattern miners do not achieve this goal, as due
to the pattern explosion typically very large numbers of highly redundant
patterns are returned.
  We pursue the ideal for sequential data, by employing a pattern set mining
approach-an approach where, instead of ranking patterns individually, we
consider results as a whole. Pattern set mining has been successfully applied
to transactional data, but has been surprisingly under studied for sequential
data.
  In this paper, we employ the MDL principle to identify the set of sequential
patterns that summarises the data best. In particular, we formalise how to
encode sequential data using sets of serial episodes, and use the encoded
length as a quality score. As search strategy, we propose two approaches: the
first algorithm selects a good pattern set from a large candidate set, while
the second is a parameter-free any-time algorithm that mines pattern sets
directly from the data. Experimentation on synthetic and real data demonstrates
we efficiently discover small sets of informative patterns.
Evaluating Crowd Density Estimators via Their Uncertainty Bounds
Authors: Jennifer Vandoni, Emanuel Aldea, Sylvie Le Hégarat-Mascle
Date: 2019-02-07 00:00:00

In this work, we use the Belief Function Theory which extends the
probabilistic framework in order to provide uncertainty bounds to different
categories of crowd density estimators. Our method allows us to compare the
multi-scale performance of the estimators, and also to characterize their
reliability for crowd monitoring applications requiring varying degrees of
prudence.
Low-cost Measurement of Industrial Shock Signals via Deep Learning Calibration
Authors: Houpu Yao, Jingjing Wen, Yi Ren, Bin Wu, Ze Ji
Date: 2019-02-07 00:00:00
Tasks: Deep Learning

Special high-end sensors with expensive hardware are usually needed to
measure shock signals with high accuracy. In this paper, we show that cheap
low-end sensors calibrated by deep neural networks are also capable to measure
high-g shocks accurately. Firstly we perform drop shock tests to collect a
dataset of shock signals measured by sensors of different fidelity. Secondly,
we propose a novel network to effectively learn both the signal peak and
overall shape. The results show that the proposed network is capable to map
low-end shock signals to its high-end counterparts with satisfactory accuracy.
To the best of our knowledge, this is the first work to apply deep learning
techniques to calibrate shock sensors.
Robustness Of Saak Transform Against Adversarial Attacks
Authors: Thiyagarajan Ramanathan, Abinaya Manimaran, Suya You, C-C Jay Kuo
Date: 2019-02-07 00:00:00
Tasks: Adversarial Robustness, Classification, Denoising, General Classification, image-classification, Image Classification

Image classification is vulnerable to adversarial attacks. This work
investigates the robustness of Saak transform against adversarial attacks
towards high performance image classification. We develop a complete image
classification system based on multi-stage Saak transform. In the Saak
transform domain, clean and adversarial images demonstrate different
distributions at different spectral dimensions. Selection of the spectral
dimensions at every stage can be viewed as an automatic denoising process.
Motivated by this observation, we carefully design strategies of feature
extraction, representation and classification that increase adversarial
robustness. The performances with well-known datasets and attacks are
demonstrated by extensive experimental evaluations.
Compatible Natural Gradient Policy Search
Authors: Joni Pajarinen, Hong Linh Thai, Riad Akrour, Jan Peters, Gerhard Neumann
Date: 2019-02-07 00:00:00
Tasks: continuous-control, Continuous Control

Trust-region methods have yielded state-of-the-art results in policy search.
A common approach is to use KL-divergence to bound the region of trust
resulting in a natural gradient policy update. We show that the natural
gradient and trust region optimization are equivalent if we use the natural
parameterization of a standard exponential policy distribution in combination
with compatible value function approximation. Moreover, we show that standard
natural gradient updates may reduce the entropy of the policy according to a
wrong schedule leading to premature convergence. To control entropy reduction
we introduce a new policy search method called compatible policy search (COPOS)
which bounds entropy loss. The experimental results show that COPOS yields
state-of-the-art results in challenging continuous control tasks and in
discrete partially observable tasks.
Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning
Authors: Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu
Date: 2019-02-07 00:00:00
Tasks: Image Generation, Image-to-Image Translation

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initializer and slow thinking solver. The initializer generates the output directly by a non-linear transformation of the input as well as a noise vector that accounts for latent variability in the output. The slow thinking solver learns an objective function in the form of a conditional energy function, so that the output can be generated by optimizing the objective function, or more rigorously by sampling from the conditional energy-based model. We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative algorithm. The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output. We demonstrate the effectiveness of the proposed method on various conditional learning tasks, e.g., class-to-image generation, image-to-image translation, and image recovery. The advantage of our method over GAN-based methods is that our method is equipped with a slow thinking process that refines the solution guided by a learned objective function.
ML Health: Fitness Tracking for Production Models
Authors: Sindhu Ghanta, Sriram Subramanian, Lior Khermosh, Swaminathan Sundararaman, Harshil Shah, Yakov Goldberg, Drew Roselli, Nisha Talagala
Date: 2019-02-07 00:00:00
Tasks: Diagnostic, Management

Deployment of machine learning (ML) algorithms in production for extended
periods of time has uncovered new challenges such as monitoring and management
of real-time prediction quality of a model in the absence of labels. However,
such tracking is imperative to prevent catastrophic business outcomes resulting
from incorrect predictions. The scale of these deployments makes manual
monitoring prohibitive, making automated techniques to track and raise alerts
imperative. We present a framework, ML Health, for tracking potential drops in
the predictive performance of ML models in the absence of labels. The framework
employs diagnostic methods to generate alerts for further investigation. We
develop one such method to monitor potential problems when production data
patterns do not match training data distributions. We demonstrate that our
method performs better than standard "distance metrics", such as RMSE,
KL-Divergence, and Wasserstein at detecting issues with mismatched data sets.
Finally, we present a working system that incorporates the ML Health approach
to monitor and manage ML deployments within a realistic full production ML
lifecycle.
SiamVGG: Visual Tracking using Deeper Siamese Networks
Authors: Yuhong Li, Xiaofan Zhang, Deming Chen
Date: 2019-02-07 00:00:00
Tasks: Object Tracking, Visual Object Tracking, Visual Tracking

Recently, we have seen a rapid development of Deep Neural Network (DNN) based visual tracking solutions. Some trackers combine the DNN-based solutions with Discriminative Correlation Filters (DCF) to extract semantic features and successfully deliver the state-of-the-art tracking accuracy. However, these solutions are highly compute-intensive, which require long processing time, resulting unsecured real-time performance. To deliver both high accuracy and reliable real-time performance, we propose a novel tracker called SiamVGG\footnote{https://github.com/leeyeehoo/SiamVGG}. It combines a Convolutional Neural Network (CNN) backbone and a cross-correlation operator, and takes advantage of the features from exemplary images for more accurate object tracking. The architecture of SiamVGG is customized from VGG-16 with the parameters shared by both exemplary images and desired input video frames. We demonstrate the proposed SiamVGG on OTB-2013/50/100 and VOT 2015/2016/2017 datasets with the state-of-the-art accuracy while maintaining a decent real-time performance of 50 FPS running on a GTX 1080Ti. Our design can achieve 2% higher Expected Average Overlap (EAO) compared to the ECO and C-COT in VOT2017 Challenge.
Hide and Speak: Towards Deep Neural Networks for Speech Steganography
Authors: Felix Kreuk, Yossi Adi, Bhiksha Raj, Rita Singh, Joseph Keshet
Date: 2019-02-07 00:00:00
Tasks: Decoder

Steganography is the science of hiding a secret message within an ordinary public message, which is referred to as Carrier. Traditionally, digital signal processing techniques, such as least significant bit encoding, were used for hiding messages. In this paper, we explore the use of deep neural networks as steganographic functions for speech data. We showed that steganography models proposed for vision are less suitable for speech, and propose a new model that includes the short-time Fourier transform and inverse-short-time Fourier transform as differentiable layers within the network, thus imposing a vital constraint on the network outputs. We empirically demonstrated the effectiveness of the proposed method comparing to deep learning based on several speech datasets and analyzed the results quantitatively and qualitatively. Moreover, we showed that the proposed approach could be applied to conceal multiple messages in a single carrier using multiple decoders or a single conditional decoder. Lastly, we evaluated our model under different channel distortions. Qualitative experiments suggest that modifications to the carrier are unnoticeable by human listeners and that the decoded messages are highly intelligible.
Illumination Invariant Foreground Object Segmentation using ForeGANs
Authors: Maryam Sultana, Soon Ki Jung
Date: 2019-02-07 00:00:00
Tasks: Foreground Segmentation, Generative Adversarial Network, Object, Segmentation, Semantic Segmentation

The foreground segmentation algorithms suffer performance degradation in the presence of various challenges such as dynamic backgrounds, and various illumination conditions. To handle these challenges, we present a foreground segmentation method, based on generative adversarial network (GAN). We aim to segment foreground objects in the presence of two aforementioned major challenges in background scenes in real environments. To address this problem, our presented GAN model is trained on background image samples with dynamic changes, after that for testing the GAN model has to generate the same background sample as test sample with similar conditions via back-propagation technique. The generated background sample is then subtracted from the given test sample to segment foreground objects. The comparison of our proposed method with five state-of-the-art methods highlights the strength of our algorithm for foreground segmentation in the presence of challenging dynamic background scenario.
Heterogeneous Edge Embeddings for Friend Recommendation
Authors: Janu Verma, Srishti Gupta, Debdoot Mukherjee, Tanmoy Chakraborty
Date: 2019-02-07 00:00:00
Tasks: Link Prediction, Network Embedding

We propose a friend recommendation system (an application of link prediction)
using edge embeddings on social networks. Most real-world social networks are
multi-graphs, where different kinds of relationships (e.g. chat, friendship)
are possible between a pair of users. Existing network embedding techniques do
not leverage signals from different edge types and thus perform inadequately on
link prediction in such networks. We propose a method to mine network
representation that effectively exploits heterogeneity in multi-graphs. We
evaluate our model on a real-world, active social network where this system is
deployed for friend recommendation for millions of users. Our method
outperforms various state-of-the-art baselines on Hike's social network in
terms of accuracy as well as user satisfaction.