pwc-archive/papers-with-abstracts huggingface dataset:
ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling
Authors: Bing Yu, Haoteng Yin, Zhanxing Zhu
Date: 2019-03-13 00:00:00
Tasks: Graph Learning, Time Series, Time Series Analysis, Traffic Prediction

The spatio-temporal graph learning is becoming an increasingly important object of graph study. Many application domains involve highly dynamic graphs where temporal information is crucial, e.g. traffic networks and financial transaction graphs. Despite the constant progress made on learning structured data, there is still a lack of effective means to extract dynamic complex features from spatio-temporal structures. Particularly, conventional models such as convolutional networks or recurrent neural networks are incapable of revealing the temporal patterns in short or long terms and exploring the spatial properties in local or global scope from spatio-temporal graphs simultaneously. To tackle this problem, we design a novel multi-scale architecture, Spatio-Temporal U-Net (ST-UNet), for graph-structured time series modeling. In this U-shaped network, a paired sampling operation is proposed in spacetime domain accordingly: the pooling (ST-Pool) coarsens the input graph in spatial from its deterministic partition while abstracts multi-resolution temporal dependencies through dilated recurrent skip connections; based on previous settings in the downsampling, the unpooling (ST-Unpool) restores the original structure of spatio-temporal graphs and resumes regular intervals within graph sequences. Experiments on spatio-temporal prediction tasks demonstrate that our model effectively captures comprehensive features in multiple scales and achieves substantial improvements over mainstream methods on several real-world datasets.
Machine Learning Solutions for High Energy Physics: Applications to Electromagnetic Shower Generation, Flavor Tagging, and the Search for di-Higgs Production
Authors: Michela Paganini
Date: 2019-03-12 00:00:00

This thesis demonstrate the efficacy of designing and developing machine
learning (ML) algorithms to selected use cases that encompass many of the
outstanding challenges in the field of experimental high energy physics.
Although simple implementations of neural networks and boosted decision trees
have been used in high energy physics for a long time, the field of ML has
quickly evolved by devising more complex, fast and stable implementations of
learning algorithms. The complexity and power of state-of-the-art deep learning
far exceeds those of the learning algorithms implemented in the CERN-developed
\texttt{ROOT} library. All aspects of experimental high energy physics have
been and will continue being revolutionized by the software- and hardware-based
technological advances spearheaded by both academic and industrial research in
other technical disciplines, and the emergent trend of increased
interdisciplinarity will soon reframe many scientific domains. This thesis
exemplifies this spirit of versatility and multidisciplinarity by bridging the
gap between ML and particle physics, and exploring original lines of work to
modernize the reconstruction, particle identification, simulation, and analysis
workflows. This contribution documents a collection of novel approaches to
augment traditional domain-specific methods with modern, automated techniques
based on industry-standard, open-source libraries. Specifically, it contributes
to setting the state-of-the-art for impact parameter-based flavor tagging and
di-Higgs searches in the $\gamma \gamma b\bar{b} $ channel with the ATLAS
detector at the LHC, it introduces and lays the foundations for the use of
generative adversarial networks for the simulation of particle showers in
calorimeters. These results substantiate the notion of ML powering particle
physics in the upcoming years and establish baselines for future applications.
Reducing the dimensionality of data using tempered distributions
Authors: Rustem Takhanov
Date: 2019-03-12 00:00:00
Tasks: Dimensionality Reduction

We reformulate unsupervised dimension reduction problem (UDR) in the language of tempered distributions, i.e. as a problem of approximating an empirical probability density function by another tempered distribution, supported in a $k$-dimensional subspace. We show that this task is connected with another classical problem of data science -- the sufficient dimension reduction problem (SDR). In fact, an algorithm for the first problem induces an algorithm for the second and vice versa. In order to reduce an optimization problem over distributions to an optimization problem over ordinary functions we introduce a nonnegative penalty function that ``forces'' the support of the model distribution to be $k$-dimensional. Then we present an algorithm for the minimization of the penalized objective, based on the infinite-dimensional low-rank optimization, which we call the alternating scheme. Also, we design an efficient approximate algorithm for a special case of the problem, where the distance between the empirical distribution and the model distribution is measured by Maximum Mean Discrepancy defined by a Mercer kernel of a certain type. We test our methods on four examples (three UDR and one SDR) using synthetic data and standard datasets.
Unsupervised Discovery of Parts, Structure, and Dynamics
Authors: Zhenjia Xu, Zhijian Liu, Chen Sun, Kevin Murphy, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu
Date: 2019-03-12 00:00:00
Tasks: Object

Humans easily recognize object parts and their hierarchical structure by
watching how they move; they can then predict how each part moves in the
future. In this paper, we propose a novel formulation that simultaneously
learns a hierarchical, disentangled object representation and a dynamics model
for object parts from unlabeled videos. Our Parts, Structure, and Dynamics
(PSD) model learns to, first, recognize the object parts via a layered image
representation; second, predict hierarchy via a structural descriptor that
composes low-level concepts into a hierarchical structure; and third, model the
system dynamics by predicting the future. Experiments on multiple real and
synthetic datasets demonstrate that our PSD model works well on all three
tasks: segmenting object parts, building their hierarchical structure, and
capturing their motion distributions.
Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
Authors: Adith Boloor, Xin He, Christopher Gill, Yevgeniy Vorobeychik, Xuan Zhang
Date: 2019-03-12 00:00:00
Tasks: Autonomous Driving

Recent advances in machine learning, especially techniques such as deep
neural networks, are promoting a range of high-stakes applications, including
autonomous driving, which often relies on deep learning for perception. While
deep learning for perception has been shown to be vulnerable to a host of
subtle adversarial manipulations of images, end-to-end demonstrations of
successful attacks, which manipulate the physical environment and result in
physical consequences, are scarce. Moreover, attacks typically involve
carefully constructed adversarial examples at the level of pixels. We
demonstrate the first end-to-end attacks on autonomous driving in simulation,
using simple physically realizable attacks: the painting of black lines on the
road. These attacks target deep neural network models for end-to-end autonomous
driving control. A systematic investigation shows that such attacks are
surprisingly easy to engineer, and we describe scenarios (e.g., right turns) in
which they are highly effective, and others that are less vulnerable (e.g.,
driving straight). Further, we use network deconvolution to demonstrate that
the attacks succeed by inducing activation patterns similar to entirely
different scenarios used in training.
Unbiased Measurement of Feature Importance in Tree-Based Methods
Authors: Zhengze Zhou, Giles Hooker
Date: 2019-03-12 00:00:00
Tasks: Feature Importance

We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as measured on out of sample data, this bias can be corrected yielding better summaries and screening tools.
Continual Learning in Practice
Authors: Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, Neil Lawrence
Date: 2019-03-12 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Continual Learning

This paper describes a reference architecture for self-maintaining systems
that can learn continually, as data arrives. In environments where data
evolves, we need architectures that manage Machine Learning (ML) models in
production, adapt to shifting data distributions, cope with outliers, retrain
when necessary, and adapt to new tasks. This represents continual AutoML or
Automatically Adaptive Machine Learning. We describe the challenges and
proposes a reference architecture.
Learning Resolution Parameters for Graph Clustering
Authors: Nate Veldt, David F. Gleich, Anthony Wirth
Date: 2019-03-12 00:00:00
Tasks: Clustering, Graph Clustering

Finding clusters of well-connected nodes in a graph is an extensively studied
problem in graph-based data analysis. Because of its many applications, a large
number of distinct graph clustering objective functions and algorithms have
already been proposed and analyzed. To aid practitioners in determining the
best clustering approach to use in different applications, we present new
techniques for automatically learning how to set clustering resolution
parameters. These parameters control the size and structure of communities that
are formed by optimizing a generalized objective function. We begin by
formalizing the notion of a parameter fitness function, which measures how well
a fixed input clustering approximately solves a generalized clustering
objective for a specific resolution parameter value. Under reasonable
assumptions, which suit two key graph clustering applications, such a parameter
fitness function can be efficiently minimized using a bisection-like method,
yielding a resolution parameter that fits well with the example clustering. We
view our framework as a type of single-shot hyperparameter tuning, as we are
able to learn a good resolution parameter with just a single example. Our
general approach can be applied to learn resolution parameters for both local
and global graph clustering objectives. We demonstrate its utility in several
experiments on real-world data where it is helpful to learn resolution
parameters from a given example clustering.
Offensive Language Analysis using Deep Learning Architecture
Authors: Ryan Ong
Date: 2019-03-12 00:00:00
Tasks: Abuse Detection, Deep Learning, Sentiment Analysis, Twitter Sentiment Analysis

SemEval-2019 Task 6 (Zampieri et al., 2019b) requires us to identify and
categorise offensive language in social media. In this paper we will describe
the process we took to tackle this challenge. Our process is heavily inspired
by Sosa (2017) where he proposed CNN-LSTM and LSTM-CNN models to conduct
twitter sentiment analysis. We decided to follow his approach as well as
further his work by testing out different variations of RNN models with CNN.
Specifically, we have divided the challenge into two parts: data processing and
sampling and choosing the optimal deep learning architecture. In preprocessing,
we experimented with two techniques, SMOTE and Class Weights to counter the
imbalance between classes. Once we are happy with the quality of our input
data, we proceed to choosing the optimal deep learning architecture for this
task. Given the quality and quantity of data we have been given, we found that
the addition of CNN layer provides very little to no additional improvement to
our model's performance and sometimes even lead to a decrease in our F1-score.
In the end, the deep learning architecture that gives us the highest macro
F1-score is a simple BiLSTM-CNN.
Learning Symmetric and Asymmetric Steganography via Adversarial Training
Authors: Zheng Li, Ge Han, Yunqing Wei, Shanqing Guo
Date: 2019-03-13 00:00:00
Tasks: Decoder

Steganography refers to the art of concealing secret messages within multiple media carriers so that an eavesdropper is unable to detect the presence and content of the hidden messages. In this paper, we firstly propose a novel key-dependent steganographic scheme that achieves steganographic objectives with adversarial training. Symmetric (secret-key) and Asymmetric (public-key) steganographic scheme are separately proposed and each scheme is successfully designed and implemented. We show that these encodings produced by our scheme improve the invisibility by 20% than previous deep-leanring-based work, and further that perform competitively remarkable undetectability 25% better than classic steganographic algorithms. Finally, we simulated our scheme in a real situation where the decoder achieved an accuracy of more than 98% of the original message.
Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
Authors: Gil Kur, Yuval Dagan, Alexander Rakhlin
Date: 2019-03-13 00:00:00
Tasks: Density Estimation, regression

In this paper, we study two problems: (1) estimation of a $d$-dimensional log-concave distribution and (2) bounded multivariate convex regression with random design with an underlying log-concave density or a compactly supported distribution with a continuous density. First, we show that for all $d \ge 4$ the maximum likelihood estimators of both problems achieve an optimal risk of $\Theta_d(n^{-2/(d+1)})$ (up to a logarithmic factor) in terms of squared Hellinger distance and $L_2$ squared distance, respectively. Previously, the optimality of both these estimators was known only for $d\le 3$. We also prove that the $\epsilon$-entropy numbers of the two aforementioned families are equal up to logarithmic factors. We complement these results by proving a sharp bound $\Theta_d(n^{-2/(d+4)})$ on the minimax rate (up to logarithmic factors) with respect to the total variation distance. Finally, we prove that estimating a log-concave density - even a uniform distribution on a convex set - up to a fixed accuracy requires the number of samples \emph{at least} exponential in the dimension. We do that by improving the dimensional constant in the best known lower bound for the minimax rate from $2^{-d}\cdot n^{-2/(d+1)}$ to $c\cdot n^{-2/(d+1)}$ (when $d\geq 2$).
Efficient Search-Based Weighted Model Integration
Authors: Zhe Zeng, Guy Van Den Broeck
Date: 2019-03-13 00:00:00
Tasks: Computational Efficiency, model, Probabilistic Programming

Weighted model integration (WMI) extends Weighted model counting (WMC) to the integration of functions over mixed discrete-continuous domains. It has shown tremendous promise for solving inference problems in graphical models and probabilistic programming. Yet, state-of-the-art tools for WMI are limited in terms of performance and ignore the independence structure that is crucial to improving efficiency. To address this limitation, we propose an efficient model integration algorithm for theories with tree primal graphs. We exploit the sparse graph structure by using search to performing integration. Our algorithm greatly improves the computational efficiency on such problems and exploits context-specific independence between variables. Experimental results show dramatic speedups compared to existing WMI solvers on problems with tree-shaped dependencies.
A Framework for On-line Learning of Underwater Vehicles Dynamic Models
Authors: Bilal Wehbe, Marc Hildebrandt, Frank Kirchner
Date: 2019-03-13 00:00:00
Tasks: Incremental Learning, regression

Learning the dynamics of robots from data can help achieve more accurate
tracking controllers, or aid their navigation algorithms. However, when the
actual dynamics of the robots change due to external conditions, on-line
adaptation of their models is required to maintain high fidelity performance.
In this work, a framework for on-line learning of robot dynamics is developed
to adapt to such changes. The proposed framework employs an incremental support
vector regression method to learn the model sequentially from data streams. In
combination with the incremental learning, strategies for including and
forgetting data are developed to obtain better generalization over the whole
state space. The framework is tested in simulation and real experimental
scenarios demonstrating its adaptation capabilities to changes in the robot's
dynamics.
Asymmetric Residual Neural Network for Accurate Human Activity Recognition
Authors: Jun Long, WuQing Sun, Zhan Yang, Osolo Ian Raymond
Date: 2019-03-13 00:00:00
Tasks: Activity Recognition, Human Activity Recognition

Human Activity Recognition (HAR) using deep neural network has become a hot topic in human-computer interaction. Machine can effectively identify human naturalistic activities by learning from a large collection of sensor data. Activity recognition is not only an interesting research problem, but also has many real-world practical applications. Based on the success of residual networks in achieving a high level of aesthetic representation of the automatic learning, we propose a novel \textbf{A}symmetric \textbf{R}esidual \textbf{N}etwork, named ARN. ARN is implemented using two identical path frameworks consisting of (1) a short time window, which is used to capture spatial features, and (2) a long time window, which is used to capture fine temporal features. The long time window path can be made very lightweight by reducing its channel capacity, yet still being able to learn useful temporal representations for activity recognition. In this paper, we mainly focus on proposing a new model to improve the accuracy of HAR. In order to demonstrate the effectiveness of ARN model, we carried out extensive experiments on benchmark datasets (i.e., OPPORTUNITY, UniMiB-SHAR) and compared with some conventional and state-of-the-art learning-based methods. Then, we discuss the influence of networks parameters on performance to provide insights about its optimization. Results from our experiments show that ARN is effective in recognizing human activities via wearable datasets.
Transmission Matrix Inference via Pseudolikelihood Decimation
Authors: Daniele Ancora, Luca Leuzzi
Date: 2019-03-13 00:00:00

One of the biggest challenges in the field of biomedical imaging is the
comprehension and the exploitation of the photon scattering through disordered
media. Many studies have pursued the solution to this puzzle, achieving
light-focusing control or reconstructing images in complex media. In the
present work, we investigate how statistical inference helps the calculation of
the transmission matrix in a complex scrambling environment, enabling its usage
like a normal optical element. We convert a linear input-output transmission
problem into a statistical formulation based on pseudolikelihood maximization,
learning the coupling matrix via random sampling of intensity realizations. Our
aim is to uncover insights from the scattering problem, encouraging the
development of novel imaging techniques for better medical investigations,
borrowing a number of statistical tools from spin-glass theory.
Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems
Authors: Kleanthis Malialis, Sam Devlin, Daniel Kudenko
Date: 2019-03-13 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Real-world congestion problems (e.g. traffic congestion) are typically very
complex and large-scale. Multiagent reinforcement learning (MARL) is a
promising candidate for dealing with this emerging complexity by providing an
autonomous and distributed solution to these problems. However, there are three
limiting factors that affect the deployability of MARL approaches to congestion
problems. These are learning time, scalability and decentralised coordination
i.e. no communication between the learning agents. In this paper we introduce
Resource Abstraction, an approach that addresses these challenges by allocating
the available resources into abstract groups. This abstraction creates new
reward functions that provide a more informative signal to the learning agents
and aid the coordination amongst them. Experimental work is conducted on two
benchmark domains from the literature, an abstract congestion problem and a
realistic traffic congestion problem. The current state-of-the-art for solving
multiagent congestion problems is a form of reward shaping called difference
rewards. We show that the system using Resource Abstraction significantly
improves the learning speed and scalability, and achieves the highest possible
or near-highest joint performance/social welfare for both congestion problems
in large-scale scenarios involving up to 1000 reinforcement learning agents.
An Exponential Efron-Stein Inequality for Lq Stable Learning Rules
Authors: Karim Abou-Moustafa, Csaba Szepesvari
Date: 2019-03-12 00:00:00
Tasks: Generalization Bounds

There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in the literature. On one hand, the literature seems to suggest that exponential generalization bounds for the estimated risk, which are optimal, can be only obtained through stringent, distribution independent and computationally intractable notions of stability such as uniform stability. On the other hand, it seems that weaker notions of stability such as hypothesis stability, although it is distribution dependent and more amenable to computation, can only yield polynomial generalization bounds for the estimated risk, which are suboptimal. In this paper, we address the gap between these two regimes of results. In particular, the main question we address here is \emph{whether it is possible to derive exponential generalization bounds for the estimated risk using a notion of stability that is computationally tractable and distribution dependent, but weaker than uniform stability. Using recent advances in concentration inequalities, and using a notion of stability that is weaker than uniform stability but distribution dependent and amenable to computation, we derive an exponential tail bound for the concentration of the estimated risk of a hypothesis returned by a general learning rule, where the estimated risk is expressed in terms of either the resubstitution estimate (empirical error), or the deleted (or, leave-one-out) estimate. As an illustration, we derive exponential tail bounds for ridge regression with unbounded responses, where we show how stability changes with the tail behavior of the response variables.