pwc-archive/papers-with-abstracts huggingface dataset:
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
Authors: Sarath Chandar, Chinnadhurai Sankar, Eugene Vorontsov, Samira Ebrahimi Kahou, Yoshua Bengio
Date: 2019-01-22 00:00:00

Modelling long-term dependencies is a challenge for recurrent neural
networks. This is primarily due to the fact that gradients vanish during
training, as the sequence length increases. Gradients can be attenuated by
transition operators and are attenuated or dropped by activation functions.
Canonical architectures like LSTM alleviate this issue by skipping information
through a memory mechanism. We propose a new recurrent architecture
(Non-saturating Recurrent Unit; NRU) that relies on a memory mechanism but
forgoes both saturating activation functions and saturating gates, in order to
further alleviate vanishing gradients. In a series of synthetic and real world
tasks, we demonstrate that the proposed model is the only model that performs
among the top 2 models across all tasks with and without long-term
dependencies, when compared against a range of other architectures.
Network connectivity dynamics affect the evolution of culturally transmitted variants
Authors: José Segovia Martín, Bradley Walker, Nicolas Fay, Monica Tamariz
Date: 2019-02-09 00:00:00

The distribution of cultural variants in a population is shaped by both
neutral evolutionary dynamics and by selection pressures, which include several
individual cognitive biases, demographic factors and social network structures.
The temporal dynamics of social network connectivity, i.e. the order in which
individuals in a population interact with each other, has been largely
unexplored. In this paper we investigate how, in a fully connected social
network, connectivity dynamics, alone and in interaction with different
cognitive biases, affect the evolution of cultural variants. Using agent-based
computer simulations, we manipulate population connectivity dynamics (early,
middle and late full-population connectivity); content bias, or a preference
for high-quality variants; coordination bias, or whether agents tend to use
self-produced variants (egocentric bias), or to switch to variants observed in
others (allocentric bias); and memory size, or the number of items that agents
can store in their memory. We show that connectivity dynamics affect the
time-course of variant spread, with lower connectivity slowing down convergence
of the population onto a single cultural variant. We also show that, compared
to a neutral evolutionary model, content bias accelerates convergence and
amplifies the effects of connectivity dynamics, whilst larger memory size and
coordination bias, especially egocentric bias, slow down convergence.
Furthermore, connectivity dynamics affect the frequency of high quality
variants (adaptiveness), with late connectivity populations showing bursts of
rapid change in adaptiveness followed by periods of relatively slower change,
and early connectivity populations following a single-peak evolutionary
dynamic. In this way, we provide for the first time a direct connection between
the order of agents' interactions and punctuational evolution.
Non-Suicidal Self-Injury Online Posts: Implications for Mental Health Professionals
Authors: Mandy M. Greaves, Cass Dykeman
Date: 2019-02-02 00:00:00

While non-suicidal self-injury (NSSI) is not a new phenomenon, there is still a limited yet little is still known about understanding of the behavior, the intent behind the behavior and what the individuals themselves say about their behavior. This study collected pro-NSSI public blog posts from Reddit on pro-NSSI and analyzed the content linguistically using LIWC software, in order to examine the use of NSSI specific words, linguistic properties and the psychological linguistic properties. were examined. The results inform current counseling practices by dispelling myths and providing insight into the inner world of people who engage in use NSSII to cope. The most frequently appearing category of For NSSI specific words categories, in the Reddit blogs was the reasons in which one engagesfor engaging in NSSI was the most frequently used in the Reddit blogs. The linguistic properties found in the analysis reflected the predicted results; authors of pro-NSSI posts used demonstrated expected results of first-person singular pronouns extensively, which indicatesing high levels of mental health distress and isolation. The psychological linguistic properties that could be observed of in these public Reddit posts were dominantly in a negative emotional tone which demonstrates youth and impulsivity. The linguistic properties found when these posts were analyzed supports the work of earlier studies that dispelled common myths about NSSI that were circulating in the mental health community. These findings suggest that the language of people who engage in NSSI supports research findings in dispelling common myths about NSSI.
Increasing city safety awareness regarding disruptive traffic stream
Authors: Olivera Kotevska
Date: 2019-01-30 00:00:00
Tasks: Inductive logic programming

Transportation systems serve the people in essence, in this study we focus in
traffic information related to violation events to respond to safety
requirements of the cities. Traffic violation events have an important role in
city safety awareness and secure travel. In this work, we describe the use of
knowledge discovery from traffic violation reports in combination with
demographics approach using inductive logic programming to automatically
extract knowledge about traffic violation behavior and their impact on the
environment.
AD-VAT: An Asymmetric Dueling mechanism for learning Visual Active Tracking
Authors: Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, Yizhou Wang
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Reinforcement Learning

Visual Active Tracking (VAT) aims at following a target object by autonomously controlling the motion system of a tracker given visual observations. Previous work has shown that the tracker can be trained in a simulator via reinforcement learning and deployed in real-world scenarios. However, during training, such a method requires manually specifying the moving path of the target object to be tracked, which cannot ensure the tracker’s generalization on the unseen object moving patterns. To learn a robust tracker for VAT, in this paper, we propose a novel adversarial RL method which adopts an Asymmetric Dueling mechanism, referred to as AD-VAT. In AD-VAT, both the tracker and the target are approximated by end-to-end neural networks, and are trained via RL in a dueling/competitive manner: i.e., the tracker intends to lockup the target, while the target tries to escape from the tracker. They are asymmetric in that the target is aware of the tracker, but not vice versa. Specifically, besides its own observation, the target is fed with the tracker’s observation and action, and learns to predict the tracker’s reward as an auxiliary task. We show that such an asymmetric dueling mechanism produces a stronger target, which in turn induces a more robust tracker. To stabilize the training, we also propose a novel partial zero-sum reward for the tracker/target. The experimental results, in both 2D and 3D environments, demonstrate that the proposed method leads to a faster convergence in training and yields more robust tracking behaviors in different testing scenarios. For supplementary videos, see: https://www.youtube.com/playlist?list=PL9rZj4Mea7wOZkdajK1TsprRg8iUf51BS
A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
Authors: Pinghua Gong, Chang-Shui Zhang, Zhaosong Lu, Jianhua Huang, Jieping Ye
Date: 2013-03-18 00:00:00
Tasks: Sparse Learning

Non-convex sparsity-inducing penalties have recently received considerable
attentions in sparse learning. Recent theoretical investigations have
demonstrated their superiority over the convex counterparts in several sparse
learning settings. However, solving the non-convex optimization problems
associated with non-convex penalties remains a big challenge. A commonly used
approach is the Multi-Stage (MS) convex relaxation (or DC programming), which
relaxes the original non-convex problem to a sequence of convex problems. This
approach is usually not very practical for large-scale problems because its
computational cost is a multiple of solving a single convex problem. In this
paper, we propose a General Iterative Shrinkage and Thresholding (GIST)
algorithm to solve the nonconvex optimization problem for a large class of
non-convex penalties. The GIST algorithm iteratively solves a proximal operator
problem, which in turn has a closed-form solution for many commonly used
penalties. At each outer iteration of the algorithm, we use a line search
initialized by the Barzilai-Borwein (BB) rule that allows finding an
appropriate step size quickly. The paper also presents a detailed convergence
analysis of the GIST algorithm. The efficiency of the proposed algorithm is
demonstrated by extensive experiments on large-scale data sets.
Are Bitcoin Bubbles Predictable? Combining a Generalized Metcalfe's Law and the LPPLS Model
Authors: Spencer Wheatley, Didier Sornette, Tobias Huber, Max Reppen, Robert N. Gantner
Date: 2018-03-15 00:00:00
Tasks: Diagnostic

We develop a strong diagnostic for bubbles and crashes in bitcoin, by
analyzing the coincidence (and its absence) of fundamental and technical
indicators. Using a generalized Metcalfe's law based on network properties, a
fundamental value is quantified and shown to be heavily exceeded, on at least
four occasions, by bubbles that grow and burst. In these bubbles, we detect a
universal super-exponential unsustainable growth. We model this universal
pattern with the Log-Periodic Power Law Singularity (LPPLS) model, which
parsimoniously captures diverse positive feedback phenomena, such as herding
and imitation. The LPPLS model is shown to provide an ex-ante warning of market
instabilities, quantifying a high crash hazard and probabilistic bracket of the
crash time consistent with the actual corrections; although, as always, the
precise time and trigger (which straw breaks the camel's back) being exogenous
and unpredictable. Looking forward, our analysis identifies a substantial but
not unprecedented overvaluation in the price of bitcoin, suggesting many months
of volatile sideways bitcoin prices ahead (from the time of writing, March
2018).
PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
Authors: Zichao Long, Yiping Lu, Bin Dong
Date: 2018-11-30 00:00:00

Partial differential equations (PDEs) are commonly derived based on empirical observations. However, recent advances of technology enable us to collect and store massive amount of data, which offers new opportunities for data-driven discovery of PDEs. In this paper, we propose a new deep neural network, called PDE-Net 2.0, to discover (time-dependent) PDEs from observed dynamic data with minor prior knowledge on the underlying mechanism that drives the dynamics. The design of PDE-Net 2.0 is based on our earlier work \cite{Long2018PDE} where the original version of PDE-Net was proposed. PDE-Net 2.0 is a combination of numerical approximation of differential operators by convolutions and a symbolic multi-layer neural network for model recovery. Comparing with existing approaches, PDE-Net 2.0 has the most flexibility and expressive power by learning both differential operators and the nonlinear response function of the underlying PDE model. Numerical experiments show that the PDE-Net 2.0 has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.
Transfer and Exploration via the Information Bottleneck
Authors: Anirudh Goyal, Riashat Islam, DJ Strouse, Zafarali Ahmed, Hugo Larochelle, Matthew Botvinick, Yoshua Bengio, Sergey Levine
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Reinforcement Learning

A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned model with an information bottleneck, we can identify decision states by examining where the model accesses the goal state through the bottleneck. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision  states and through new regions of the state space.
Modular Latent Spaces for Shape Correspondences
Authors: Vignesh Ganapathi-Subramanian, Olga Diamanti, Leonidas J. Guibas
Date: 2018-06-30 00:00:00
Proceeding: Symposium in Geometry Processing 2018 2018 6

We consider the problem of transporting shape descriptors across shapes in a collection in a modular fashion, in order to establish correspondences between them. A common goal when mapping between multiple shapes is consistency, namely that compositions of maps along a cycle of shapes should be approximately an identity map. Existing attempts to enforce consistency typically require recomputing correspondences whenever a new shape is added to the collection, which can quickly become intractable. Instead, we propose an approach that is fully modular, where the bulk of the computation is done on each shape independently. To achieve this, we use intermediate nonlinear embedding spaces, computed individually on every shape; the embedding functions use ideas from diffusion geometry and capture how different descriptors on the same shape inter‐relate. We then establish linear mappings between the different embedding spaces, via a shared latent space. The introduction of nonlinear embeddings allows for more nuanced correspondences, while the modularity of the construction allows for parallelizable calculation and efficient addition of new shapes. We compare the performance of our framework to standard functional correspondence techniques and showcase the use of this framework to simple interpolation and extrapolation tasks.
Automated Variational Inference in Probabilistic Programming
Authors: David Wingate, Theophane Weber
Date: 2013-01-07 00:00:00
Tasks: Probabilistic Programming, Variational Inference

We present a new algorithm for approximate inference in probabilistic
programs, based on a stochastic gradient for variational programs. This method
is efficient without restrictions on the probabilistic program; it is
particularly practical for distributions which are not analytically tractable,
including highly structured distributions that arise in probabilistic programs.
We show how to automatically derive mean-field probabilistic programs and
optimize them, and demonstrate that our perspective improves inference
efficiency over other algorithms.
FreeLabel: A Publicly Available Annotation Tool based on Freehand Traces
Authors: Philipe A. Dias, Zhou Shen, Amy Tabb, Henry Medeiros
Date: 2019-02-18 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Large-scale annotation of image segmentation datasets is often prohibitively
expensive, as it usually requires a huge number of worker hours to obtain
high-quality results. Abundant and reliable data has been, however, crucial for
the advances on image understanding tasks achieved by deep learning models. In
this paper, we introduce FreeLabel, an intuitive open-source web interface that
allows users to obtain high-quality segmentation masks with just a few freehand
scribbles, in a matter of seconds. The efficacy of FreeLabel is quantitatively
demonstrated by experimental results on the PASCAL dataset as well as on a
dataset from the agricultural domain. Designed to benefit the computer vision
community, FreeLabel can be used for both crowdsourced or private annotation
and has a modular structure that can be easily adapted for any image dataset.
WIDER Face and Pedestrian Challenge 2018: Methods and Results
Authors: Chen Change Loy, Dahua Lin, Wanli Ouyang, Yuanjun Xiong, Shuo Yang, Qingqiu Huang, Dongzhan Zhou, Wei Xia, Quanquan Li, Ping Luo, Junjie Yan, Jian-Feng Wang, Zuoxin Li, Ye Yuan, Boxun Li, Shuai Shao, Gang Yu, Fangyun Wei, Xiang Ming, Dong Chen, Shifeng Zhang, Cheng Chi, Zhen Lei, Stan Z. Li, Hongkai Zhang, Bingpeng Ma, Hong Chang, Shiguang Shan, Xilin Chen, Wu Liu, Boyan Zhou, Huaxiong Li, Peng Cheng, Tao Mei, Artem Kukharenko, Artem Vasenin, Nikolay Sergievskiy, Hua Yang, Liangqi Li, Qiling Xu, Yuan Hong, Lin Chen, Mingjun Sun, Yirong Mao, Shiying Luo, Yongjun Li, Ruiping Wang, Qiaokang Xie, Ziyang Wu, Lei Lu, Yiheng Liu, Wengang Zhou
Date: 2019-02-19 00:00:00
Tasks: Face Detection, Pedestrian Detection, Person Search, valid

This paper presents a review of the 2018 WIDER Challenge on Face and
Pedestrian. The challenge focuses on the problem of precise localization of
human faces and bodies, and accurate association of identities. It comprises of
three tracks: (i) WIDER Face which aims at soliciting new approaches to advance
the state-of-the-art in face detection, (ii) WIDER Pedestrian which aims to
find effective and efficient approaches to address the problem of pedestrian
detection in unconstrained environments, and (iii) WIDER Person Search which
presents an exciting challenge of searching persons across 192 movies. In
total, 73 teams made valid submissions to the challenge tracks. We summarize
the winning solutions for all three tracks. and present discussions on open
problems and potential research directions in these topics.
Detector-in-Detector: Multi-Level Analysis for Human-Parts
Authors: Xiaojie Li, Lu Yang, Qing Song, Fuqiang Zhou
Date: 2019-02-19 00:00:00
Tasks: Body Detection, Face Detection, Hand Detection, object-detection, Object Detection

Vision-based person, hand or face detection approaches have achieved
incredible success in recent years with the development of deep convolutional
neural network (CNN). In this paper, we take the inherent correlation between
the body and body parts into account and propose a new framework to boost up
the detection performance of the multi-level objects. In particular, we adopt a
region-based object detection structure with two carefully designed detectors
to separately pay attention to the human body and body parts in a
coarse-to-fine manner, which we call Detector-in-Detector network (DID-Net).
The first detector is designed to detect human body, hand, and face. The second
detector, based on the body detection results of the first detector, mainly
focus on the detection of small hand and face inside each body. The framework
is trained in an end-to-end way by optimizing a multi-task loss. Due to the
lack of human body, face and hand detection dataset, we have collected and
labeled a new large dataset named Human-Parts with 14,962 images and 106,879
annotations. Experiments show that our method can achieve excellent performance
on Human-Parts.
Learning to Generalize from Sparse and Underspecified Rewards
Authors: Rishabh Agarwal, Chen Liang, Dale Schuurmans, Mohammad Norouzi
Date: 2019-02-19 00:00:00
Tasks: Bayesian Optimization, Semantic Parsing

We consider the problem of learning from sparse and underspecified rewards, where an agent receives a complex input, such as a natural language instruction, and needs to generate a complex response, such as an action sequence, while only receiving binary success-failure feedback. Such success-failure rewards are often underspecified: they do not distinguish between purposeful and accidental success. Generalization from underspecified rewards hinges on discounting spurious trajectories that attain accidental success, while learning from sparse feedback requires effective exploration. We address exploration by using a mode covering direction of KL divergence to collect a diverse set of successful trajectories, followed by a mode seeking KL divergence to train a robust policy. We propose Meta Reward Learning (MeRL) to construct an auxiliary reward function that provides more refined feedback for learning. The parameters of the auxiliary reward function are optimized with respect to the validation performance of a trained policy. The MeRL approach outperforms our alternative reward learning technique based on Bayesian Optimization, and achieves the state-of-the-art on weakly-supervised semantic parsing. It improves previous work by 1.2% and 2.4% on WikiTableQuestions and WikiSQL datasets respectively.
2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs
Authors: Amirhossein Taghvaei, Amin Jalali
Date: 2019-02-19 00:00:00

We provide a framework to approximate the 2-Wasserstein distance and the
optimal transport map, amenable to efficient training as well as statistical
and geometric analysis. With the quadratic cost and considering the Kantorovich
dual form of the optimal transportation problem, the Brenier theorem states
that the optimal potential function is convex and the optimal transport map is
the gradient of the optimal potential function. Using this geometric structure,
we restrict the optimization problem to different parametrized classes of
convex functions and pay special attention to the class of input-convex neural
networks. We analyze the statistical generalization and the discriminative
power of the resulting approximate metric, and we prove a restricted
moment-matching property for the approximate optimal map. Finally, we discuss a
numerical algorithm to solve the restricted optimization problem and provide
numerical experiments to illustrate and compare the proposed approach with the
established regularization-based approaches. We further discuss practical
implications of our proposal in a modular and interpretable design for GANs
which connects the generator training with discriminator computations to allow
for learning an overall composite generator.
Approximating Continuous Functions on Persistence Diagrams Using Template Functions
Authors: Jose A. Perea, Elizabeth Munch, Firas A. Khasawneh
Date: 2019-02-19 00:00:00
Tasks: Time Series Analysis, Topological Data Analysis

The persistence diagram is an increasingly useful tool from Topological Data Analysis, but its use alongside typical machine learning techniques requires mathematical finesse. The most success to date has come from methods that map persistence diagrams into vector spaces, in a way which maximizes the structure preserved. This process is commonly referred to as featurization. In this paper, we describe a mathematical framework for featurization called \emph{template functions}, and we show that it addresses the problem of approximating continuous functions on compact subsets of the space of persistence diagrams. Specifically, we begin by characterizing relative compactness with respect to the bottleneck distance, and then provide explicit theoretical methods for constructing compact-open dense subsets of continuous functions on persistence diagrams. These dense subsets -- obtained via template functions -- are leveraged for supervised learning tasks with persistence diagrams. Specifically, we test the method for classification and regression algorithms on several examples including shape data and dynamical systems.