pwc-archive/papers-with-abstracts huggingface dataset:
Detecting Online Hate Speech Using Context Aware Models
Authors: Lei Gao, Ruihong Huang
Date: 2017-10-20 00:00:00
Proceeding: RANLP 2017 9
Tasks: Hate Speech Detection

In the wake of a polarizing election, the cyber world is laden with hate
speech. Context accompanying a hate speech text is useful for identifying hate
speech, which however has been largely overlooked in existing datasets and hate
speech detection models. In this paper, we provide an annotated corpus of hate
speech with context information well kept. Then we propose two types of hate
speech detection models that incorporate context information, a logistic
regression model with context features and a neural network model with learning
components for context. Our evaluation shows that both models outperform a
strong baseline by around 3% to 4% in F1 score and combining these two models
further improve the performance by another 7% in F1 score.
Domain Adaptation Using Adversarial Learning for Autonomous Navigation
Authors: Jaeyoon Yoo, Yongjun Hong, Yung-Kyun Noh, Sungroh Yoon
Date: 2017-12-11 00:00:00
Tasks: Autonomous Navigation, Domain Adaptation, Image Generation

Autonomous navigation has become an increasingly popular machine learning
application. Recent advances in deep learning have also resulted in great
improvements to autonomous navigation. However, prior outdoor autonomous
navigation depends on various expensive sensors or large amounts of real
labeled data which is difficult to acquire and sometimes erroneous. The
objective of this study is to train an autonomous navigation model that uses a
simulator (instead of real labeled data) and an inexpensive monocular camera.
In order to exploit the simulator satisfactorily, our proposed method is based
on domain adaptation with adversarial learning. Specifically, we propose our
model with 1) a dilated residual block in the generator, 2) cycle loss, and 3)
style loss to improve the adversarial learning performance for satisfactory
domain adaptation. In addition, we perform a theoretical analysis that supports
the justification of our proposed method. We present empirical results of
navigation in outdoor courses with various intersections using a commercial
radio controlled car. We observe that our proposed method allows us to learn a
favorable navigation model by generating images with realistic textures. To the
best of our knowledge, this is the first work to apply domain adaptation with
adversarial learning to autonomous navigation in real outdoor environments. Our
proposed method can also be applied to precise image generation or other
robotic tasks.
Recognizing Explicit and Implicit Hate Speech Using a Weakly Supervised Two-path Bootstrapping Approach
Authors: Lei Gao, Alexis Kuppersmith, Ruihong Huang
Date: 2017-10-20 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: General Classification, Hate Speech Detection

In the wake of a polarizing election, social media is laden with hateful
content. To address various limitations of supervised hate speech
classification methods including corpus bias and huge cost of annotation, we
propose a weakly supervised two-path bootstrapping approach for an online hate
speech detection model leveraging large-scale unlabeled data. This system
significantly outperforms hate speech detection systems that are trained in a
supervised manner using manually annotated data. Applying this model on a large
quantity of tweets collected before, after, and on election day reveals
motivations and patterns of inflammatory language.
Improved Algorithms for Collaborative PAC Learning
Authors: Huy L. Nguyen, Lydia Zakynthinou
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: All, PAC learning

We study a recent model of collaborative PAC learning where $k$ players with
$k$ different tasks collaborate to learn a single classifier that works for all
tasks. Previous work showed that when there is a classifier that has very small
error on all tasks, there is a collaborative algorithm that finds a single
classifier for all tasks and has $O((\ln (k))^2)$ times the worst-case sample
complexity for learning a single task. In this work, we design new algorithms
for both the realizable and the non-realizable setting, having sample
complexity only $O(\ln (k))$ times the worst-case sample complexity for
learning a single task. The sample complexity upper bounds of our algorithms
match previous lower bounds and in some range of parameters are even better
than previous algorithms that are allowed to output different classifiers for
different tasks.
Opening the black box of deep learning
Authors: Dian Lei, Xiaoxiao Chen, Jianfei Zhao
Date: 2018-05-22 00:00:00
Tasks: Deep Learning

The great success of deep learning shows that its technology contains
profound truth, and understanding its internal mechanism not only has important
implications for the development of its technology and effective application in
various fields, but also provides meaningful insights into the understanding of
human brain mechanism. At present, most of the theoretical research on deep
learning is based on mathematics. This dissertation proposes that the neural
network of deep learning is a physical system, examines deep learning from
three different perspectives: microscopic, macroscopic, and physical world
views, answers multiple theoretical puzzles in deep learning by using physics
principles. For example, from the perspective of quantum mechanics and
statistical physics, this dissertation presents the calculation methods for
convolution calculation, pooling, normalization, and Restricted Boltzmann
Machine, as well as the selection of cost functions, explains why deep learning
must be deep, what characteristics are learned in deep learning, why
Convolutional Neural Networks do not have to be trained layer by layer, and the
limitations of deep learning, etc., and proposes the theoretical direction and
basis for the further development of deep learning now and in the future. The
brilliance of physics flashes in deep learning, we try to establish the deep
learning technology based on the scientific theory of physics.
Learning sentence embeddings using Recursive Networks
Authors: Anson Bastos
Date: 2018-05-22 00:00:00
Tasks: POS, Reverse Dictionary, Sentence, Sentence Embeddings

Learning sentence vectors that generalise well is a challenging task. In this
paper we compare three methods of learning phrase embeddings: 1) Using LSTMs,
2) using recursive nets, 3) A variant of the method 2 using the POS information
of the phrase. We train our models on dictionary definitions of words to obtain
a reverse dictionary application similar to Felix et al. [1]. To see if our
embeddings can be transferred to a new task we also train and test on the
rotten tomatoes dataset [2]. We train keeping the sentence embeddings fixed as
well as with fine tuning.
Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators
Authors: Shereen Oraby, Lena Reed, Shubhangi Tandon, T. S. Sharath, Stephanie Lukin, Marilyn Walker
Date: 2018-05-22 00:00:00
Proceeding: WS 2018 7

Natural language generators for task-oriented dialogue must effectively
realize system dialogue actions and their associated semantics. In many
applications, it is also desirable for generators to control the style of an
utterance. To date, work on task-oriented neural generation has primarily
focused on semantic fidelity rather than achieving stylistic goals, while work
on style has been done in contexts where it is difficult to measure content
preservation. Here we present three different sequence-to-sequence models and
carefully test how well they disentangle content and style. We use a
statistical generator, Personage, to synthesize a new corpus of over 88,000
restaurant domain utterances whose style varies according to models of
personality, giving us total control over both the semantic content and the
stylistic variation in the training data. We then vary the amount of explicit
stylistic supervision given to the three models. We show that our most explicit
model can simultaneously achieve high fidelity to both semantic and stylistic
goals: this model adds a context vector of 36 stylistic parameters as input to
the hidden state of the encoder at each time step, showing the benefits of
explicit stylistic supervision, even when the amount of training data is large.
A Solvable High-Dimensional Model of GAN
Authors: Chuang Wang, Hong Hu, Yue M. Lu
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: model, Vocal Bursts Intensity Prediction

We present a theoretical analysis of the training process for a single-layer GAN fed by high-dimensional input data. The training dynamics of the proposed model at both microscopic and macroscopic scales can be exactly analyzed in the high-dimensional limit. In particular, we prove that the macroscopic quantities measuring the quality of the training process converge to a deterministic process characterized by an ordinary differential equation (ODE), whereas the microscopic states containing all the detailed weights remain stochastic, whose dynamics can be described by a stochastic differential equation (SDE). This analysis provides a new perspective different from recent analyses in the limit of small learning rate, where the microscopic state is always considered deterministic, and the contribution of noise is ignored. From our analysis, we show that the level of the background noise is essential to the convergence of the training process: setting the noise level too strong leads to failure of feature recovery, whereas setting the noise too weak causes oscillation. Although this work focuses on a simple copy model of GAN, we believe the analysis methods and insights developed here would prove useful in the theoretical understanding of other variants of GANs with more advanced training algorithms.
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
Authors: Hongge Chen, huan zhang, Pin-Yu Chen, Jin-Feng Yi, Cho-Jui Hsieh
Date: 2017-12-06 00:00:00
Proceeding: ACL 2018 7
Tasks: Caption Generation, Decoder, Image Captioning

Visual language grounding is widely studied in modern neural image captioning
systems, which typically adopts an encoder-decoder framework consisting of two
principal components: a convolutional neural network (CNN) for image feature
extraction and a recurrent neural network (RNN) for language caption
generation. To study the robustness of language grounding to adversarial
perturbations in machine vision and perception, we propose Show-and-Fool, a
novel algorithm for crafting adversarial examples in neural image captioning.
The proposed algorithm provides two evaluation approaches, which check whether
neural image captioning systems can be mislead to output some randomly chosen
captions or keywords. Our extensive experiments show that our algorithm can
successfully craft visually-similar adversarial examples with randomly targeted
captions or keywords, and the adversarial examples can be made highly
transferable to other image captioning systems. Consequently, our approach
leads to new robustness implications of neural image captioning and novel
insights in visual language grounding.
How To Solve Moral Conundrums with Computability Theory
Authors: Min Baek
Date: 2018-05-22 00:00:00
Tasks: Ethics, Philosophy

Various moral conundrums plague population ethics: the Non-Identity Problem, the Procreation Asymmetry, the Repugnant Conclusion, and more. I argue that the aforementioned moral conundrums have a structure neatly accounted for, and solved by, some ideas in computability theory. I introduce a mathematical model based on computability theory and show how previous arguments pertaining to these conundrums fit into the model. This paper proceeds as follows. First, I do a very brief survey of the history of computability theory in moral philosophy. Second, I follow various papers, and show how their arguments fit into, or don't fit into, our model. Third, I discuss the implications of our model to the question why the human race should or should not continue to exist. Finally, I show that our model may be interpreted according to a Confucian-Taoist moral principle.
Solving high-dimensional partial differential equations using deep learning
Authors: Jiequn Han, Arnulf Jentzen, Weinan E
Date: 2017-07-09 00:00:00
Tasks: Deep Learning, Deep Reinforcement Learning, Reinforcement Learning, Vocal Bursts Intensity Prediction

Developing algorithms for solving high-dimensional partial differential
equations (PDEs) has been an exceedingly difficult task for a long time, due to
the notoriously difficult problem known as the "curse of dimensionality". This
paper introduces a deep learning-based approach that can handle general
high-dimensional parabolic PDEs. To this end, the PDEs are reformulated using
backward stochastic differential equations and the gradient of the unknown
solution is approximated by neural networks, very much in the spirit of deep
reinforcement learning with the gradient acting as the policy function.
Numerical results on examples including the nonlinear Black-Scholes equation,
the Hamilton-Jacobi-Bellman equation, and the Allen-Cahn equation suggest that
the proposed algorithm is quite effective in high dimensions, in terms of both
accuracy and cost. This opens up new possibilities in economics, finance,
operational research, and physics, by considering all participating agents,
assets, resources, or particles together at the same time, instead of making ad
hoc assumptions on their inter-relationships.
Nearest neighbor density functional estimation from inverse Laplace transform
Authors: J. Jon Ryu, Shouvik Ganguly, Young-Han Kim, Yung-Kyun Noh, Daniel D. Lee
Date: 2018-05-22 00:00:00

A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f.$ Some instantiations of the proposed estimator recover existing $k$-nearest neighbor based estimators of Shannon and R\'enyi entropies and Kullback--Leibler and R\'enyi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_2$-consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities.
Constrained Sparse Subspace Clustering with Side-Information
Authors: Chun-Guang Li, Junjian Zhang, Jun Guo
Date: 2018-05-21 00:00:00
Tasks: Clustering

Subspace clustering refers to the problem of segmenting high dimensional data
drawn from a union of subspaces into the respective subspaces. In some
applications, partial side-information to indicate "must-link" or "cannot-link"
in clustering is available. This leads to the task of subspace clustering with
side-information. However, in prior work the supervision value of the
side-information for subspace clustering has not been fully exploited. To this
end, in this paper, we present an enhanced approach for constrained subspace
clustering with side-information, termed Constrained Sparse Subspace Clustering
plus (CSSC+), in which the side-information is used not only in the stage of
learning an affinity matrix but also in the stage of spectral clustering.
Moreover, we propose to estimate clustering accuracy based on the partial
side-information and theoretically justify the connection to the ground-truth
clustering accuracy in terms of the Rand index. We conduct experiments on three
cancer gene expression datasets to validate the effectiveness of our proposals.
Reducing Parameter Space for Neural Network Training
Authors: Tong Qin, Ling Zhou, Dongbin Xiu
Date: 2018-05-22 00:00:00

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a bounded interval, as opposed to the real line. We show that the NNs in the reduced parameter space are mathematically equivalent to the standard NNs with parameters in the whole space. The reduced parameter space shall facilitate the optimization procedure for the network training, as the search space becomes (much) smaller. We demonstrate the improved training performance using numerical examples.
Interpreting Blackbox Models via Model Extraction
Authors: Osbert Bastani, Carolyn Kim, Hamsa Bastani
Date: 2017-05-23 00:00:00
Tasks: model, Model extraction

Interpretability has become incredibly important as machine learning is
increasingly used to inform consequential decisions. We propose to construct
global explanations of complex, blackbox models in the form of a decision tree
approximating the original model---as long as the decision tree is a good
approximation, then it mirrors the computation performed by the blackbox model.
We devise a novel algorithm for extracting decision tree explanations that
actively samples new training points to avoid overfitting. We evaluate our
algorithm on a random forest to predict diabetes risk and a learned controller
for cart-pole. Compared to several baselines, our decision trees are both
substantially more accurate and equally or more interpretable based on a user
study. Finally, we describe several insights provided by our interpretations,
including a causal issue validated by a physician.
Scalable Centralized Deep Multi-Agent Reinforcement Learning via Policy Gradients
Authors: Arbaaz Khan, Clark Zhang, Daniel D. Lee, Vijay Kumar, Alejandro Ribeiro
Date: 2018-05-22 00:00:00
Tasks: Deep Reinforcement Learning, Distributed Optimization, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we explore using deep reinforcement learning for problems with
multiple agents. Most existing methods for deep multi-agent reinforcement
learning consider only a small number of agents. When the number of agents
increases, the dimensionality of the input and control spaces increase as well,
and these methods do not scale well. To address this, we propose casting the
multi-agent reinforcement learning problem as a distributed optimization
problem. Our algorithm assumes that for multi-agent settings, policies of
individual agents in a given population live close to each other in parameter
space and can be approximated by a single policy. With this simple assumption,
we show our algorithm to be extremely effective for reinforcement learning in
multi-agent settings. We demonstrate its effectiveness against existing
comparable approaches on co-operative and competitive tasks.
Information-theoretic Limits for Community Detection in Network Models
Authors: Chuyang Ke, Jean Honorio
Date: 2018-02-16 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Community Detection, Stochastic Block Model

We analyze the information-theoretic limits for the recovery of node labels
in several network models. This includes the Stochastic Block Model, the
Exponential Random Graph Model, the Latent Space Model, the Directed
Preferential Attachment Model, and the Directed Small-world Model. For the
Stochastic Block Model, the non-recoverability condition depends on the
probabilities of having edges inside a community, and between different
communities. For the Latent Space Model, the non-recoverability condition
depends on the dimension of the latent space, and how far and spread are the
communities in the latent space. For the Directed Preferential Attachment Model
and the Directed Small-world Model, the non-recoverability condition depends on
the ratio between homophily and neighborhood size. We also consider dynamic
versions of the Stochastic Block Model and the Latent Space Model.
Large Scale computation of Means and Clusters for Persistence Diagrams using Optimal Transport
Authors: Théo Lacombe, Marco Cuturi, Steve Oudot
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Clustering

Persistence diagrams (PDs) are now routinely used to summarize the underlying
topology of complex data. Despite several appealing properties, incorporating
PDs in learning pipelines can be challenging because their natural geometry is
not Hilbertian. Indeed, this was recently exemplified in a string of papers
which show that the simple task of averaging a few PDs can be computationally
prohibitive. We propose in this article a tractable framework to carry out
standard tasks on PDs at scale, notably evaluating distances, estimating
barycenters and performing clustering. This framework builds upon a
reformulation of PD metrics as optimal transport (OT) problems. Doing so, we
can exploit recent computational advances: the OT problem on a planar grid,
when regularized with entropy, is convex can be solved in linear time using the
Sinkhorn algorithm and convolutions. This results in scalable computations that
can stream on GPUs. We demonstrate the efficiency of our approach by carrying
out clustering with diagrams metrics on several thousands of PDs, a scale never
seen before in the literature.
Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
Authors: Haonan Yu, Xiaochen Lian, Haichao Zhang, Wei Xu
Date: 2018-05-22 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Sentence Embeddings

Recently there has been a rising interest in training agents, embodied in
virtual environments, to perform language-directed tasks by deep reinforcement
learning. In this paper, we propose a simple but effective neural language
grounding module for embodied agents that can be trained end to end from
scratch taking raw pixels, unstructured linguistic commands, and sparse rewards
as the inputs. We model the language grounding process as a language-guided
transformation of visual features, where latent sentence embeddings are used as
the transformation matrices. In several language-directed navigation tasks that
feature challenging partial observability and require simple reasoning, our
module significantly outperforms the state of the art. We also release
XWorld3D, an easy-to-customize 3D environment that can potentially be modified
to evaluate a variety of embodied agents.