pwc-archive/papers-with-abstracts huggingface dataset:
Controversy Rules - Discovering Regions Where Classifiers (Dis-)Agree Exceptionally
Authors: Oren Zeev-Ben-Mordehai, Wouter Duivesteijn, Mykola Pechenizkiy
Date: 2018-08-22 00:00:00
Tasks: General Classification, General Knowledge

Finding regions for which there is higher controversy among different
classifiers is insightful with regards to the domain and our models. Such
evaluation can falsify assumptions, assert some, or also, bring to the
attention unknown phenomena. The present work describes an algorithm, which is
based on the Exceptional Model Mining framework, and enables that kind of
investigations. We explore several public datasets and show the usefulness of
this approach in classification tasks. We show in this paper a few interesting
observations about those well explored datasets, some of which are general
knowledge, and other that as far as we know, were not reported before.
Finding Good Representations of Emotions for Text Classification
Authors: Ji Ho Park
Date: 2018-08-22 00:00:00
Tasks: Abusive Language, Classification, Emotion Recognition, General Classification, Robust classification, Sentence, text-classification, Text Classification

It is important for machines to interpret human emotions properly for better
human-machine communications, as emotion is an essential part of human-to-human
communications. One aspect of emotion is reflected in the language we use. How
to represent emotions in texts is a challenge in natural language processing
(NLP). Although continuous vector representations like word2vec have become the
new norm for NLP problems, their limitations are that they do not take emotions
into consideration and can unintentionally contain bias toward certain
identities like different genders.
  This thesis focuses on improving existing representations in both word and
sentence levels by explicitly taking emotions inside text and model bias into
account in their training process. Our improved representations can help to
build more robust machine learning models for affect-related text
classification like sentiment/emotion analysis and abusive language detection.
  We first propose representations called emotional word vectors (EVEC), which
is learned from a convolutional neural network model with an emotion-labeled
corpus, which is constructed using hashtags. Secondly, we extend to learning
sentence-level representations with a huge corpus of texts with the pseudo task
of recognizing emojis. Our results show that, with the representations trained
from millions of tweets with weakly supervised labels such as hashtags and
emojis, we can solve sentiment/emotion analysis tasks more effectively.
  Lastly, as examples of model bias in representations of existing approaches,
we explore a specific problem of automatic detection of abusive language. We
address the issue of gender bias in various neural network models by conducting
experiments to measure and reduce those biases in the representations in order
to build more robust classification models.
Neural Architecture Optimization
Authors: Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, Tie-Yan Liu
Date: 2018-08-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Decoder, Evolutionary Algorithms, General Classification, GPU, image-classification, Image Classification, Language Modeling, Language Modelling, Neural Architecture Search, Reinforcement Learning

Automatic neural architecture design has shown its potential in discovering powerful neural network architectures. Existing methods, no matter based on reinforcement learning or evolutionary algorithms (EA), conduct architecture search in a discrete space, which is highly inefficient. In this paper, we propose a simple and efficient method to automatic neural architecture design based on continuous optimization. We call this new approach neural architecture optimization (NAO). There are three key components in our proposed approach: (1) An encoder embeds/maps neural network architectures into a continuous space. (2) A predictor takes the continuous representation of a network as input and predicts its accuracy. (3) A decoder maps a continuous representation of a network back to its architecture. The performance predictor and the encoder enable us to perform gradient based optimization in the continuous space to find the embedding of a new architecture with potentially better accuracy. Such a better embedding is then decoded to a network by the decoder. Experiments show that the architecture discovered by our method is very competitive for image classification task on CIFAR-10 and language modeling task on PTB, outperforming or on par with the best results of previous architecture search methods with a significantly reduction of computational resources. Specifically we obtain 1.93% test set error rate for CIFAR-10 image classification task and 56.0 test set perplexity of PTB language modeling task. Furthermore, combined with the recent proposed weight sharing mechanism, we discover powerful architecture on CIFAR-10 (with error rate 2.93%) and on PTB (with test set perplexity 56.6), with very limited computational resources (less than 10 GPU hours) for both tasks.
Reducing Gender Bias in Abusive Language Detection
Authors: Ji Ho Park, Jamin Shin, Pascale Fung
Date: 2018-08-22 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Abusive Language, Data Augmentation, Word Embeddings

Abusive language detection models tend to have a problem of being biased
toward identity words of a certain group of people because of imbalanced
training datasets. For example, "You are a good woman" was considered "sexist"
when trained on an existing dataset. Such model bias is an obstacle for models
to be robust enough for practical use. In this work, we measure gender biases
on models trained with different abusive language datasets, while analyzing the
effect of different pre-trained word embeddings and model architectures. We
also experiment with three bias mitigation methods: (1) debiased word
embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a
larger corpus. These methods can effectively reduce gender bias by 90-98% and
can be extended to correct model bias in other scenarios.
Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents
Authors: Ganbin Zhou, Rongyu Cao, Xiang Ao, Ping Luo, Fen Lin, Leyu Lin, Qing He
Date: 2018-08-22 00:00:00
Tasks: Knowledge Base Construction

In this study, we focus on extracting knowledgeable snippets and annotating
knowledgeable documents from Web corpus, consisting of the documents from
social media and We-media. Informally, knowledgeable snippets refer to the text
describing concepts, properties of entities, or relations among entities, while
knowledgeable documents are the ones with enough knowledgeable snippets. These
knowledgeable snippets and documents could be helpful in multiple applications,
such as knowledge base construction and knowledge-oriented service. Previous
studies extracted the knowledgeable snippets using the pattern-based method.
Here, we propose the semantic-based method for this task. Specifically, a CNN
based model is developed to extract knowledgeable snippets and annotate
knowledgeable documents simultaneously. Additionally, a "low-level sharing,
high-level splitting" structure of CNN is designed to handle the documents from
different content domains. Compared with building multiple domain-specific
CNNs, this joint model not only critically saves the training time, but also
improves the prediction accuracy visibly. The superiority of the proposed
method is demonstrated in a real dataset from Wechat public platform.
Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective
Authors: Vishesh Jain, Frederic Koehler, Andrej Risteski
Date: 2018-08-22 00:00:00

The free energy is a key quantity of interest in Ising models, but
unfortunately, computing it in general is computationally intractable. Two
popular (variational) approximation schemes for estimating the free energy of
general Ising models (in particular, even in regimes where correlation decay
does not hold) are: (i) the mean-field approximation with roots in statistical
physics, which estimates the free energy from below, and (ii) hierarchies of
convex relaxations with roots in theoretical computer science, which estimate
the free energy from above. We show, surprisingly, that the tight regime for
both methods to compute the free energy to leading order is identical.
  More precisely, we show that the mean-field approximation is within
$O((n\|J\|_{F})^{2/3})$ of the free energy, where $\|J\|_F$ denotes the
Frobenius norm of the interaction matrix of the Ising model. This
simultaneously subsumes both the breakthrough work of Basak and Mukherjee, who
showed the tight result that the mean-field approximation is within $o(n)$
whenever $\|J\|_{F} = o(\sqrt{n})$, as well as the work of Jain, Koehler, and
Mossel, who gave the previously best known non-asymptotic bound of
$O((n\|J\|_{F})^{2/3}\log^{1/3}(n\|J\|_{F}))$. We give a simple, algorithmic
proof of this result using a convex relaxation proposed by Risteski based on
the Sherali-Adams hierarchy, automatically giving sub-exponential time
approximation schemes for the free energy in this entire regime. Our
algorithmic result is tight under Gap-ETH.
  We furthermore combine our techniques with spin glass theory to prove (in a
strong sense) the optimality of correlation rounding, refuting a recent
conjecture of Allen, O'Donnell, and Zhou. Finally, we give the tight
generalization of all of these results to $k$-MRFs, capturing as a special case
previous work on approximating MAX-$k$-CSP.
Approximating Poker Probabilities with Deep Learning
Authors: Brandon Da Silva
Date: 2018-08-22 00:00:00
Tasks: Card Games, Deep Learning, Game of Poker

Many poker systems, whether created with heuristics or machine learning, rely
on the probability of winning as a key input. However calculating the precise
probability using combinatorics is an intractable problem, so instead we
approximate it. Monte Carlo simulation is an effective technique that can be
used to approximate the probability that a player will win and/or tie a hand.
However, without the use of a memory-intensive lookup table or a supercomputer,
it becomes infeasible to run millions of times when training an agent with
self-play. To combat the space-time tradeoff, we use deep learning to
approximate the probabilities obtained from the Monte Carlo simulation with
high accuracy. The learned model proves to be a lightweight alternative to
Monte Carlo simulation, which ultimately allows us to use the probabilities as
inputs during self-play efficiently. The source code and optimized neural
network can be found at
https://github.com/brandinho/Poker-Probability-Approximation
Don't Use Large Mini-Batches, Use Local SGD
Authors: Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel, Martin Jaggi
Date: 2018-08-22 00:00:00
Proceeding: ICLR 2020 1

Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major roadblock, as models trained with large batches often do not generalize well, i.e. they do not show good accuracy on new data. As a remedy, we propose a \emph{post-local} SGD and show that it significantly improves the generalization performance compared to large-batch training on standard benchmarks while enjoying the same efficiency (time-to-accuracy) and scalability. We further provide an extensive study of the communication efficiency vs. performance trade-offs associated with a host of \emph{local SGD} variants.
Model Interpretation: A Unified Derivative-based Framework for Nonparametric Regression and Supervised Machine Learning
Authors: Xiaoyu Liu, Jie Chen, Joel Vaughan, Vijayan Nair, Agus Sudjianto
Date: 2018-08-22 00:00:00
Tasks: BIG-bench Machine Learning, regression

Interpreting a nonparametric regression model with many predictors is known
to be a challenging problem. There has been renewed interest in this topic due
to the extensive use of machine learning algorithms and the difficulty in
understanding and explaining their input-output relationships. This paper
develops a unified framework using a derivative-based approach for existing
tools in the literature, including the partial-dependence plots, marginal plots
and accumulated effects plots. It proposes a new interpretation technique
called the accumulated total derivative effects plot and demonstrates how its
components can be used to develop extensive insights in complex regression
models with correlated predictors. The techniques are illustrated through
simulation results.
A Characterwise Windowed Approach to Hebrew Morphological Segmentation
Authors: Amir Zeldes
Date: 2018-08-22 00:00:00
Proceeding: WS 2018 10
Tasks: Binary Classification, General Classification, Morphological Analysis, Text Segmentation

This paper presents a novel approach to the segmentation of orthographic word
forms in contemporary Hebrew, focusing purely on splitting without carrying out
morphological analysis or disambiguation. Casting the analysis task as
character-wise binary classification and using adjacent character and
word-based lexicon-lookup features, this approach achieves over 98% accuracy on
the benchmark SPMRL shared task data for Hebrew, and 97% accuracy on a new out
of domain Wikipedia dataset, an improvement of ~4% and 5% over previous state
of the art performance.
Coarse-to-Fine Annotation Enrichment for Semantic Segmentation Learning
Authors: Yadan Luo, Ziwei Wang, Zi Huang, Yang Yang, Cong Zhao
Date: 2018-08-22 00:00:00
Tasks: Segmentation, Semantic Segmentation, Weakly supervised segmentation

Rich high-quality annotated data is critical for semantic segmentation
learning, yet acquiring dense and pixel-wise ground-truth is both labor- and
time-consuming. Coarse annotations (e.g., scribbles, coarse polygons) offer an
economical alternative, with which training phase could hardly generate
satisfactory performance unfortunately. In order to generate high-quality
annotated data with a low time cost for accurate segmentation, in this paper,
we propose a novel annotation enrichment strategy, which expands existing
coarse annotations of training data to a finer scale. Extensive experiments on
the Cityscapes and PASCAL VOC 2012 benchmarks have shown that the neural
networks trained with the enriched annotations from our framework yield a
significant improvement over that trained with the original coarse labels. It
is highly competitive to the performance obtained by using human annotated
dense annotations. The proposed method also outperforms among other
state-of-the-art weakly-supervised segmentation methods.
Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model
Authors: Deli Chen, Shuming Ma, Pengcheng Yang, Xu sun
Date: 2018-08-22 00:00:00
Tasks: Informativeness, Text Matching

With the development of information technology, there is an explosive growth
in the number of online comment concerning news, blogs and so on. The massive
comments are overloaded, and often contain some misleading and unwelcome
information. Therefore, it is necessary to identify high-quality comments and
filter out low-quality comments. In this work, we introduce a novel task:
high-quality comment identification (HQCI), which aims to automatically assess
the quality of online comments. First, we construct a news comment corpus,
which consists of news, comments, and the corresponding quality label. Second,
we analyze the dataset, and find the quality of comments can be measured in
three aspects: informativeness, consistency, and novelty. Finally, we propose a
novel multi-target text matching model, which can measure three aspects by
referring to the news and surrounding comments. Experimental results show that
our method can outperform various baselines by a large margin on the news
dataset.
Neural Latent Extractive Document Summarization
Authors: Xingxing Zhang, Mirella Lapata, Furu Wei, Ming Zhou
Date: 2018-08-22 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Document Summarization, Extractive Document Summarization, Extractive Summarization, Extractive Text Summarization, Sentence

Extractive summarization models require sentence-level labels, which are
usually created heuristically (e.g., with rule-based methods) given that most
summarization datasets only have document-summary pairs. Since these labels
might be suboptimal, we propose a latent variable extractive model where
sentences are viewed as latent variables and sentences with activated variables
are used to infer gold summaries. During training the loss comes
\emph{directly} from gold summaries. Experiments on the CNN/Dailymail dataset
show that our model improves over a strong extractive baseline trained on
heuristically approximated labels and also performs competitively to several
recent models.
Keyphrase Generation with Correlation Constraints
Authors: Jun Chen, Xiao-Ming Zhang, Yu Wu, Zhao Yan, Zhoujun Li
Date: 2018-08-22 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Diversity, Keyphrase Generation

In this paper, we study automatic keyphrase generation. Although conventional
approaches to this task show promising results, they neglect correlation among
keyphrases, resulting in duplication and coverage issues. To solve these
problems, we propose a new sequence-to-sequence architecture for keyphrase
generation named CorrRNN, which captures correlation among multiple keyphrases
in two ways. First, we employ a coverage vector to indicate whether the word in
the source document has been summarized by previous phrases to improve the
coverage for keyphrases. Second, preceding phrases are taken into account to
eliminate duplicate phrases and improve result coherence. Experiment results
show that our model significantly outperforms the state-of-the-art method on
benchmark datasets in terms of both accuracy and diversity.
Can 3D Pose be Learned from 2D Projections Alone?
Authors: Dylan Drover, Rohith MV, Ching-Hang Chen, Amit Agrawal, Ambrish Tyagi, Cong Phuoc Huynh
Date: 2018-08-22 00:00:00
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation

3D pose estimation from a single image is a challenging task in computer
vision. We present a weakly supervised approach to estimate 3D pose points,
given only 2D pose landmarks. Our method does not require correspondences
between 2D and 3D points to build explicit 3D priors. We utilize an adversarial
framework to impose a prior on the 3D structure, learned solely from their
random 2D projections. Given a set of 2D pose landmarks, the generator network
hypothesizes their depths to obtain a 3D skeleton. We propose a novel Random
Projection layer, which randomly projects the generated 3D skeleton and sends
the resulting 2D pose to the discriminator. The discriminator improves by
discriminating between the generated poses and pose samples from a real
distribution of 2D poses. Training does not require correspondence between the
2D inputs to either the generator or the discriminator. We apply our approach
to the task of 3D human pose estimation. Results on Human3.6M dataset
demonstrates that our approach outperforms many previous supervised and weakly
supervised approaches.
Efficient sparse semismooth Newton methods for the clustered lasso problem
Authors: Meixia Lin, Yong-Jin Liu, Defeng Sun, Kim-Chuan Toh
Date: 2018-08-22 00:00:00

We focus on solving the clustered lasso problem, which is a least squares
problem with the $\ell_1$-type penalties imposed on both the coefficients and
their pairwise differences to learn the group structure of the regression
parameters. Here we first reformulate the clustered lasso regularizer as a
weighted ordered-lasso regularizer, which is essential in reducing the
computational cost from $O(n^2)$ to $O(n\log (n))$. We then propose an inexact
semismooth Newton augmented Lagrangian ({\sc Ssnal}) algorithm to solve the
clustered lasso problem or its dual via this equivalent formulation, depending
on whether the sample size is larger than the dimension of the features. An
essential component of the {\sc Ssnal} algorithm is the computation of the
generalized Jacobian of the proximal mapping of the clustered lasso
regularizer. Based on the new formulation, we derive an efficient procedure for
its computation. Comprehensive results on the global convergence and local
linear convergence of the {\sc Ssnal} algorithm are established. For the
purpose of exposition and comparison, we also summarize/design several
first-order methods that can be used to solve the problem under consideration,
but with the key improvement from the new formulation of the clustered lasso
regularizer. As a demonstration of the applicability of our algorithms,
numerical experiments on the clustered lasso problem are performed. The
experiments show that the {\sc Ssnal} algorithm substantially outperforms the
best alternative algorithm for the clustered lasso problem.
Fisher Information and Natural Gradient Learning of Random Deep Networks
Authors: Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Date: 2018-08-22 00:00:00

A deep neural network is a hierarchical nonlinear model transforming input
signals to output signals. Its input-output relation is considered to be
stochastic, being described for a given input by a parameterized conditional
probability distribution of outputs. The space of parameters consisting of
weights and biases is a Riemannian manifold, where the metric is defined by the
Fisher information matrix. The natural gradient method uses the steepest
descent direction in a Riemannian manifold, so it is effective in learning,
avoiding plateaus. It requires inversion of the Fisher information matrix,
however, which is practically impossible when the matrix has a huge number of
dimensions. Many methods for approximating the natural gradient have therefore
been introduced. The present paper uses statistical neurodynamical method to
reveal the properties of the Fisher information matrix in a net of random
connections under the mean field approximation. We prove that the Fisher
information matrix is unit-wise block diagonal supplemented by small order
terms of off-block-diagonal elements, which provides a justification for the
quasi-diagonal natural gradient method by Y. Ollivier. A unitwise
block-diagonal Fisher metrix reduces to the tensor product of the Fisher
information matrices of single units. We further prove that the Fisher
information matrix of a single unit has a simple reduced form, a sum of a
diagonal matrix and a rank 2 matrix of weight-bias correlations. We obtain the
inverse of Fisher information explicitly. We then have an explicit form of the
natural gradient, without relying on the numerical matrix inversion, which
drastically speeds up stochastic gradient learning.