pwc-archive/papers-with-abstracts huggingface dataset:
Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling
Authors: Yunzhe Tao, Qi Sun, Qiang Du, Wei Liu
Date: 2018-06-02 00:00:00
Proceeding: NeurIPS 2018 12

Nonlocal neural networks have been proposed and shown to be effective in
several computer vision tasks, where the nonlocal operations can directly
capture long-range dependencies in the feature space. In this paper, we study
the nature of diffusion and damping effect of nonlocal networks by doing
spectrum analysis on the weight matrices of the well-trained networks, and then
propose a new formulation of the nonlocal block. The new block not only learns
the nonlocal interactions but also has stable dynamics, thus allowing deeper
nonlocal structures. Moreover, we interpret our formulation from the general
nonlocal modeling perspective, where we make connections between the proposed
nonlocal network and other nonlocal models, such as nonlocal diffusion process
and Markov jump process.
A Geometric Approach for Real-time Monitoring of Dynamic Large Scale Graphs: AS-level graphs illustrated
Authors: Loqman Salamatian, Dali Kaafar, Kavé Salamatian
Date: 2018-06-02 00:00:00

The monitoring of large dynamic networks is a major chal- lenge for a wide
range of application. The complexity stems from properties of the underlying
graphs, in which slight local changes can lead to sizable variations of global
prop- erties, e.g., under certain conditions, a single link cut that may be
overlooked during monitoring can result in splitting the graph into two
disconnected components. Moreover, it is often difficult to determine whether a
change will propagate globally or remain local. Traditional graph theory
measure such as the centrality or the assortativity of the graph are not
satisfying to characterize global properties of the graph. In this paper, we
tackle the problem of real-time monitoring of dynamic large scale graphs by
developing a geometric approach that leverages notions of geometric curvature
and recent development in graph embeddings using Ollivier-Ricci curvature [47].
We illustrate the use of our method by consid- ering the practical case of
monitoring dynamic variations of global Internet using topology changes
information provided by combining several BGP feeds. In particular, we use our
method to detect major events and changes via the geometry of the embedding of
the graph.
Beyond the One Step Greedy Approach in Reinforcement Learning
Authors: Yonathan Efroni, Gal Dalal, Bruno Scherrer, Shie Mannor
Date: 2018-02-10 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The famous Policy Iteration algorithm alternates between policy improvement
and policy evaluation. Implementations of this algorithm with several variants
of the latter evaluation stage, e.g, $n$-step and trace-based returns, have
been analyzed in previous works. However, the case of multiple-step lookahead
policy improvement, despite the recent increase in empirical evidence of its
strength, has to our knowledge not been carefully analyzed yet. In this work,
we introduce the first such analysis. Namely, we formulate variants of
multiple-step policy improvement, derive new algorithms using these definitions
and prove their convergence. Moreover, we show that recent prominent
Reinforcement Learning algorithms are, in fact, instances of our framework. We
thus shed light on their empirical success and give a recipe for deriving new
algorithms for future study.
Emotion Detection in Text: a Review
Authors: Armin Seyeditabari, Narges Tabari, Wlodek Zadrozny
Date: 2018-06-02 00:00:00
Tasks: Marketing

In recent years, emotion detection in text has become more popular due to its
vast potential applications in marketing, political science, psychology,
human-computer interaction, artificial intelligence, etc. Access to a huge
amount of textual data, especially opinionated and self-expression text also
played a special role to bring attention to this field. In this paper, we
review the work that has been done in identifying emotion expressions in text
and argue that although many techniques, methodologies, and models have been
created to detect emotion in text, there are various reasons that make these
methods insufficient. Although, there is an essential need to improve the
design and architecture of current systems, factors such as the complexity of
human emotions, and the use of implicit and metaphorical language in expressing
it, lead us to think that just re-purposing standard methodologies will not be
enough to capture these complexities, and it is important to pay attention to
the linguistic intricacies of emotion expression.
Optimal Clustering under Uncertainty
Authors: Lori A. Dalton, Marco E. Benalcázar, Edward R. Dougherty
Date: 2018-06-02 00:00:00
Tasks: Clustering, parameter estimation, Point Processes, Robust classification

Classical clustering algorithms typically either lack an underlying
probability framework to make them predictive or focus on parameter estimation
rather than defining and minimizing a notion of error. Recent work addresses
these issues by developing a probabilistic framework based on the theory of
random labeled point processes and characterizing a Bayes clusterer that
minimizes the number of misclustered points. The Bayes clusterer is analogous
to the Bayes classifier. Whereas determining a Bayes classifier requires full
knowledge of the feature-label distribution, deriving a Bayes clusterer
requires full knowledge of the point process. When uncertain of the point
process, one would like to find a robust clusterer that is optimal over the
uncertainty, just as one may find optimal robust classifiers with uncertain
feature-label distributions. Herein, we derive an optimal robust clusterer by
first finding an effective random point process that incorporates all
randomness within its own probabilistic structure and from which a Bayes
clusterer can be derived that provides an optimal robust clusterer relative to
the uncertainty. This is analogous to the use of effective class-conditional
distributions in robust classification. After evaluating the performance of
robust clusterers in synthetic mixtures of Gaussians models, we apply the
framework to granular imaging, where we make use of the asymptotic
granulometric moment theory for granular images to relate robust clustering
theory to the application.
Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)
Authors: Linwei Hu, Jie Chen, Vijayan N. Nair, Agus Sudjianto
Date: 2018-06-02 00:00:00
Tasks: Clustering

Supervised Machine Learning (SML) algorithms such as Gradient Boosting,
Random Forest, and Neural Networks have become popular in recent years due to
their increased predictive performance over traditional statistical methods.
This is especially true with large data sets (millions or more observations and
hundreds to thousands of predictors). However, the complexity of the SML models
makes them opaque and hard to interpret without additional tools. There has
been a lot of interest recently in developing global and local diagnostics for
interpreting and explaining SML models. In this paper, we propose locally
interpretable models and effects based on supervised partitioning (trees)
referred to as LIME-SUP. This is in contrast with the KLIME approach that is
based on clustering the predictor space. We describe LIME-SUP based on fitting
trees to the fitted response (LIM-SUP-R) as well as the derivatives of the
fitted response (LIME-SUP-D). We compare the results with KLIME and describe
its advantages using simulation and real data.
Efficient active learning of sparse halfspaces
Authors: Chicheng Zhang
Date: 2018-05-07 00:00:00
Tasks: Active Learning, Attribute

We study the problem of efficient PAC active learning of homogeneous linear
classifiers (halfspaces) in $\mathbb{R}^d$, where the goal is to learn a
halfspace with low error using as few label queries as possible. Under the
extra assumption that there is a $t$-sparse halfspace that performs well on the
data ($t \ll d$), we would like our active learning algorithm to be {\em
attribute efficient}, i.e. to have label requirements sublinear in $d$. In this
paper, we provide a computationally efficient algorithm that achieves this
goal. Under certain distributional assumptions on the data, our algorithm
achieves a label complexity of $O(t \cdot \mathrm{polylog}(d, \frac 1
\epsilon))$. In contrast, existing algorithms in this setting are either
computationally inefficient, or subject to label requirements polynomial in $d$
or $\frac 1 \epsilon$.
Online Learning Sensing Matrix and Sparsifying Dictionary Simultaneously for Compressive Sensing
Authors: Tao Hong, Zhihui Zhu
Date: 2017-01-04 00:00:00
Tasks: Compressive Sensing

This paper considers the problem of simultaneously learning the Sensing
Matrix and Sparsifying Dictionary (SMSD) on a large training dataset. To
address the formulated joint learning problem, we propose an online algorithm
that consists of a closed-form solution for optimizing the sensing matrix with
a fixed sparsifying dictionary and a stochastic method for learning the
sparsifying dictionary on a large dataset when the sensing matrix is given.
Benefiting from training on a large dataset, the obtained compressive sensing
(CS) system by the proposed algorithm yields a much better performance in terms
of signal recovery accuracy than the existing ones. The simulation results on
natural images demonstrate the effectiveness of the suggested online algorithm
compared with the existing methods.
Signal and Noise Statistics Oblivious Orthogonal Matching Pursuit
Authors: Sreejith Kallummil, Sheetal Kalyani
Date: 2018-06-02 00:00:00
Proceeding: ICML 2018 7
Tasks: regression

Orthogonal matching pursuit (OMP) is a widely used algorithm for recovering
sparse high dimensional vectors in linear regression models. The optimal
performance of OMP requires \textit{a priori} knowledge of either the sparsity
of regression vector or noise statistics. Both these statistics are rarely
known \textit{a priori} and are very difficult to estimate. In this paper, we
present a novel technique called residual ratio thresholding (RRT) to operate
OMP without any \textit{a priori} knowledge of sparsity and noise statistics
and establish finite sample and large sample support recovery guarantees for
the same. Both analytical results and numerical simulations in real and
synthetic data sets indicate that RRT has a performance comparable to OMP with
\textit{a priori} knowledge of sparsity and noise statistics.
A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization
Authors: Li Wang, Junlin Yao, Yunzhe Tao, Li Zhong, Wei Liu, Qiang Du
Date: 2018-05-09 00:00:00
Tasks: Abstractive Text Summarization, Diversity, Informativeness, Text Summarization

In this paper, we propose a deep learning approach to tackle the automatic summarization tasks by incorporating topic information into the convolutional sequence-to-sequence (ConvS2S) model and using self-critical sequence training (SCST) for optimization. Through jointly attending to topics and word-level alignment, our approach can improve coherence, diversity, and informativeness of generated summaries via a biased probability generation mechanism. On the other hand, reinforcement training, like SCST, directly optimizes the proposed model with respect to the non-differentiable metric ROUGE, which also avoids the exposure bias during inference. We carry out the experimental evaluation with state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets. The empirical results demonstrate the superiority of our proposed method in the abstractive summarization.
Binary Classification with Karmic, Threshold-Quasi-Concave Metrics
Authors: Bowei Yan, Oluwasanmi Koyejo, Kai Zhong, Pradeep Ravikumar
Date: 2018-06-02 00:00:00
Proceeding: ICML 2018 7
Tasks: Binary Classification, Classification, General Classification

Complex performance measures, beyond the popular measure of accuracy, are
increasingly being used in the context of binary classification. These complex
performance measures are typically not even decomposable, that is, the loss
evaluated on a batch of samples cannot typically be expressed as a sum or
average of losses evaluated at individual samples, which in turn requires new
theoretical and methodological developments beyond standard treatments of
supervised learning. In this paper, we advance this understanding of binary
classification for complex performance measures by identifying two key
properties: a so-called Karmic property, and a more technical
threshold-quasi-concavity property, which we show is milder than existing
structural assumptions imposed on performance measures. Under these properties,
we show that the Bayes optimal classifier is a threshold function of the
conditional probability of positive class. We then leverage this result to come
up with a computationally practical plug-in classifier, via a novel threshold
estimator, and further, provide a novel statistical analysis of classification
error with respect to complex performance measures.
Squeeze-and-Excitation on Spatial and Temporal Deep Feature Space for Action Recognition
Authors: Gaoyun An, Wen Zhou, Yuxuan Wu, Zhenxing Zheng, Yongwen Liu
Date: 2018-06-02 00:00:00
Tasks: Action Recognition, Temporal Action Localization

Spatial and temporal features are two key and complementary information for
human action recognition. In order to make full use of the intra-frame spatial
characteristics and inter-frame temporal relationships, we propose the
Squeeze-and-Excitation Long-term Recurrent Convolutional Networks (SE-LRCN) for
human action recognition. The Squeeze and Excitation operations are used to
implement the feature recalibration. In SE-LRCN, Squeeze-and-Excitation
ResNet-34 (SE-ResNet-34) network is adopted to extract spatial features to
enhance the dependencies and importance of feature channels of pixel
granularity. We also propose the Squeeze-and-Excitation Long Short-Term Memory
(SE-LSTM) network to model the temporal relationship, and to enhance the
dependencies and importance of feature channels of frame granularity. We
evaluate the proposed model on two challenging benchmarks, HMDB51 and UCF101,
and the proposed SE-LRCN achieves the competitive results with the
state-of-the-art.
DAQN: Deep Auto-encoder and Q-Network
Authors: Daiki Kimura
Date: 2018-06-02 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The deep reinforcement learning method usually requires a large number of
training images and executing actions to obtain sufficient results. When it is
extended a real-task in the real environment with an actual robot, the method
will be required more training images due to complexities or noises of the
input images, and executing a lot of actions on the real robot also becomes a
serious problem. Therefore, we propose an extended deep reinforcement learning
method that is applied a generative model to initialize the network for
reducing the number of training trials. In this paper, we used a deep q-network
method as the deep reinforcement learning method and a deep auto-encoder as the
generative model. We conducted experiments on three different tasks: a
cart-pole game, an atari game, and a real-game with an actual robot. The
proposed method trained efficiently on all tasks than the previous method,
especially 2.5 times faster on a task with real environment images.
A Novel Framework for Recurrent Neural Networks with Enhancing Information Processing and Transmission between Units
Authors: Xi Chen, Zhi-Hong Deng, Gehui Shen, Ting Huang
Date: 2018-06-02 00:00:00
Tasks: General Classification, image-classification, Image Classification, Language Modelling, text-classification, Text Classification

This paper proposes a novel framework for recurrent neural networks (RNNs)
inspired by the human memory models in the field of cognitive neuroscience to
enhance information processing and transmission between adjacent RNNs' units.
The proposed framework for RNNs consists of three stages that is working
memory, forget, and long-term store. The first stage includes taking input data
into sensory memory and transferring it to working memory for preliminary
treatment. And the second stage mainly focuses on proactively forgetting the
secondary information rather than the primary in the working memory. And
finally, we get the long-term store normally using some kind of RNN's unit. Our
framework, which is generalized and simple, is evaluated on 6 datasets which
fall into 3 different tasks, corresponding to text classification, image
classification and language modelling. Experiments reveal that our framework
can obviously improve the performance of traditional recurrent neural networks.
And exploratory task shows the ability of our framework of correctly forgetting
the secondary information.
Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning
Authors: Myeongjun Jang, Seungwan Seo, Pilsung Kang
Date: 2018-02-09 00:00:00
Tasks: Imputation, Language Modeling, Language Modelling, Machine Translation, Paraphrase Identification, Sentence, speech-recognition, Speech Recognition, Text Summarization

Sequence-to-sequence (Seq2seq) models have played an important role in the
recent success of various natural language processing methods, such as machine
translation, text summarization, and speech recognition. However, current
Seq2seq models have trouble preserving global latent information from a long
sequence of words. Variational autoencoder (VAE) alleviates this problem by
learning a continuous semantic space of the input sentence. However, it does
not solve the problem completely. In this paper, we propose a new recurrent
neural network (RNN)-based Seq2seq model, RNN semantic variational autoencoder
(RNN--SVAE), to better capture the global latent information of a sequence of
words. To reflect the meaning of words in a sentence properly, without regard
to its position within the sentence, we construct a document information vector
using the attention information between the final state of the encoder and
every prior hidden state. Then, the mean and standard deviation of the
continuous semantic space are learned by using this vector to take advantage of
the variational method. By using the document information vector to find the
semantic space of the sentence, it becomes possible to better capture the
global latent feature of the sentence. Experimental results of three natural
language tasks (i.e., language modeling, missing word imputation, paraphrase
identification) confirm that the proposed RNN--SVAE yields higher performance
than two benchmark models.
Beyond Trade-off: Accelerate FCN-based Face Detector with Higher Accuracy
Authors: Guanglu Song, Yu Liu, Ming Jiang, Yujie Wang, Junjie Yan, Biao Leng
Date: 2018-04-14 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Detection, Philosophy, valid

Fully convolutional neural network (FCN) has been dominating the game of face
detection task for a few years with its congenital capability of
sliding-window-searching with shared kernels, which boiled down all the
redundant calculation, and most recent state-of-the-art methods such as
Faster-RCNN, SSD, YOLO and FPN use FCN as their backbone. So here comes one
question: Can we find a universal strategy to further accelerate FCN with
higher accuracy, so could accelerate all the recent FCN-based methods? To
analyze this, we decompose the face searching space into two orthogonal
directions, `scale' and `spatial'. Only a few coordinates in the space expanded
by the two base vectors indicate foreground. So if FCN could ignore most of the
other points, the searching space and false alarm should be significantly
boiled down. Based on this philosophy, a novel method named scale estimation
and spatial attention proposal ($S^2AP$) is proposed to pay attention to some
specific scales and valid locations in the image pyramid. Furthermore, we adopt
a masked-convolution operation based on the attention result to accelerate FCN
calculation. Experiments show that FCN-based method RPN can be accelerated by
about $4\times$ with the help of $S^2AP$ and masked-FCN and at the same time it
can also achieve the state-of-the-art on FDDB, AFW and MALF face detection
benchmarks as well.
AP18-OLR Challenge: Three Tasks and Their Baselines
Authors: Zhiyuan Tang, Dong Wang, Qing Chen
Date: 2018-06-02 00:00:00
Tasks: Open Set Learning

The third oriental language recognition (OLR) challenge AP18-OLR is
introduced in this paper, including the data profile, the tasks and the
evaluation principles. Following the events in the last two years, namely
AP16-OLR and AP17-OLR, the challenge this year focuses on more challenging
tasks, including (1) short-duration utterances, (2) confusing languages, and
(3) open-set recognition. The same as the previous events, the data of AP18-OLR
is also provided by SpeechOcean and the NSFC M2ASR project. Baselines based on
both the i-vector model and neural networks are constructed for the
participants' reference. We report the baseline results on the three tasks and
demonstrate that the three tasks are truly challenging. All the data is free
for participants, and the Kaldi recipes for the baselines have been published
online.
Multiplex Communities and the Emergence of International Conflict
Authors: Caleb Pomeroy, Niheer Dasandi, Slava Jankin Mikhaylov
Date: 2018-06-02 00:00:00
Tasks: Community Detection

Advances in community detection reveal new insights into multiplex and multilayer networks. Less work, however, investigates the relationship between these communities and outcomes in social systems. We leverage these advances to shed light on the relationship between the cooperative mesostructure of the international system and the onset of interstate conflict. We detect communities based upon weaker signals of affinity expressed in United Nations votes and speeches, as well as stronger signals observed across multiple layers of bilateral cooperation. Communities of diplomatic affinity display an expected negative relationship with conflict onset. Ties in communities based upon observed cooperation, however, display no effect under a standard model specification and a positive relationship with conflict under an alternative specification. These results align with some extant hypotheses but also point to a paucity in our understanding of the relationship between community structure and behavioral outcomes in networks.
Between Progress and Potential Impact of AI: the Neglected Dimensions
Authors: Fernando Martínez-Plumed, Shahar Avin, Miles Brundage, Allan Dafoe, Sean Ó hÉigeartaigh, José Hernández-Orallo
Date: 2018-06-02 00:00:00
Tasks: Atari Games, Board Games, image-classification, Image Classification

We reframe the analysis of progress in AI by incorporating into an overall framework both the task performance of a system, and the time and resource costs incurred in the development and deployment of the system. These costs include: data, expert knowledge, human oversight, software resources, computing cycles, hardware and network facilities, and (what kind of) time. These costs are distributed over the life cycle of the system, and may place differing demands on different developers and users. The multidimensional performance and cost space we present can be collapsed to a single utility metric that measures the value of the system for different stakeholders. Even without a single utility function, AI advances can be generically assessed by whether they expand the Pareto surface. We label these types of costs as neglected dimensions of AI progress, and explore them using four case studies: Alpha* (Go, Chess, and other board games), ALE (Atari games), ImageNet (Image classification) and Virtual Personal Assistants (Siri, Alexa, Cortana, and Google Assistant). This broader model of progress in AI will lead to novel ways of estimating the potential societal use and impact of an AI system, and the establishment of milestones for future progress.