pwc-archive/papers-with-abstracts huggingface dataset:
Basic concepts and tools for the Toki Pona minimal and constructed language: description of the language and main issues; analysis of the vocabulary; text synthesis and syntax highlighting; Wordnet synsets
Authors: Renato Fabbri
Date: 2017-12-26 00:00:00
Tasks: Sentence

A minimal constructed language (conlang) is useful for experiments and
comfortable for making tools. The Toki Pona (TP) conlang is minimal both in the
vocabulary (with only 14 letters and 124 lemmas) and in the (about) 10 syntax
rules. The language is useful for being a used and somewhat established minimal
conlang with at least hundreds of fluent speakers. This article exposes current
concepts and resources for TP, and makes available Python (and Vim) scripted
routines for the analysis of the language, synthesis of texts, syntax
highlighting schemes, and the achievement of a preliminary TP Wordnet. Focus is
on the analysis of the basic vocabulary, as corpus analyses were found. The
synthesis is based on sentence templates, relates to context by keeping track
of used words, and renders larger texts by using a fixed number of phonemes
(e.g. for poems) and number of sentences, words and letters (e.g. for
paragraphs). Syntax highlighting reflects morphosyntactic classes given in the
official dictionary and different solutions are described and implemented in
the well-established Vim text editor. The tentative TP Wordnet is made
available in three patterns of relations between synsets and word lemmas. In
summary, this text holds potentially novel conceptualizations about, and tools
and results in analyzing, synthesizing and syntax highlighting the TP language.
Deep Unsupervised Clustering Using Mixture of Autoencoders
Authors: Dejiao Zhang, Yifan Sun, Brian Eriksson, Laura Balzano
Date: 2017-12-21 00:00:00
Tasks: Clustering

Unsupervised clustering is one of the most fundamental challenges in machine
learning. A popular hypothesis is that data are generated from a union of
low-dimensional nonlinear manifolds; thus an approach to clustering is
identifying and separating these manifolds. In this paper, we present a novel
approach to solve this problem by using a mixture of autoencoders. Our model
consists of two parts: 1) a collection of autoencoders where each autoencoder
learns the underlying manifold of a group of similar objects, and 2) a mixture
assignment neural network, which takes the concatenated latent vectors from the
autoencoders as input and infers the distribution over clusters. By jointly
optimizing the two parts, we simultaneously assign data to clusters and learn
the underlying manifolds of each cluster.
A model for interpreting social interactions in local image regions
Authors: Guy Ben-Yosef, Alon Yachin, Shimon Ullman
Date: 2017-12-26 00:00:00

Understanding social interactions (such as 'hug' or 'fight') is a basic and
important capacity of the human visual system, but a challenging and still open
problem for modeling. In this work we study visual recognition of social
interactions, based on small but recognizable local regions. The approach is
based on two novel key components: (i) A given social interaction can be
recognized reliably from reduced images (called 'minimal images'). (ii) The
recognition of a social interaction depends on identifying components and
relations within the minimal image (termed 'interpretation'). We show
psychophysics data for minimal images and modeling results for their
interpretation. We discuss the integration of minimal configurations in
recognizing social interactions in a detailed, high-resolution image.
TIP: Typifying the Interpretability of Procedures
Authors: Amit Dhurandhar, Vijay Iyengar, Ronny Luss, Karthikeyan Shanmugam
Date: 2017-06-09 00:00:00
Tasks: Knowledge Distillation

We provide a novel notion of what it means to be interpretable, looking past
the usual association with human understanding. Our key insight is that
interpretability is not an absolute concept and so we define it relative to a
target model, which may or may not be a human. We define a framework that
allows for comparing interpretable procedures by linking them to important
practical aspects such as accuracy and robustness. We characterize many of the
current state-of-the-art interpretable methods in our framework portraying its
general applicability. Finally, principled interpretable strategies are
proposed and empirically evaluated on synthetic data, as well as on the largest
public olfaction dataset that was made recently available \cite{olfs}. We also
experiment on MNIST with a simple target model and different oracle models of
varying complexity. This leads to the insight that the improvement in the
target model is not only a function of the oracle model's performance, but also
its relative complexity with respect to the target model. Further experiments
on CIFAR-10, a real manufacturing dataset and FICO dataset showcase the benefit
of our methods over Knowledge Distillation when the target models are simple
and the complex model is a neural network.
Social Media Writing Style Fingerprint
Authors: Himank Yadav, Juliang Li
Date: 2017-12-11 00:00:00
Tasks: Authorship Attribution, Authorship Verification

We present our approach for computer-aided social media text authorship
attribution based on recent advances in short text authorship verification. We
use various natural language techniques to create word-level and
character-level models that act as hidden layers to simulate a simple neural
network. The choice of word-level and character-level models in each layer was
informed through validation performance. The output layer of our system uses an
unweighted majority vote vector to arrive at a conclusion. We also considered
writing bias in social media posts while collecting our training dataset to
increase system robustness. Our system achieved a precision, recall, and
F-measure of 0.82, 0.926 and 0.869 respectively.
Wasserstein Distributionally Robust Optimization and Variation Regularization
Authors: Rui Gao, Xi Chen, Anton J. Kleywegt
Date: 2017-12-17 00:00:00
Tasks: Stochastic Optimization

Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existing results often require restrictive assumptions, such as smoothness or convexity, that are not satisfied for many problems. In this paper, we develop a general theory on the variation regularization effect of the Wasserstein DRO - a new form of regularization that generalizes total-variation regularization, Lipschitz regularization and gradient regularization. Our results cover possibly non-convex and non-smooth losses and losses on non-Euclidean spaces. Examples include multi-item newsvendor, portfolio selection, linear prediction, neural networks, manifold learning, and intensity estimation for Poisson processes, etc. As an application of our theory of variation regularization, we derive new generalization guarantees for adversarial robust learning.
Scalable Prototype Selection by Genetic Algorithms and Hashing
Authors: Yenisel Plasencia-Calaña, Mauricio Orozco-Alzate, Heydi Méndez-Vázquez, Edel García-Reyes, Robert P. W. Duin
Date: 2017-12-26 00:00:00
Tasks: Prototype Selection

Classification in the dissimilarity space has become a very active research
area since it provides a possibility to learn from data given in the form of
pairwise non-metric dissimilarities, which otherwise would be difficult to cope
with. The selection of prototypes is a key step for the further creation of the
space. However, despite previous efforts to find good prototypes, how to select
the best representation set remains an open issue. In this paper we proposed
scalable methods to select the set of prototypes out of very large datasets.
The methods are based on genetic algorithms, dissimilarity-based hashing, and
two different unsupervised and supervised scalable criteria. The unsupervised
criterion is based on the Minimum Spanning Tree of the graph created by the
prototypes as nodes and the dissimilarities as edges. The supervised criterion
is based on counting matching labels of objects and their closest prototypes.
The suitability of these type of algorithms is analyzed for the specific case
of dissimilarity representations. The experimental results showed that the
methods select good prototypes taking advantage of the large datasets, and they
do so at low runtimes.
Optimal structure and parameter learning of Ising models
Authors: Andrey Y. Lokhov, Marc Vuffray, Sidhant Misra, Michael Chertkov
Date: 2016-12-15 00:00:00

Reconstruction of structure and parameters of an Ising model from binary
samples is a problem of practical importance in a variety of disciplines,
ranging from statistical physics and computational biology to image processing
and machine learning. The focus of the research community shifted towards
developing universal reconstruction algorithms which are both computationally
efficient and require the minimal amount of expensive data. We introduce a new
method, Interaction Screening, which accurately estimates the model parameters
using local optimization problems. The algorithm provably achieves perfect
graph structure recovery with an information-theoretically optimal number of
samples, notably in the low-temperature regime which is known to be the hardest
for learning. The efficacy of Interaction Screening is assessed through
extensive numerical tests on synthetic Ising models of various topologies with
different types of interactions, as well as on a real data produced by a D-Wave
quantum computer. This study shows that the Interaction Screening method is an
exact, tractable and optimal technique universally solving the inverse Ising
problem.
Modeling Past and Future for Neural Machine Translation
Authors: Zaixiang Zheng, Hao Zhou, Shu-Jian Huang, Lili Mou, Xin-yu Dai, Jia-Jun Chen, Zhaopeng Tu
Date: 2017-11-27 00:00:00
Proceeding: TACL 2018 1
Tasks: Decoder, Machine Translation, NMT, Translation

Existing neural machine translation systems do not explicitly model what has
been translated and what has not during the decoding phase. To address this
problem, we propose a novel mechanism that separates the source information
into two parts: translated Past contents and untranslated Future contents,
which are modeled by two additional recurrent layers. The Past and Future
contents are fed to both the attention model and the decoder states, which
offers NMT systems the knowledge of translated and untranslated contents.
Experimental results show that the proposed approach significantly improves
translation performance in Chinese-English, German-English and English-German
translation tasks. Specifically, the proposed model outperforms the
conventional coverage model in both of the translation quality and the
alignment error rate.
A Real-Time Autonomous Highway Accident Detection Model Based on Big Data Processing and Computational Intelligence
Authors: A. Murat Ozbayoglu, Gokhan Kucukayan, Erdogan Dogdu
Date: 2017-12-26 00:00:00

Due to increasing urban population and growing number of motor vehicles,
traffic congestion is becoming a major problem of the 21st century. One of the
main reasons behind traffic congestion is accidents which can not only result
in casualties and losses for the participants, but also in wasted and lost time
for the others that are stuck behind the wheels. Early detection of an accident
can save lives, provides quicker road openings, hence decreases wasted time and
resources, and increases efficiency. In this study, we propose a preliminary
real-time autonomous accident-detection system based on computational
intelligence techniques. Istanbul City traffic-flow data for the year 2015 from
various sensor locations are populated using big data processing methodologies.
The extracted features are then fed into a nearest neighbor model, a regression
tree, and a feed-forward neural network model. For the output, the possibility
of an occurrence of an accident is predicted. The results indicate that even
though the number of false alarms dominates the real accident cases, the system
can still provide useful information that can be used for status verification
and early reaction to possible accidents.
Large-Scale 3D Scene Classification With Multi-View Volumetric CNN
Authors: Dror Aiger, Brett Allen, Aleksey Golovinskiy
Date: 2017-12-26 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Scene Classification

We introduce a method to classify imagery using a convo- lutional neural
network (CNN) on multi-view image pro- jections. The power of our method comes
from using pro- jections of multiple images at multiple depth planes near the
reconstructed surface. This enables classification of categories whose salient
aspect is appearance change un- der different viewpoints, such as water, trees,
and other materials with complex reflection/light response proper- ties. Our
method does not require boundary labelling in images and works on pixel-level
classification with a small (few pixels) context, which simplifies the cre-
ation of a training set. We demonstrate this application on large-scale aerial
imagery collections, and extend the per-pixel classification to robustly create
a consistent 2D classification which can be used to fill the gaps in non-
reconstructible water regions. We also apply our method to classify tree
regions. In both cases, the training data can quickly be generated using a
small number of manually- created polygons on a map. We show that even with a
very simple and standard network our CNN outperforms the state-of-the-art image
classification, the Inception-V3 model retrained from a large collection of
aerial images.
Aircraft Fuselage Defect Detection using Deep Neural Networks
Authors: Touba Malekzadeh, Milad Abdollahzadeh, Hossein Nejati, Ngai-Man Cheung
Date: 2017-12-26 00:00:00
Tasks: Defect Detection

To ensure flight safety of aircraft structures, it is necessary to have regular maintenance using visual and nondestructive inspection (NDI) methods. In this paper, we propose an automatic image-based aircraft defect detection using Deep Neural Networks (DNNs). To the best of our knowledge, this is the first work for aircraft defect detection using DNNs. We perform a comprehensive evaluation of state-of-the-art feature descriptors and show that the best performance is achieved by vgg-f DNN as feature extractor with a linear SVM classifier. To reduce the processing time, we propose to apply SURF key point detector to identify defect patch candidates. Our experiment results suggest that we can achieve over 96% accuracy at around 15s processing time for a high-resolution (20-megapixel) image on a laptop.
The Robust Manifold Defense: Adversarial Training using Generative Models
Authors: Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, Alexandros G. Dimakis
Date: 2017-12-26 00:00:00

We propose a new type of attack for finding adversarial examples for image classifiers. Our method exploits spanners, i.e. deep neural networks whose input space is low-dimensional and whose output range approximates the set of images of interest. Spanners may be generators of GANs or decoders of VAEs. The key idea in our attack is to search over latent code pairs to find ones that generate nearby images with different classifier outputs. We argue that our attack is stronger than searching over perturbations of real images. Moreover, we show that our stronger attack can be used to reduce the accuracy of Defense-GAN to 3\%, resolving an open problem from the well-known paper by Athalye et al. We combine our attack with normal adversarial training to obtain the most robust known MNIST classifier, significantly improving the state of the art against PGD attacks. Our formulation involves solving a min-max problem, where the min player sets the parameters of the classifier and the max player is running our attack, and is thus searching for adversarial examples in the {\em low-dimensional} input space of the spanner. All code and models are available at \url{https://github.com/ajiljalal/manifold-defense.git}
Model-free Nonconvex Matrix Completion: Local Minima Analysis and Applications in Memory-efficient Kernel PCA
Authors: Ji Chen, Xiao-Dong Li
Date: 2017-11-06 00:00:00
Tasks: Clustering, Dimensionality Reduction, Matrix Completion

This work studies low-rank approximation of a positive semidefinite matrix
from partial entries via nonconvex optimization. We characterized how well
local-minimum based low-rank factorization approximates a fixed positive
semidefinite matrix without any assumptions on the rank-matching, the condition
number or eigenspace incoherence parameter. Furthermore, under certain
assumptions on rank-matching and well-boundedness of condition numbers and
eigenspace incoherence parameters, a corollary of our main theorem improves the
state-of-the-art sampling rate results for nonconvex matrix completion with no
spurious local minima in Ge et al. [2016, 2017]. In addition, we investigated
when the proposed nonconvex optimization results in accurate low-rank
approximations even in presence of large condition numbers, large incoherence
parameters, or rank mismatching. We also propose to apply the nonconvex
optimization to memory-efficient Kernel PCA. Compared to the well-known
Nystr\"{o}m methods, numerical experiments indicate that the proposed nonconvex
optimization approach yields more stable results in both low-rank approximation
and clustering.
Actionable Email Intent Modeling with Reparametrized RNNs
Authors: Chu-Cheng Lin, Dongyeop Kang, Michael Gamon, Madian Khabsa, Ahmed Hassan Awadallah, Patrick Pantel
Date: 2017-12-26 00:00:00
Tasks: Action Classification

Emails in the workplace are often intentional calls to action for its
recipients. We propose to annotate these emails for what action its recipient
will take. We argue that our approach of action-based annotation is more
scalable and theory-agnostic than traditional speech-act-based email intent
annotation, while still carrying important semantic and pragmatic information.
We show that our action-based annotation scheme achieves good inter-annotator
agreement. We also show that we can leverage threaded messages from other
domains, which exhibit comparable intents in their conversation, with domain
adaptive RAINBOW (Recurrently AttentIve Neural Bag-Of-Words). On a collection
of datasets consisting of IRC, Reddit, and email, our reparametrized RNNs
outperform common multitask/multidomain approaches on several speech act
related tasks. We also experiment with a minimally supervised scenario of email
recipient action classification, and find the reparametrized RNNs learn a
useful representation.
Simple Recurrent Units for Highly Parallelizable Recurrence
Authors: Tao Lei, Yu Zhang, Sida I. Wang, Hui Dai, Yoav Artzi
Date: 2017-09-08 00:00:00
Proceeding: EMNLP 2018 10
Tasks: General Classification, Machine Translation, Question Answering, Text Classification, Translation

Common recurrent neural architectures scale poorly due to the intrinsic
difficulty in parallelizing their state computations. In this work, we propose
the Simple Recurrent Unit (SRU), a light recurrent unit that balances model
capacity and scalability. SRU is designed to provide expressive recurrence,
enable highly parallelized implementation, and comes with careful
initialization to facilitate training of deep models. We demonstrate the
effectiveness of SRU on multiple NLP tasks. SRU achieves 5--9x speed-up over
cuDNN-optimized LSTM on classification and question answering datasets, and
delivers stronger results than LSTM and convolutional models. We also obtain an
average of 0.7 BLEU improvement over the Transformer model on translation by
incorporating SRU into the architecture.
A Polynomial-Time Deterministic Approach to the Traveling Salesperson Problem
Authors: Ali Jazayeri, Hiroki Sayama
Date: 2016-08-04 00:00:00

We propose a new polynomial-time deterministic algorithm that produces an
approximated solution for the traveling salesperson problem. The proposed
algorithm ranks cities based on their priorities calculated using a power
function of means and standard deviations of their distances from other cities
and then connects the cities to their neighbors in the order of their
priorities. When connecting a city, a neighbor is selected based on their
neighbors' priorities calculated as another power function that additionally
includes their distance from the focal city to be connected. This repeats until
all the cities are connected into a single loop. The time complexity of the
proposed algorithm is $O(n^2)$, where $n$ is the number of cities. Numerical
evaluation shows that, despite its simplicity, the proposed algorithm produces
shorter tours with less time complexity than other conventional tour
construction heuristics. The proposed algorithm can be used by itself or as an
initial tour generator for other more complex heuristic optimization
algorithms.
Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning
Authors: Meng Qu, Xiang Ren, Yu Zhang, Jiawei Han
Date: 2017-11-09 00:00:00
Tasks: Knowledge Base Completion, Relation, Relation Extraction

Extracting relations from text corpora is an important task in text mining.
It becomes particularly challenging when focusing on weakly-supervised relation
extraction, that is, utilizing a few relation instances (i.e., a pair of
entities and their relation) as seeds to extract more instances from corpora.
Existing distributional approaches leverage the corpus-level co-occurrence
statistics of entities to predict their relations, and require large number of
labeled instances to learn effective relation classifiers. Alternatively,
pattern-based approaches perform bootstrapping or apply neural networks to
model the local contexts, but still rely on large number of labeled instances
to build reliable models. In this paper, we study integrating the
distributional and pattern-based methods in a weakly-supervised setting, such
that the two types of methods can provide complementary supervision for each
other to build an effective, unified model. We propose a novel co-training
framework with a distributional module and a pattern module. During training,
the distributional module helps the pattern module discriminate between the
informative patterns and other patterns, and the pattern module generates some
highly-confident instances to improve the distributional module. The whole
framework can be effectively optimized by iterating between improving the
pattern module and updating the distributional module. We conduct experiments
on two tasks: knowledge base completion with text corpora and corpus-level
relation extraction. Experimental results prove the effectiveness of our
framework in the weakly-supervised setting.
Deep Meta Learning for Real-Time Target-Aware Visual Tracking
Authors: Janghoon Choi, Junseok Kwon, Kyoung Mu Lee
Date: 2017-12-26 00:00:00
Proceeding: ICCV 2019 10
Tasks: Meta-Learning, Real-Time Visual Tracking, Visual Tracking

In this paper, we propose a novel on-line visual tracking framework based on the Siamese matching network and meta-learner network, which run at real-time speeds. Conventional deep convolutional feature-based discriminative visual tracking algorithms require continuous re-training of classifiers or correlation filters, which involve solving complex optimization tasks to adapt to the new appearance of a target object. To alleviate this complex process, our proposed algorithm incorporates and utilizes a meta-learner network to provide the matching network with new appearance information of the target objects by adding target-aware feature space. The parameters for the target-specific feature space are provided instantly from a single forward-pass of the meta-learner network. By eliminating the necessity of continuously solving complex optimization tasks in the course of tracking, experimental results demonstrate that our algorithm performs at a real-time speed while maintaining competitive performance among other state-of-the-art tracking algorithms.