pwc-archive/papers-with-abstracts huggingface dataset:
Communication Efficient Parallel Algorithms for Optimization on Manifolds
Authors: Bayan Saparbayeva, Michael Minyi Zhang, Lizhen Lin
Date: 2018-10-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Distributed Computing, Low-Rank Matrix Completion, Matrix Completion, valid

The last decade has witnessed an explosion in the development of models,
theory and computational algorithms for "big data" analysis. In particular,
distributed computing has served as a natural and dominating paradigm for
statistical inference. However, the existing literature on parallel inference
almost exclusively focuses on Euclidean data and parameters. While this
assumption is valid for many applications, it is increasingly more common to
encounter problems where the data or the parameters lie on a non-Euclidean
space, like a manifold for example. Our work aims to fill a critical gap in the
literature by generalizing parallel inference algorithms to optimization on
manifolds. We show that our proposed algorithm is both communication efficient
and carries theoretical convergence guarantees. In addition, we demonstrate the
performance of our algorithm to the estimation of Fr\'echet means on simulated
spherical data and the low-rank matrix completion problem over Grassmann
manifolds applied to the Netflix prize data set.
Towards improved lossy image compression: Human image reconstruction with public-domain images
Authors: Ashutosh Bhown, Soham Mukherjee, Sean Yang, Shubham Chandak, Irena Fischer-Hwang, Kedar Tatwawadi, Judith Fan, Tsachy Weissman
Date: 2018-10-25 00:00:00
Tasks: Image Compression, Image Reconstruction, MS-SSIM, SSIM

Lossy image compression has been studied extensively in the context of typical loss functions such as RMSE, MS-SSIM, etc. However, compression at low bitrates generally produces unsatisfying results. Furthermore, the availability of massive public image datasets appears to have hardly been exploited in image compression. Here, we present a paradigm for eliciting human image reconstruction in order to perform lossy image compression. In this paradigm, one human describes images to a second human, whose task is to reconstruct the target image using publicly available images and text instructions. The resulting reconstructions are then evaluated by human raters on the Amazon Mechanical Turk platform and compared to reconstructions obtained using state-of-the-art compressor WebP. Our results suggest that prioritizing semantic visual elements may be key to achieving significant improvements in image compression, and that our paradigm can be used to develop a more human-centric loss function. The images, results and additional data are available at https://compression.stanford.edu/human-compression
Improving Document Binarization via Adversarial Noise-Texture Augmentation
Authors: Ankan Kumar Bhunia, Ayan Kumar Bhunia, Aneeshan Sain, Partha Pratim Roy
Date: 2018-10-25 00:00:00
Tasks: Adversarial Robustness, Binarization, Domain Adaptation, Transfer Learning

Binarization of degraded document images is an elementary step in most of the
problems in document image analysis domain. The paper re-visits the
binarization problem by introducing an adversarial learning approach. We
construct a Texture Augmentation Network that transfers the texture element of
a degraded reference document image to a clean binary image. In this way, the
network creates multiple versions of the same textual content with various
noisy textures, thus enlarging the available document binarization datasets. At
last, the newly generated images are passed through a Binarization network to
get back the clean version. By jointly training the two networks we can
increase the adversarial robustness of our system. Also, it is noteworthy that
our model can learn from unpaired data. Experimental results suggest that the
proposed method achieves superior performance over widely used DIBCO datasets.
A Large-Scale Corpus for Conversation Disentanglement
Authors: Jonathan K. Kummerfeld, Sai R. Gouravajhala, Joseph Peper, Vignesh Athreya, Chulaka Gunasekara, Jatin Ganhotra, Siva Sankalp Patel, Lazaros Polymenakos, Walter S. Lasecki
Date: 2018-10-25 00:00:00
Proceeding: ACL 2019 7
Tasks: Conversation Disentanglement, Disentanglement

Disentangling conversations mixed together in a single stream of messages is a difficult task, made harder by the lack of large manually annotated datasets. We created a new dataset of 77,563 messages manually annotated with reply-structure graphs that both disentangle conversations and define internal conversation structure. Our dataset is 16 times larger than all previously released datasets combined, the first to include adjudication of annotation disagreements, and the first to include context. We use our data to re-examine prior work, in particular, finding that 80% of conversations in a widely used dialogue corpus are either missing messages or contain extra messages. Our manually-annotated data presents an opportunity to develop robust data-driven methods for conversation disentanglement, which will help advance dialogue research.
Mimetic vs Anchored Value Alignment in Artificial Intelligence
Authors: Tae Wan Kim, Thomas Donaldson, John Hooker
Date: 2018-10-25 00:00:00

"Value alignment" (VA) is considered as one of the top priorities in AI
research. Much of the existing research focuses on the "A" part and not the "V"
part of "value alignment." This paper corrects that neglect by emphasizing the
"value" side of VA and analyzes VA from the vantage point of requirements in
value theory, in particular, of avoiding the "naturalistic fallacy"--a major
epistemic caveat. The paper begins by isolating two distinct forms of VA:
"mimetic" and "anchored." Then it discusses which VA approach better avoids the
naturalistic fallacy. The discussion reveals stumbling blocks for VA approaches
that neglect implications of the naturalistic fallacy. Such problems are more
serious in mimetic VA since the mimetic process imitates human behavior that
may or may not rise to the level of correct ethical behavior. Anchored VA,
including hybrid VA, in contrast, holds more promise for future VA since it
anchors alignment by normative concepts of intrinsic value.
UniMorph 2.0: Universal Morphology
Authors: Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J. Mielke, Arya D. McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner, Mans Hulden
Date: 2018-10-25 00:00:00
Proceeding: LREC 2018 5
Tasks: LEMMA

The Universal Morphology UniMorph project is a collaborative effort to improve how NLP handles complex morphology across the world's languages. The project releases annotated morphological data using a universal tagset, the UniMorph schema. Each inflected form is associated with a lemma, which typically carries its underlying lexical meaning, and a bundle of morphological features from our schema. Additional supporting data and tools are also released on a per-language basis when available. UniMorph is based at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University in Baltimore, Maryland and is sponsored by the DARPA LORELEI program. This paper details advances made to the collection, annotation, and dissemination of project resources since the initial UniMorph release described at LREC 2016. lexical resources} }
Provable Gaussian Embedding with One Observation
Authors: Ming Yu, Zhuoran Yang, Tuo Zhao, Mladen Kolar, Zhaoran Wang
Date: 2018-10-25 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning

The success of machine learning methods heavily relies on having an
appropriate representation for data at hand. Traditionally, machine learning
approaches relied on user-defined heuristics to extract features encoding
structural information about data. However, recently there has been a surge in
approaches that learn how to encode the data automatically in a low dimensional
space. Exponential family embedding provides a probabilistic framework for
learning low-dimensional representation for various types of high-dimensional
data. Though successful in practice, theoretical underpinnings for exponential
family embeddings have not been established. In this paper, we study the
Gaussian embedding model and develop the first theoretical results for
exponential family embedding models. First, we show that, under mild condition,
the embedding structure can be learned from one observation by leveraging the
parameter sharing between different contexts even though the data are dependent
with each other. Second, we study properties of two algorithms used for
learning the embedding structure and establish convergence results for each of
them. The first algorithm is based on a convex relaxation, while the other
solved the non-convex formulation of the problem directly. Experiments
demonstrate the effectiveness of our approach.
Radiomic Synthesis Using Deep Convolutional Neural Networks
Authors: Vishwa S. Parekh, Michael A. Jacobs
Date: 2018-10-25 00:00:00

Radiomics is a rapidly growing field that deals with modeling the textural information present in the different tissues of interest for clinical decision support. However, the process of generating radiomic images is computationally very expensive and could take substantial time per radiological image for certain higher order features, such as, gray-level co-occurrence matrix(GLCM), even with high-end GPUs. To that end, we developed RadSynth, a deep convolutional neural network(CNN) model, to efficiently generate radiomic images. RadSynth was tested on a breast cancer patient cohort of twenty-four patients(ten benign, ten malignant and four normal) for computation of GLCM entropy images from post-contrast DCE-MRI. RadSynth produced excellent synthetic entropy images compared to traditional GLCM entropy images. The average percentage difference and correlation between the two techniques were 0.07 $\pm$ 0.06 and 0.97, respectively. In conclusion, RadSynth presents a new powerful tool for fast computation and visualization of the textural information present in the radiological images.
Generalised framework for multi-criteria method selection
Authors: Jarosław Wątróbski, Jarosław Jankowski, Paweł Ziemba, Artur Karczmarczyk, Magdalena Zioło
Date: 2018-10-25 00:00:00
Tasks: Decision Making

Multi-Criteria Decision Analysis (MCDA) methods are widely used in various
fields and disciplines. While most of the research has been focused on the
development and improvement of new MCDA methods, relatively limited attention
has been paid to their appropriate selection for the given decision problem.
Their improper application decreases the quality of recommendations, as
different MCDA methods deliver inconsistent results. The current paper presents
a methodological and practical framework for selecting suitable MCDA methods
for a particular decision situation. A set of 56 available MCDA methods was
analyzed and, based on that, a hierarchical set of methods characteristics and
the rule base were obtained. This analysis, rules and modelling of the
uncertainty in the decision problem description allowed to build a framework
supporting the selection of a MCDA method for a given decision-making
situation. The practical studies indicate consistency between the methods
recommended with the proposed approach and those used by the experts in
reference cases. The results of the research also showed that the proposed
approach can be used as a general framework for selecting an appropriate MCDA
method for a given area of decision support, even in cases of data gaps in the
decision-making problem description. The proposed framework was implemented
within a web platform available for public use at www.mcda.it.
RELF: Robust Regression Extended with Ensemble Loss Function
Authors: Hamideh Hajiabadi, Reza Monsefi, Hadi Sadoghi Yazdi
Date: 2018-10-25 00:00:00
Tasks: Meta-Learning, regression

Ensemble techniques are powerful approaches that combine several weak
learners to build a stronger one. As a meta-learning framework, ensemble
techniques can easily be applied to many machine learning methods. Inspired by
ensemble techniques, in this paper we propose an ensemble loss functions
applied to a simple regressor. We then propose a half-quadratic learning
algorithm in order to find the parameter of the regressor and the optimal
weights associated with each loss function. Moreover, we show that our proposed
loss function is robust in noisy environments. For a particular class of loss
functions, we show that our proposed ensemble loss function is Bayes consistent
and robust. Experimental evaluations on several datasets demonstrate that our
proposed ensemble loss function significantly improves the performance of a
simple regressor in comparison with state-of-the-art methods.
Teaching Syntax by Adversarial Distraction
Authors: Ju-ho Kim, Christopher Malon, Asim Kadav
Date: 2018-10-25 00:00:00
Proceeding: WS 2018 11
Tasks: General Classification

Existing entailment datasets mainly pose problems which can be answered
without attention to grammar or word order. Learning syntax requires comparing
examples where different grammar and word order change the desired
classification. We introduce several datasets based on synthetic
transformations of natural entailment examples in SNLI or FEVER, to teach
aspects of grammar and word order. We show that without retraining, popular
entailment models are unaware that these syntactic differences change meaning.
With retraining, some but not all popular entailment models can learn to
compare the syntax properly.
Automating Generation of Low Precision Deep Learning Operators
Authors: Meghan Cowan, Thierry Moreau, Tianqi Chen, Luis Ceze
Date: 2018-10-25 00:00:00
Tasks: CPU, Deep Learning

State of the art deep learning models have made steady progress in the fields
of computer vision and natural language processing, at the expense of growing
model sizes and computational complexity. Deploying these models on low power
and mobile devices poses a challenge due to their limited compute capabilities
and strict energy budgets. One solution that has generated significant research
interest is deploying highly quantized models that operate on low precision
inputs and weights less than eight bits, trading off accuracy for performance.
These models have a significantly reduced memory footprint (up to 32x
reduction) and can replace multiply-accumulates with bitwise operations during
compute intensive convolution and fully connected layers.
  Most deep learning frameworks rely on highly engineered linear algebra
libraries such as ATLAS or Intel's MKL to implement efficient deep learning
operators. To date, none of the popular deep learning directly support low
precision operators, partly due to a lack of optimized low precision libraries.
In this paper we introduce a work flow to quickly generate high performance low
precision deep learning operators for arbitrary precision that target multiple
CPU architectures and include optimizations such as memory tiling and
vectorization. We present an extensive case study on low power ARM Cortex-A53
CPU, and show how we can generate 1-bit, 2-bit convolutions with speedups up to
16x over an optimized 16-bit integer baseline and 2.3x better than handwritten
implementations.
Uniform Convergence of Gradients for Non-Convex Learning and Optimization
Authors: Dylan J. Foster, Ayush Sekhari, Karthik Sridharan
Date: 2018-10-25 00:00:00
Proceeding: NeurIPS 2018 12

We investigate 1) the rate at which refined properties of the empirical
risk---in particular, gradients---converge to their population counterparts in
standard non-convex learning tasks, and 2) the consequences of this convergence
for optimization. Our analysis follows the tradition of norm-based capacity
control. We propose vector-valued Rademacher complexities as a simple,
composable, and user-friendly tool to derive dimension-free uniform convergence
bounds for gradients in non-convex learning problems. As an application of our
techniques, we give a new analysis of batch gradient descent methods for
non-convex generalized linear models and non-convex robust regression, showing
how to use any algorithm that finds approximate stationary points to obtain
optimal sample complexity, even when dimension is high or possibly infinite and
multiple passes over the dataset are allowed.
  Moving to non-smooth models we show----in contrast to the smooth case---that
even for a single ReLU it is not possible to obtain dimension-independent
convergence rates for gradients in the worst case. On the positive side, it is
still possible to obtain dimension-independent rates under a new type of
distributional assumption.
Design Flow of Accelerating Hybrid Extremely Low Bit-width Neural Network in Embedded FPGA
Authors: Junsong Wang, Qiuwen Lou, Xiaofan Zhang, Chao Zhu, Yonghua Lin, Deming Chen
Date: 2018-07-31 00:00:00
Tasks: Edge-computing, GPU, Quantization

Neural network accelerators with low latency and low energy consumption are
desirable for edge computing. To create such accelerators, we propose a design
flow for accelerating the extremely low bit-width neural network (ELB-NN) in
embedded FPGAs with hybrid quantization schemes. This flow covers both network
training and FPGA-based network deployment, which facilitates the design space
exploration and simplifies the tradeoff between network accuracy and
computation efficiency. Using this flow helps hardware designers to deliver a
network accelerator in edge devices under strict resource and power
constraints. We present the proposed flow by supporting hybrid ELB settings
within a neural network. Results show that our design can deliver very high
performance peaking at 10.3 TOPS and classify up to 325.3 image/s/watt while
running large-scale neural networks for less than 5W using embedded FPGA. To
the best of our knowledge, it is the most energy efficient solution in
comparison to GPU or other FPGA implementations reported so far in the
literature.
One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks
Authors: Tianhe Yu, Pieter Abbeel, Sergey Levine, Chelsea Finn
Date: 2018-10-25 00:00:00
Tasks: Imitation Learning

We consider the problem of learning multi-stage vision-based tasks on a real
robot from a single video of a human performing the task, while leveraging
demonstration data of subtasks with other objects. This problem presents a
number of major challenges. Video demonstrations without teleoperation are easy
for humans to provide, but do not provide any direct supervision. Learning
policies from raw pixels enables full generality but calls for large function
approximators with many parameters to be learned. Finally, compound tasks can
require impractical amounts of demonstration data, when treated as a monolithic
skill. To address these challenges, we propose a method that learns both how to
learn primitive behaviors from video demonstrations and how to dynamically
compose these behaviors to perform multi-stage tasks by "watching" a human
demonstrator. Our results on a simulated Sawyer robot and real PR2 robot
illustrate our method for learning a variety of order fulfillment and kitchen
serving tasks with novel objects and raw pixel inputs.
On the dissection of degenerate cosmologies with machine learning
Authors: Julian Merten, Carlo Giocoli, Marco Baldi, Massimo Meneghetti, Austin Peel, Florian Lalande, Jean-Luc Starck, Valeria Pettorino
Date: 2018-10-25 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Pathfinder, Robust classification

Based on the DUSTGRAIN-pathfinder suite of simulations, we investigate
observational degeneracies between nine models of modified gravity and massive
neutrinos. Three types of machine learning techniques are tested for their
ability to discriminate lensing convergence maps by extracting dimensional
reduced representations of the data. Classical map descriptors such as the
power spectrum, peak counts and Minkowski functionals are combined into a joint
feature vector and compared to the descriptors and statistics that are common
to the field of digital image processing. To learn new features directly from
the data we use a Convolutional Neural Network (CNN). For the mapping between
feature vectors and the predictions of their underlying model, we implement two
different classifiers; one based on a nearest-neighbour search and one that is
based on a fully connected neural network. We find that the neural network
provides a much more robust classification than the nearest-neighbour approach
and that the CNN provides the most discriminating representation of the data.
It achieves the cleanest separation between the different models and the
highest classification success rate of 59% for a single source redshift. Once
we perform a tomographic CNN analysis, the total classification accuracy
increases significantly to 76% with no observational degeneracies remaining.
Visualising the filter responses of the CNN at different network depths
provides us with the unique opportunity to learn from very complex models and
to understand better why they perform so well.
Superensemble Classifier for Improving Predictions in Imbalanced Datasets
Authors: Tanujit Chakraborty, Ashis Kumar Chakraborty
Date: 2018-10-25 00:00:00
Tasks: feature selection, General Classification, imbalanced classification

Learning from an imbalanced dataset is a tricky proposition. Because these
datasets are biased towards one class, most existing classifiers tend not to
perform well on minority class examples. Conventional classifiers usually aim
to optimize the overall accuracy without considering the relative distribution
of each class. This article presents a superensemble classifier, to tackle and
improve predictions in imbalanced classification problems, that maps Hellinger
distance decision trees (HDDT) into radial basis function network (RBFN)
framework. Regularity conditions for universal consistency and the idea of
parameter optimization of the proposed model are provided. The proposed
distribution-free model can be applied for feature selection cum imbalanced
classification problems. We have also provided enough numerical evidence using
various real-life data sets to assess the performance of the proposed model.
Its effectiveness and competitiveness with respect to different
state-of-the-art models are shown.
Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories
Authors: Naima Otberdout, Anis Kacem, Mohamed Daoudi, Lahoucine Ballihi, Stefano Berretti
Date: 2018-10-25 00:00:00
Tasks: Classification, Facial Expression Recognition, Facial Expression Recognition (FER), General Classification, valid

In this paper, we propose a new approach for facial expression recognition using deep covariance descriptors. The solution is based on the idea of encoding local and global Deep Convolutional Neural Network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By conducting the classification of static facial expressions using Support Vector Machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that both the proposed static and dynamic approaches achieve state-of-the-art performance for facial expression recognition outperforming many recent approaches.