pwc-archive/papers-with-abstracts huggingface dataset:
On Discrimination Discovery and Removal in Ranked Data using Causal Graph
Authors: Yongkai Wu, Lu Zhang, Xintao Wu
Date: 2018-03-05 00:00:00
Tasks: Fairness

Predictive models learned from historical data are widely used to help
companies and organizations make decisions. However, they may digitally
unfairly treat unwanted groups, raising concerns about fairness and
discrimination. In this paper, we study the fairness-aware ranking problem
which aims to discover discrimination in ranked datasets and reconstruct the
fair ranking. Existing methods in fairness-aware ranking are mainly based on
statistical parity that cannot measure the true discriminatory effect since
discrimination is causal. On the other hand, existing methods in causal-based
anti-discrimination learning focus on classification problems and cannot be
directly applied to handle the ranked data. To address these limitations, we
propose to map the rank position to a continuous score variable that represents
the qualification of the candidates. Then, we build a causal graph that
consists of both the discrete profile attributes and the continuous score. The
path-specific effect technique is extended to the mixed-variable causal graph
to identify both direct and indirect discrimination. The relationship between
the path-specific effects for the ranked data and those for the binary decision
is theoretically analyzed. Finally, algorithms for discovering and removing
discrimination from a ranked dataset are developed. Experiments using the real
dataset show the effectiveness of our approaches.
Style Memory: Making a Classifier Network Generative
Authors: Rey Wiyatno, Jeff Orchard
Date: 2018-03-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Classification, General Classification

Deep networks have shown great performance in classification tasks. However,
the parameters learned by the classifier networks usually discard stylistic
information of the input, in favour of information strictly relevant to
classification. We introduce a network that has the capacity to do both
classification and reconstruction by adding a "style memory" to the output
layer of the network. We also show how to train such a neural network as a deep
multi-layer autoencoder, jointly minimizing both classification and
reconstruction losses. The generative capacity of our network demonstrates that
the combination of style-memory neurons with the classifier neurons yield good
reconstructions of the inputs when the classification is correct. We further
investigate the nature of the style memory, and how it relates to composing
digits and letters. Finally, we propose that this architecture enables the
bidirectional flow of information used in predictive coding, and that such
bidirectional networks can help mitigate against being fooled by ambiguous or
adversarial input.
ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
Authors: Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, Simon Lucey
Date: 2018-03-05 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generative Adversarial Network

We address the problem of finding realistic geometric corrections to a
foreground object such that it appears natural when composited into a
background image. To achieve this, we propose a novel Generative Adversarial
Network (GAN) architecture that utilizes Spatial Transformer Networks (STNs) as
the generator, which we call Spatial Transformer GANs (ST-GANs). ST-GANs seek
image realism by operating in the geometric warp parameter space. In
particular, we exploit an iterative STN warping scheme and propose a sequential
training strategy that achieves better results compared to naive training of a
single generator. One of the key advantages of ST-GAN is its applicability to
high-resolution images indirectly since the predicted warp parameters are
transferable between reference frames. We demonstrate our approach in two
applications: (1) visualizing how indoor furniture (e.g. from product images)
might be perceived in a room, (2) hallucinating how accessories like glasses
would look when matched with real portraits.
Conducting Credit Assignment by Aligning Local Representations
Authors: Alexander G. Ororbia, Ankur Mali, Daniel Kifer, C. Lee Giles
Date: 2018-03-05 00:00:00

Using back-propagation and its variants to train deep networks is often
problematic for new users. Issues such as exploding gradients, vanishing
gradients, and high sensitivity to weight initialization strategies often make
networks difficult to train, especially when users are experimenting with new
architectures. Here, we present Local Representation Alignment (LRA), a
training procedure that is much less sensitive to bad initializations, does not
require modifications to the network architecture, and can be adapted to
networks with highly nonlinear and discrete-valued activation functions.
Furthermore, we show that one variation of LRA can start with a null
initialization of network weights and still successfully train networks with a
wide variety of nonlinearities, including tanh, ReLU-6, softplus, signum and
others that may draw their inspiration from biology.
  A comprehensive set of experiments on MNIST and the much harder Fashion MNIST
data sets show that LRA can be used to train networks robustly and effectively,
succeeding even when back-propagation fails and outperforming other alternative
learning algorithms, such as target propagation and feedback alignment.
L2-Nonexpansive Neural Networks
Authors: Haifeng Qian, Mark N. Wegman
Date: 2018-02-22 00:00:00
Proceeding: ICLR 2019 5

This paper proposes a class of well-conditioned neural networks in which a
unit amount of change in the inputs causes at most a unit amount of change in
the outputs or any of the internal layers. We develop the known methodology of
controlling Lipschitz constants to realize its full potential in maximizing
robustness, with a new regularization scheme for linear layers, new ways to
adapt nonlinearities and a new loss function. With MNIST and CIFAR-10
classifiers, we demonstrate a number of advantages. Without needing any
adversarial training, the proposed classifiers exceed the state of the art in
robustness against white-box L2-bounded adversarial attacks. They generalize
better than ordinary networks from noisy data with partially random labels.
Their outputs are quantitatively meaningful and indicate levels of confidence
and generalization, among other desirable properties.
Visualizing the Loss Landscape of Neural Nets
Authors: Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein
Date: 2017-12-28 00:00:00
Proceeding: ICLR 2018 1

Neural network training relies on our ability to find "good" minimizers of
highly non-convex loss functions. It is well-known that certain network
architecture designs (e.g., skip connections) produce loss functions that train
easier, and well-chosen training parameters (batch size, learning rate,
optimizer) produce minimizers that generalize better. However, the reasons for
these differences, and their effects on the underlying loss landscape, are not
well understood. In this paper, we explore the structure of neural loss
functions, and the effect of loss landscapes on generalization, using a range
of visualization methods. First, we introduce a simple "filter normalization"
method that helps us visualize loss function curvature and make meaningful
side-by-side comparisons between loss functions. Then, using a variety of
visualizations, we explore how network architecture affects the loss landscape,
and how training parameters affect the shape of minimizers.
Understanding the Loss Surface of Neural Networks for Binary Classification
Authors: Shiyu Liang, Ruoyu Sun, Yixuan Li, R. Srikant
Date: 2018-02-19 00:00:00
Proceeding: ICML 2018 7
Tasks: Binary Classification, Classification, General Classification

It is widely conjectured that the reason that training algorithms for neural
networks are successful because all local minima lead to similar performance,
for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al.,
2014). Performance is typically measured in terms of two metrics: training
performance and generalization performance. Here we focus on the training
performance of single-layered neural networks for binary classification, and
provide conditions under which the training error is zero at all local minima
of a smooth hinge loss function. Our conditions are roughly in the following
form: the neurons have to be strictly convex and the surrogate loss function
should be a smooth version of hinge loss. We also provide counterexamples to
show that when the loss function is replaced with quadratic loss or logistic
loss, the result may not hold.
PronouncUR: An Urdu Pronunciation Lexicon Generator
Authors: Haris Bin Zia, Agha Ali Raza, Awais Athar
Date: 2018-01-01 00:00:00
Proceeding: LREC 2018 5
Tasks: Grapheme-to-Phoneme Conversion, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

State-of-the-art speech recognition systems rely heavily on three basic
components: an acoustic model, a pronunciation lexicon and a language model. To
build these components, a researcher needs linguistic as well as technical
expertise, which is a barrier in low-resource domains. Techniques to construct
these three components without having expert domain knowledge are in great
demand. Urdu, despite having millions of speakers all over the world, is a
low-resource language in terms of standard publically available linguistic
resources. In this paper, we present a grapheme-to-phoneme conversion tool for
Urdu that generates a pronunciation lexicon in a form suitable for use with
speech recognition systems from a list of Urdu words. The tool predicts the
pronunciation of words using a LSTM-based model trained on a handcrafted expert
lexicon of around 39,000 words and shows an accuracy of 64% upon internal
evaluation. For external evaluation on a speech recognition task, we obtain a
word error rate comparable to one achieved using a fully handcrafted expert
lexicon.
Event-triggered Learning for Resource-efficient Networked Control
Authors: Friedrich Solowjow, Dominik Baumann, Jochen Garcke, Sebastian Trimpe
Date: 2018-03-05 00:00:00
Tasks: State Estimation

Common event-triggered state estimation (ETSE) algorithms save communication
in networked control systems by predicting agents' behavior, and transmitting
updates only when the predictions deviate significantly. The effectiveness in
reducing communication thus heavily depends on the quality of the dynamics
models used to predict the agents' states or measurements. Event-triggered
learning is proposed herein as a novel concept to further reduce communication:
whenever poor communication performance is detected, an identification
experiment is triggered and an improved prediction model learned from data.
Effective learning triggers are obtained by comparing the actual communication
rate with the one that is expected based on the current model. By analyzing
statistical properties of the inter-communication times and leveraging powerful
convergence results, the proposed trigger is proven to limit learning
experiments to the necessary instants. Numerical and physical experiments
demonstrate that event-triggered learning improves robustness toward changing
environments and yields lower communication rates than common ETSE.
Faithful Semantical Embedding of a Dyadic Deontic Logic in HOL
Authors: Christoph Benzmüller, Ali Farjami, Xavier Parent
Date: 2018-02-23 00:00:00

A shallow semantical embedding of a dyadic deontic logic by Carmo and Jones
in classical higher-order logic is presented. This embedding is proven sound
and complete, that is, faithful.
  The work presented here provides the theoretical foundation for the
implementation and automation of dyadic deontic logic within off-the-shelf
higher-order theorem provers and proof assistants.
AspEm: Embedding Learning by Aspects in Heterogeneous Information Networks
Authors: Yu Shi, Huan Gui, Qi Zhu, Lance Kaplan, Jiawei Han
Date: 2018-03-05 00:00:00
Tasks: Link Prediction, Network Embedding

Heterogeneous information networks (HINs) are ubiquitous in real-world
applications. Due to the heterogeneity in HINs, the typed edges may not fully
align with each other. In order to capture the semantic subtlety, we propose
the concept of aspects with each aspect being a unit representing one
underlying semantic facet. Meanwhile, network embedding has emerged as a
powerful method for learning network representation, where the learned
embedding can be used as features in various downstream applications.
Therefore, we are motivated to propose a novel embedding learning
framework---AspEm---to preserve the semantic information in HINs based on
multiple aspects. Instead of preserving information of the network in one
semantic space, AspEm encapsulates information regarding each aspect
individually. In order to select aspects for embedding purpose, we further
devise a solution for AspEm based on dataset-wide statistics. To corroborate
the efficacy of AspEm, we conducted experiments on two real-words datasets with
two types of applications---classification and link prediction. Experiment
results demonstrate that AspEm can outperform baseline network embedding
learning methods by considering multiple aspects, where the aspects can be
selected from the given HIN in an unsupervised manner.
A generalized parametric 3D shape representation for articulated pose estimation
Authors: Meng Ding, Guoliang Fan
Date: 2018-03-05 00:00:00
Tasks: 3D Shape Representation, Pose Estimation

We present a novel parametric 3D shape representation, Generalized sum of
Gaussians (G-SoG), which is particularly suitable for pose estimation of
articulated objects. Compared with the original sum-of-Gaussians (SoG), G-SoG
can handle both isotropic and anisotropic Gaussians, leading to a more flexible
and adaptable shape representation yet with much fewer anisotropic Gaussians
involved. An articulated shape template can be developed by embedding G-SoG in
a tree-structured skeleton model to represent an articulated object. We further
derive a differentiable similarity function between G-SoG (the template) and
SoG (observed data) that can be optimized analytically for efficient pose
estimation. The experimental results on a standard human pose estimation
dataset show the effectiveness and advantages of G-SoG over the original SoG as
well as the promise compared with the recent algorithms that use more
complicated shape models.
Kinematic Morphing Networks for Manipulation Skill Transfer
Authors: Peter Englert, Marc Toussaint
Date: 2018-03-05 00:00:00

The transfer of a robot skill between different geometric environments is
non-trivial since a wide variety of environments exists, sensor observations as
well as robot motions are high-dimensional, and the environment might only be
partially observed. We consider the problem of extracting a low-dimensional
description of the manipulated environment in form of a kinematic model. This
allows us to transfer a skill by defining a policy on a prototype model and
morphing the observed environment to this prototype. A deep neural network is
used to map depth image observations of the environment to morphing parameter,
which include transformation and configuration parameters of the prototype
model. Using the concatenation property of affine transformations and the
ability to convert point clouds to depth images allows to apply the network in
an iterative manner. The network is trained on data generated in a simulator
and on augmented data that is created by using network predictions. The
algorithm is evaluated on different tasks, where it is shown that iterative
predictions lead to a higher accuracy than one-step predictions.
Linking ImageNet WordNet Synsets with Wikidata
Authors: Finn Årup Nielsen
Date: 2018-03-05 00:00:00
Tasks: Deep Learning, General Classification, image-classification, Image Classification, Knowledge Graphs

The linkage of ImageNet WordNet synsets to Wikidata items will leverage deep
learning algorithm with access to a rich multilingual knowledge graph. Here I
will describe our on-going efforts in linking the two resources and issues
faced in matching the Wikidata and WordNet knowledge graphs. I show an example
on how the linkage can be used in a deep learning setting with real-time image
classification and labeling in a non-English language and discuss what
opportunities lies ahead.
XNORBIN: A 95 TOp/s/W Hardware Accelerator for Binary Convolutional Neural Networks
Authors: Andrawes Al Bahou, Geethan Karunaratne, Renzo Andri, Lukas Cavigelli, Luca Benini
Date: 2018-03-05 00:00:00
Tasks: Quantization

Deploying state-of-the-art CNNs requires power-hungry processors and off-chip
memory. This precludes the implementation of CNNs in low-power embedded
systems. Recent research shows CNNs sustain extreme quantization, binarizing
their weights and intermediate feature maps, thereby saving 8-32\x memory and
collapsing energy-intensive sum-of-products into XNOR-and-popcount operations.
  We present XNORBIN, an accelerator for binary CNNs with computation tightly
coupled to memory for aggressive data reuse. Implemented in UMC 65nm technology
XNORBIN achieves an energy efficiency of 95 TOp/s/W and an area efficiency of
2.0 TOp/s/MGE at 0.8 V.
How to Start Training: The Effect of Initialization and Architecture
Authors: Boris Hanin, David Rolnick
Date: 2018-03-05 00:00:00
Proceeding: NeurIPS 2018 12

We identify and study two common failure modes for early training in deep
ReLU nets. For each we give a rigorous proof of when it occurs and how to avoid
it, for fully connected and residual architectures. The first failure mode,
exploding/vanishing mean activation length, can be avoided by initializing
weights from a symmetric distribution with variance 2/fan-in and, for ResNets,
by correctly weighting the residual modules. We prove that the second failure
mode, exponentially large variance of activation length, never occurs in
residual nets once the first failure mode is avoided. In contrast, for fully
connected nets, we prove that this failure mode can happen and is avoided by
keeping constant the sum of the reciprocals of layer widths. We demonstrate
empirically the effectiveness of our theoretical results in predicting when
networks are able to start training. In particular, we note that many popular
initializations fail our criteria, whereas correct initialization and
architecture allows much deeper networks to be trained.
Neural Architectures for Open-Type Relation Argument Extraction
Authors: Benjamin Roth, Costanza Conforti, Nina Poerner, Sanjeev Karn, Hinrich Schütze
Date: 2018-03-05 00:00:00
Tasks: Question Answering, Relation, Sentence, Vocal Bursts Type Prediction

In this work, we introduce the task of Open-Type Relation Argument Extraction
(ORAE): Given a corpus, a query entity Q and a knowledge base relation (e.g.,"Q
authored notable work with title X"), the model has to extract an argument of
non-standard entity type (entities that cannot be extracted by a standard named
entity tagger, e.g. X: the title of a book or a work of art) from the corpus. A
distantly supervised dataset based on WikiData relations is obtained and
released to address the task.
  We develop and compare a wide range of neural models for this task yielding
large improvements over a strong baseline obtained with a neural question
answering system. The impact of different sentence encoding architectures and
answer extraction methods is systematically compared. An encoder based on gated
recurrent units combined with a conditional random fields tagger gives the best
results.
New Ideas for Brain Modelling 5
Authors: Kieran Greer
Date: 2018-03-05 00:00:00

This paper describes an automatic process for combining patterns and features, to guide a search process and make predictions. It is based on the functionality that a human brain might have, which is a highly distributed network of simple neuronal components that can apply some level of matching and cross-referencing over retrieved patterns. The process uses memory in a dynamic way and it is directed through the pattern matching. The first half of the paper describes the mechanisms for neuronal search, memory and prediction. The second half of the paper then presents a formal language for defining cognitive processes, that is, pattern-based sequences and transitions. The language can define an outer framework for nested pattern sets that can be linked to perform the cognitive act. The language also has a mathematical basis, allowing for the rule construction process to be systematic and consistent. The new information can be used to integrate the cognitive model together. A theory about linking can suggest that only (mostly) nodes that represent the same thing link together.
Q-Learning Algorithm for VoLTE Closed-Loop Power Control in Indoor Small Cells
Authors: Faris B. Mismar, Brian L. Evans
Date: 2017-07-10 00:00:00
Tasks: Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a reinforcement learning (RL) based closed loop power control
algorithm for the downlink of the voice over LTE (VoLTE) radio bearer for an
indoor environment served by small cells. The main contributions of our paper
are to 1) use RL to solve performance tuning problems in an indoor cellular
network for voice bearers and 2) show that our derived lower bound loss in
effective signal to interference plus noise ratio due to neighboring cell
failure is sufficient for VoLTE power control purposes in practical cellular
networks. In our simulation, the proposed RL-based power control algorithm
significantly improves both voice retainability and mean opinion score compared
to current industry standards. The improvement is due to maintaining an
effective downlink signal to interference plus noise ratio against adverse
network operational issues and faults.
Machine Learning in Downlink Coordinated Multipoint in Heterogeneous Networks
Authors: Faris B. Mismar, Brian L. Evans
Date: 2016-08-30 00:00:00
Tasks: BIG-bench Machine Learning, PICO

We propose a method for downlink coordinated multipoint (DL CoMP) in
heterogeneous fifth generation New Radio (NR) networks. The primary
contribution of our paper is an algorithm to enhance the trigger of DL CoMP
using online machine learning. We use support vector machine (SVM) classifiers
to enhance the user downlink throughput in a realistic frequency division
duplex network environment. Our simulation results show improvement in both the
macro and pico base station downlink throughputs due to the informed triggering
of the multiple radio streams as learned by the SVM classifier.
Tuning Over-Relaxed ADMM
Authors: Guilherme França, José Bento
Date: 2017-03-10 00:00:00
Tasks: valid

The framework of Integral Quadratic Constraints (IQC) reduces the computation
of upper bounds on the convergence rate of several optimization algorithms to a
semi-definite program (SDP). In the case of over-relaxed Alternating Direction
Method of Multipliers (ADMM), an explicit and closed form solution to this SDP
was derived in our recent work [1]. The purpose of this paper is twofold.
First, we summarize these results. Second, we explore one of its consequences
which allows us to obtain general and simple formulas for optimal parameter
selection. These results are valid for arbitrary strongly convex objective
functions.