pwc-archive/papers-with-abstracts huggingface dataset:
Utilizing Class Information for Deep Network Representation Shaping
Authors: Daeyoung Choi, Wonjong Rhee
Date: 2018-09-25 00:00:00
Tasks: General Classification

Statistical characteristics of deep network representations, such as sparsity
and correlation, are known to be relevant to the performance and
interpretability of deep learning. When a statistical characteristic is
desired, often an adequate regularizer can be designed and applied during the
training phase. Typically, such a regularizer aims to manipulate a statistical
characteristic over all classes together. For classification tasks, however, it
might be advantageous to enforce the desired characteristic per class such that
different classes can be better distinguished. Motivated by the idea, we design
two class-wise regularizers that explicitly utilize class information:
class-wise Covariance Regularizer (cw-CR) and class-wise Variance Regularizer
(cw-VR). cw-CR targets to reduce the covariance of representations calculated
from the same class samples for encouraging feature independence. cw-VR is
similar, but variance instead of covariance is targeted to improve feature
compactness. For the sake of completeness, their counterparts without using
class information, Covariance Regularizer (CR) and Variance Regularizer (VR),
are considered together. The four regularizers are conceptually simple and
computationally very efficient, and the visualization shows that the
regularizers indeed perform distinct representation shaping. In terms of
classification performance, significant improvements over the baseline and
L1/L2 weight regularization methods were found for 21 out of 22 tasks over
popular benchmark datasets. In particular, cw-VR achieved the best performance
for 13 tasks including ResNet-32/110.
Triply Supervised Decoder Networks for Joint Detection and Segmentation
Authors: Jiale Cao, Yanwei Pang, Xuelong. Li
Date: 2018-09-25 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, object-detection, Object Detection, Segmentation, Self-Driving Cars, Semantic Segmentation

Joint object detection and semantic segmentation can be applied to many
fields, such as self-driving cars and unmanned surface vessels. An initial and
important progress towards this goal has been achieved by simply sharing the
deep convolutional features for the two tasks. However, this simple scheme is
unable to make full use of the fact that detection and segmentation are
mutually beneficial. To overcome this drawback, we propose a framework called
TripleNet where triple supervisions including detection-oriented supervision,
class-aware segmentation supervision, and class-agnostic segmentation
supervision are imposed on each layer of the decoder network. Class-agnostic
segmentation supervision provides an objectness prior knowledge for both
semantic segmentation and object detection. Besides the three types of
supervisions, two light-weight modules (i.e., inner-connected module and
attention skip-layer fusion) are also incorporated into each layer of the
decoder. In the proposed framework, detection and segmentation can sufficiently
boost each other. Moreover, class-agnostic and class-aware segmentation on each
decoder layer are not performed at the test stage. Therefore, no extra
computational costs are introduced at the test stage. Experimental results on
the VOC2007 and VOC2012 datasets demonstrate that the proposed TripleNet is
able to improve both the detection and segmentation accuracies without adding
extra computational costs.
Gradient-Based Low-Light Image Enhancement
Authors: Masayuki Tanaka, Takashi Shibata, Masatoshi Okutomi
Date: 2018-09-25 00:00:00
Tasks: Image Enhancement, Low-Light Image Enhancement

A low-light image enhancement is a highly demanded image processing
technique, especially for consumer digital cameras and cameras on mobile
phones. In this paper, a gradient-based low-light image enhancement algorithm
is proposed. The key is to enhance the gradients of dark region, because the
gradients are more sensitive for human visual system than absolute values. In
addition, we involve the intensity-range constraints for the image integration.
By using the intensity-range constraints, we can integrate the output image
with enhanced gradients preserving the given gradient information while
enforcing the intensity range of the output image within a certain intensity
range. Experiments demonstrate that the proposed gradient-based low-light image
enhancement can effectively enhance the low-light images.
Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
Authors: Xiang Kong, Qizhe Xie, Zihang Dai, Eduard Hovy
Date: 2018-09-25 00:00:00
Tasks: Image Captioning, Machine Translation, Text Generation, Translation

Mixture of Softmaxes (MoS) has been shown to be effective at addressing the expressiveness limitation of Softmax-based models. Despite the known advantage, MoS is practically sealed by its large consumption of memory and computational time due to the need of computing multiple Softmaxes. In this work, we set out to unleash the power of MoS in practical applications by investigating improved word coding schemes, which could effectively reduce the vocabulary size and hence relieve the memory and computation burden. We show both BPE and our proposed Hybrid-LightRNN lead to improved encoding mechanisms that can halve the time and memory consumption of MoS without performance losses. With MoS, we achieve an improvement of 1.5 BLEU scores on IWSLT 2014 German-to-English corpus and an improvement of 0.76 CIDEr score on image captioning. Moreover, on the larger WMT 2014 machine translation dataset, our MoS-boosted Transformer yields 29.5 BLEU score for English-to-German and 42.1 BLEU score for English-to-French, outperforming the single-Softmax Transformer by 0.8 and 0.4 BLEU scores respectively and achieving the state-of-the-art result on WMT 2014 English-to-German task.
Covfefe: A Computer Vision Approach For Estimating Force Exertion
Authors: Vaneet Aggarwal, Hamed Asadi, Mayank Gupta, Jae Joong Lee, Denny Yu
Date: 2018-09-25 00:00:00
Tasks: General Classification, Photoplethysmography (PPG)

Cumulative exposure to repetitive and forceful activities may lead to
musculoskeletal injuries which not only reduce workers' efficiency and
productivity, but also affect their quality of life. Thus, widely accessible
techniques for reliable detection of unsafe muscle force exertion levels for
human activity is necessary for their well-being. However, measurement of force
exertion levels is challenging and the existing techniques pose a great
challenge as they are either intrusive, interfere with human-machine interface,
and/or subjective in the nature, thus are not scalable for all workers. In this
work, we use face videos and the photoplethysmography (PPG) signals to classify
force exertion levels of 0\%, 50\%, and 100\% (representing rest, moderate
effort, and high effort), thus providing a non-intrusive and scalable approach.
Efficient feature extraction approaches have been investigated, including
standard deviation of the movement of different landmarks of the face,
distances between peaks and troughs in the PPG signals. We note that the PPG
signals can be obtained from the face videos, thus giving an efficient
classification algorithm for the force exertion levels using face videos. Based
on the data collected from 20 subjects, features extracted from the face videos
give 90\% accuracy in classification among the 100\% and the combination of 0\%
and 50\% datasets. Further combining the PPG signals provide 81.7\% accuracy.
The approach is also shown to be robust to the correctly identify force level
when the person is talking, even though such datasets are not included in the
training.
MedAL: Deep Active Learning Sampling Method for Medical Image Analysis
Authors: Asim Smailagic, Hae Young Noh, Pedro Costa, Devesh Walawalkar, Kartik Khandelwal, Mostafa Mirshekari, Jonathon Fagert, Adrián Galdrán, Susu Xu
Date: 2018-09-25 00:00:00
Tasks: Active Learning, Diabetic Retinopathy Detection, Medical Image Analysis

Deep learning models have been successfully used in medical image analysis
problems but they require a large amount of labeled images to obtain good
performance.Deep learning models have been successfully used in medical image
analysis problems but they require a large amount of labeled images to obtain
good performance. However, such large labeled datasets are costly to acquire.
Active learning techniques can be used to minimize the number of required
training labels while maximizing the model's performance.In this work, we
propose a novel sampling method that queries the unlabeled examples that
maximize the average distance to all training set examples in a learned feature
space. We then extend our sampling method to define a better initial training
set, without the need for a trained model, by using ORB feature descriptors. We
validate MedAL on 3 medical image datasets and show that our method is robust
to different dataset properties. MedAL is also efficient, achieving 80%
accuracy on the task of Diabetic Retinopathy detection using only 425 labeled
images, corresponding to a 32% reduction in the number of required labeled
examples compared to the standard uncertainty sampling technique, and a 40%
reduction compared to random sampling.
Tree-Based Optimization: A Meta-Algorithm for Metaheuristic Optimization
Authors: Benyamin Ghojogh, Saeed Sharifian, Hoda Mohammadzade
Date: 2018-09-25 00:00:00
Tasks: Metaheuristic Optimization

Designing search algorithms for finding global optima is one of the most
active research fields, recently. These algorithms consist of two main
categories, i.e., classic mathematical and metaheuristic algorithms. This
article proposes a meta-algorithm, Tree-Based Optimization (TBO), which uses
other heuristic optimizers as its sub-algorithms in order to improve the
performance of search. The proposed algorithm is based on mathematical tree
subject and improves performance and speed of search by iteratively removing
parts of the search space having low fitness, in order to minimize and purify
the search space. The experimental results on several well-known benchmarks
show the outperforming performance of TBO algorithm in finding the global
solution. Experiments on high dimensional search spaces show significantly
better performance when using the TBO algorithm. The proposed algorithm
improves the search algorithms in both accuracy and speed aspects, especially
for high dimensional searching such as in VLSI CAD tools for Integrated Circuit
(IC) design.
Graph filtering for data reduction and reconstruction
Authors: Ioannis D. Schizas
Date: 2018-09-25 00:00:00
Tasks: Dimensionality Reduction

A novel approach is put forth that utilizes data similarity, quantified on a
graph, to improve upon the reconstruction performance of principal component
analysis. The tasks of data dimensionality reduction and reconstruction are
formulated as graph filtering operations, that enable the exploitation of data
node connectivity in a graph via the adjacency matrix. The unknown reducing and
reconstruction filters are determined by optimizing a mean-square error cost
that entails the data, as well as their graph adjacency matrix. Working in the
graph spectral domain enables the derivation of simple gradient descent
recursions used to update the matrix filter taps. Numerical tests in real image
datasets demonstrate the better reconstruction performance of the novel method
over standard principal component analysis.
Neural Networks with Structural Resistance to Adversarial Attacks
Authors: Luca de Alfaro
Date: 2018-09-25 00:00:00
Proceeding: ICLR 2019 5

In adversarial attacks to machine-learning classifiers, small perturbations
are added to input that is correctly classified. The perturbations yield
adversarial examples, which are virtually indistinguishable from the
unperturbed input, and yet are misclassified. In standard neural networks used
for deep learning, attackers can craft adversarial examples from most input to
cause a misclassification of their choice.
  We introduce a new type of network units, called RBFI units, whose non-linear
structure makes them inherently resistant to adversarial attacks. On
permutation-invariant MNIST, in absence of adversarial attacks, networks using
RBFI units match the performance of networks using sigmoid units, and are
slightly below the accuracy of networks with ReLU units. When subjected to
adversarial attacks, networks with RBFI units retain accuracies above 90% for
attacks that degrade the accuracy of networks with ReLU or sigmoid units to
below 2%. RBFI networks trained with regular input are superior in their
resistance to adversarial attacks even to ReLU and sigmoid networks trained
with the help of adversarial examples.
  The non-linear structure of RBFI units makes them difficult to train using
standard gradient descent. We show that networks of RBFI units can be
efficiently trained to high accuracies using pseudogradients, computed using
functions especially crafted to facilitate learning instead of their true
derivatives. We show that the use of pseudogradients makes training deep RBFI
networks practical, and we compare several structural alternatives of RBFI
networks for their accuracy.
Resilient Computing with Reinforcement Learning on a Dynamical System: Case Study in Sorting
Authors: Aleksandra Faust, James B. Aimone, Conrad D. James, Lydia Tapia
Date: 2018-09-25 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Robots and autonomous agents often complete goal-based tasks with limited
resources, relying on imperfect models and sensor measurements. In particular,
reinforcement learning (RL) and feedback control can be used to help a robot
achieve a goal. Taking advantage of this body of work, this paper formulates
general computation as a feedback-control problem, which allows the agent to
autonomously overcome some limitations of standard procedural language
programming: resilience to errors and early program termination. Our
formulation considers computation to be trajectory generation in the program's
variable space. The computing then becomes a sequential decision making
problem, solved with reinforcement learning (RL), and analyzed with Lyapunov
stability theory to assess the agent's resilience and progression to the goal.
We do this through a case study on a quintessential computer science problem,
array sorting. Evaluations show that our RL sorting agent makes steady progress
to an asymptotically stable goal, is resilient to faulty components, and
performs less array manipulations than traditional Quicksort and Bubble sort.
Low Precision Policy Distillation with Application to Low-Power, Real-time Sensation-Cognition-Action Loop with Neuromorphic Computing
Authors: Jeffrey L. McKinstry, Davis R. Barch, Deepika Bablani, Michael V. Debole, Steven K. Esser, Jeffrey A. Kusnitz, John V. Arthur, Dharmendra S. Modha
Date: 2018-09-25 00:00:00
Tasks: Atari Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Low precision networks in the reinforcement learning (RL) setting are
relatively unexplored because of the limitations of binary activations for
function approximation. Here, in the discrete action ATARI domain, we
demonstrate, for the first time, that low precision policy distillation from a
high precision network provides a principled, practical way to train an RL
agent. As an application, on 10 different ATARI games, we demonstrate real-time
end-to-end game playing on low-power neuromorphic hardware by converting a
sequence of game frames into discrete actions.
Asynchronous decentralized accelerated stochastic gradient descent
Authors: Guanghui Lan, Yi Zhou
Date: 2018-09-24 00:00:00
Tasks: Stochastic Optimization

In this work, we introduce an asynchronous decentralized accelerated
stochastic gradient descent type of method for decentralized stochastic
optimization, considering communication and synchronization are the major
bottlenecks. We establish $\mathcal{O}(1/\epsilon)$ (resp.,
$\mathcal{O}(1/\sqrt{\epsilon})$) communication complexity and
$\mathcal{O}(1/\epsilon^2)$ (resp., $\mathcal{O}(1/\epsilon)$) sampling
complexity for solving general convex (resp., strongly convex) problems.
Evaluating Fairness Metrics in the Presence of Dataset Bias
Authors: J. Henry Hinnefeld, Peter Cooman, Nat Mammo, Rupert Deese
Date: 2018-09-24 00:00:00
Tasks: Bias Detection, Causal Inference, Decision Making, Fairness

Data-driven algorithms play a large role in decision making across a variety
of industries. Increasingly, these algorithms are being used to make decisions
that have significant ramifications for people's social and economic
well-being, e.g. in sentencing, loan approval, and policing. Amid the
proliferation of such systems there is a growing concern about their potential
discriminatory impact. In particular, machine learning systems which are
trained on biased data have the potential to learn and perpetuate those biases.
A central challenge for practitioners is thus to determine whether their models
display discriminatory bias. Here we present a case study in which we frame the
issue of bias detection as a causal inference problem with observational data.
We enumerate two main causes of bias, sampling bias and label bias, and we
investigate the abilities of six different fairness metrics to detect each bias
type. Based on these investigations, we propose a set of best practice
guidelines to select the fairness metric that is most likely to detect bias if
it is present. Additionally, we aim to identify the conditions in which certain
fairness metrics may fail to detect bias and instead give practitioners a false
belief that their biased model is making fair decisions.
No Multiplication? No Floating Point? No Problem! Training Networks for Efficient Inference
Authors: Shumeet Baluja, David Marwood, Michele Covell, Nick Johnston
Date: 2018-09-24 00:00:00
Tasks: Multi-class Classification

For successful deployment of deep neural networks on
highly--resource-constrained devices (hearing aids, earbuds, wearables), we
must simplify the types of operations and the memory/power resources used
during inference. Completely avoiding inference-time floating-point operations
is one of the simplest ways to design networks for these highly-constrained
environments. By discretizing both our in-network non-linearities and our
network weights, we can move to simple, compact networks without floating point
operations, without multiplications, and avoid all non-linear function
computations. Our approach allows us to explore the spectrum of possible
networks, ranging from fully continuous versions down to networks with bi-level
weights and activations. Our results show that discretization can be done
without loss of performance and that we can train a network that will
successfully operate without floating-point, without multiplication, and with
less RAM on both regression tasks (auto encoding) and multi-class
classification tasks (ImageNet). The memory needed to deploy our discretized
networks is less than one third of the equivalent architecture that does use
floating-point operations.
Flexible Mixture Modeling on Constrained Spaces
Authors: Putu Ayu Sudyanti, Vinayak Rao
Date: 2018-09-24 00:00:00
Tasks: Bayesian Inference, Data Augmentation

This paper addresses challenges in flexibly modeling multimodal data that lie on constrained spaces. Such data are commonly found in spatial applications, such as climatology and criminology, where measurements are restricted to a geographical area. Other settings include domains where unsuitable recordings are discarded, such as flow-cytometry measurements. A simple approach to modeling such data is through the use of mixture models, especially nonparametric mixture models. Mixture models, while flexible and theoretically well-understood, are unsuitable for settings involving complicated constraints, leading to difficulties in specifying the component distributions and in evaluating normalization constants. Bayesian inference over the parameters of these models results in posterior distributions that are doubly-intractable. We address this problem via an algorithm based on rejection sampling and data augmentation. We view samples from a truncated distribution as outcomes of a rejection sampling scheme, where proposals are made from a simple mixture model and are rejected if they violate the constraints. Our scheme proceeds by imputing the rejected samples given mixture parameters and then resampling parameters given all samples. We study two modeling approaches: mixtures of truncated Gaussians and truncated mixtures of Gaussians, along with their associated Markov chain Monte Carlo sampling algorithms. We also discuss variations of the models, as well as approximations to improve mixing, reduce computational cost, and lower variance. We present results on simulated data and apply our algorithms to two problems; one involving flow-cytometry data, and the other, crime recorded in the city of Chicago.
Towards Automated Post-Earthquake Inspections with Deep Learning-based Condition-Aware Models
Authors: Vedhus Hoskere, Yasutaka Narazaki, Tu A. Hoang, Billie F. Spencer Jr
Date: 2018-09-24 00:00:00
Tasks: Semantic Segmentation

In the aftermath of an earthquake, rapid structural inspections are required
to get citizens back in to their homes and offices in a safe and timely manner.
These inspections gfare typically conducted by municipal authorities through
structural engineer volunteers. As manual inspec-tions can be time consuming,
laborious and dangerous, research has been underway to develop methods to help
speed up and increase the automation of the entire process. Researchers
typi-cally envisage the use of unmanned aerial vehicles (UAV) for data
acquisition and computer vision for data processing to extract actionable
information. In this work we propose a new framework to generate vision-based
condition-aware models that can serve as the basis for speeding up or
automating higher level inspection decisions. The condition-aware models are
generated by projecting the inference of trained deep-learning models on a set
of images of a structure onto a 3D mesh model generated through multi-view
stereo from the same image set. Deep fully convolutional residual networks are
used for semantic segmentation of images of buildings to provide (i) damage
information such as cracks and spalling (ii) contextual infor-mation such as
the presence of a building and visually identifiable components like windows
and doors. The proposed methodology was implemented on a damaged building that
was sur-veyed by the authors after the Central Mexico Earthquake in September
2017 and qualitative-ly evaluated. Results demonstrate the promise of the
proposed method towards the ultimate goal of rapid and automated
post-earthquake inspections.
Stochastic Answer Networks for SQuAD 2.0
Authors: Xiaodong Liu, Wei Li, Yuwei Fang, Aerin Kim, Kevin Duh, Jianfeng Gao
Date: 2018-09-24 00:00:00
Tasks: Machine Reading Comprehension, Question Answering, Reading Comprehension

This paper presents an extension of the Stochastic Answer Network (SAN), one
of the state-of-the-art machine reading comprehension models, to be able to
judge whether a question is unanswerable or not. The extended SAN contains two
components: a span detector and a binary classifier for judging whether the
question is unanswerable, and both components are jointly optimized.
Experiments show that SAN achieves the results competitive to the
state-of-the-art on Stanford Question Answering Dataset (SQuAD) 2.0. To
facilitate the research on this field, we release our code:
https://github.com/kevinduh/san_mrc.