pwc-archive/papers-with-abstracts huggingface dataset:
Large Scale Correlation Clustering Optimization
Authors: Shai Bagon, Meirav Galun
Date: 2011-12-13 00:00:00
Tasks: Clustering, Face Identification, Model Selection, Semantic Segmentation

Clustering is a fundamental task in unsupervised learning. The focus of this
paper is the Correlation Clustering functional which combines positive and
negative affinities between the data points. The contribution of this paper is
two fold: (i) Provide a theoretic analysis of the functional. (ii) New
optimization algorithms which can cope with large scale problems (>100K
variables) that are infeasible using existing methods. Our theoretic analysis
provides a probabilistic generative interpretation for the functional, and
justifies its intrinsic "model-selection" capability. Furthermore, we draw an
analogy between optimizing this functional and the well known Potts energy
minimization. This analogy allows us to suggest several new optimization
algorithms, which exploit the intrinsic "model-selection" capability of the
functional to automatically recover the underlying number of clusters. We
compare our algorithms to existing methods on both synthetic and real data. In
addition we suggest two new applications that are made possible by our
algorithms: unsupervised face identification and interactive multi-object
segmentation by rough boundary delineation.
The Off-Topic Memento Toolkit
Authors: Shawn M. Jones, Michele C. Weigle, Michael L. Nelson
Date: 2018-06-18 00:00:00

Web archive collections are created with a particular purpose in mind. A
curator selects seeds, or original resources, which are then captured by an
archiving system and stored as archived web pages, or mementos. The systems
that build web archive collections are often configured to revisit the same
original resource multiple times. This is incredibly useful for understanding
an unfolding news story or the evolution of an organization. Unfortunately,
over time, some of these original resources can go off-topic and no longer suit
the purpose for which the collection was originally created. They can go
off-topic due to web site redesigns, changes in domain ownership, financial
issues, hacking, technical problems, or because their content has moved on from
the original topic. Even though they are off-topic, the archiving system will
still capture them, thus it becomes imperative to anyone performing research on
these collections to identify these off-topic mementos. Hence, we present the
Off-Topic Memento Toolkit, which allows users to detect off-topic mementos
within web archive collections. The mementos identified by this toolkit can
then be separately removed from a collection or merely excluded from downstream
analysis. The following similarity measures are available: byte count, word
count, cosine similarity, Jaccard distance, S{\o}rensen-Dice distance, Simhash
using raw text content, Simhash using term frequency, and Latent Semantic
Indexing via the gensim library. We document the implementation of each of
these similarity measures. We possess a gold standard dataset generated by
manual analysis, which contains both off-topic and on-topic mementos. Using
this gold standard dataset, we establish a default threshold corresponding to
the best F1 score for each measure. We also provide an overview of potential
future directions that the toolkit may take.
Compositional GAN: Learning Image-Conditional Binary Composition
Authors: Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, Trevor Darrell
Date: 2018-07-19 00:00:00

Generative Adversarial Networks (GANs) can produce images of remarkable
complexity and realism but are generally structured to sample from a single
latent source ignoring the explicit spatial interaction between multiple
entities that could be present in a scene. Capturing such complex interactions
between different objects in the world, including their relative scaling,
spatial layout, occlusion, or viewpoint transformation is a challenging
problem. In this work, we propose a novel self-consistent
Composition-by-Decomposition (CoDe) network to compose a pair of objects. Given
object images from two distinct distributions, our model can generate a
realistic composite image from their joint distribution following the texture
and shape of the input objects. We evaluate our approach through qualitative
experiments and user evaluations. Our results indicate that the learned model
captures potential interactions between the two object domains, and generates
realistic composed scenes at test time.
Capsule Networks against Medical Imaging Data Challenges
Authors: Amelia Jiménez-Sánchez, Shadi Albarqouni, Diana Mateus
Date: 2018-07-19 00:00:00
Tasks: General Classification, image-classification, Image Classification, Medical Image Analysis, Medical Image Classification

A key component to the success of deep learning is the availability of
massive amounts of training data. Building and annotating large datasets for
solving medical image classification problems is today a bottleneck for many
applications. Recently, capsule networks were proposed to deal with
shortcomings of Convolutional Neural Networks (ConvNets). In this work, we
compare the behavior of capsule networks against ConvNets under typical
datasets constraints of medical image analysis, namely, small amounts of
annotated data and class-imbalance. We evaluate our experiments on MNIST,
Fashion-MNIST and medical (histological and retina images) publicly available
datasets. Our results suggest that capsule networks can be trained with less
amount of data for the same or better performance and are more robust to an
imbalanced class distribution, which makes our approach very promising for the
medical imaging community.
Transfer Learning for Action Unit Recognition
Authors: Yen Khye Lim, Zukang Liao, Stavros Petridis, Maja Pantic
Date: 2018-07-19 00:00:00
Tasks: Action Unit Detection, Facial Action Unit Detection, Facial Expression Recognition, Facial Expression Recognition (FER), Transfer Learning

This paper presents a classifier ensemble for Facial Expression Recognition
(FER) based on models derived from transfer learning. The main experimentation
work is conducted for facial action unit detection using feature extraction and
fine-tuning convolutional neural networks (CNNs). Several classifiers for
extracted CNN codes such as Linear Discriminant Analysis (LDA), Support Vector
Machines (SVMs) and Long Short-Term Memory (LSTM) are compared and evaluated.
Multi-model ensembles are also used to further improve the performance. We have
found that VGG-Face and ResNet are the relatively optimal pre-trained models
for action unit recognition using feature extraction and the ensemble of
VGG-Net variants and ResNet achieves the best result.
Partial recovery bounds for clustering with the relaxed $K$means
Authors: Christophe Giraud, Nicolas Verzelen
Date: 2018-07-19 00:00:00
Tasks: Clustering, Stochastic Block Model

We investigate the clustering performances of the relaxed $K$means in the
setting of sub-Gaussian Mixture Model (sGMM) and Stochastic Block Model (SBM).
After identifying the appropriate signal-to-noise ratio (SNR), we prove that
the misclassification error decay exponentially fast with respect to this SNR.
These partial recovery bounds for the relaxed $K$means improve upon results
currently known in the sGMM setting. In the SBM setting, applying the relaxed
$K$means SDP allows to handle general connection probabilities whereas other
SDPs investigated in the literature are restricted to the assortative case
(where within group probabilities are larger than between group probabilities).
Again, this partial recovery bound complements the state-of-the-art results.
All together, these results put forward the versatility of the relaxed
$K$means.
Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks
Authors: João Loula, Marco Baroni, Brenden M. Lake
Date: 2018-07-19 00:00:00
Proceeding: WS 2018 11

Systematic compositionality is the ability to recombine meaningful units with
regular and predictable outcomes, and it's seen as key to humans' capacity for
generalization in language. Recent work has studied systematic compositionality
in modern seq2seq models using generalization to novel navigation instructions
in a grounded environment as a probing tool, requiring models to quickly
bootstrap the meaning of new words. We extend this framework here to settings
where the model needs only to recombine well-trained functional words (such as
"around" and "right") in novel contexts. Our findings confirm and strengthen
the earlier ones: seq2seq models can be impressively good at generalizing to
novel combinations of previously-seen input, but only when they receive
extensive training on the specific pattern to be generalized (e.g.,
generalizing from many examples of "X around right" to "jump around right"),
while failing when generalization requires novel application of compositional
rules (e.g., inferring the meaning of "around right" from those of "right" and
"around").
Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Authors: David Berthelot, Colin Raffel, Aurko Roy, Ian Goodfellow
Date: 2018-07-19 00:00:00
Proceeding: ICLR 2019 5

Autoencoders provide a powerful framework for learning compressed
representations by encoding all of the information needed to reconstruct a data
point in a latent code. In some cases, autoencoders can "interpolate": By
decoding the convex combination of the latent codes for two datapoints, the
autoencoder can produce an output which semantically mixes characteristics from
the datapoints. In this paper, we propose a regularization procedure which
encourages interpolated outputs to appear more realistic by fooling a critic
network which has been trained to recover the mixing coefficient from
interpolated data. We then develop a simple benchmark task where we can
quantitatively measure the extent to which various autoencoders can interpolate
and show that our regularizer dramatically improves interpolation in this
setting. We also demonstrate empirically that our regularizer produces latent
codes which are more effective on downstream tasks, suggesting a possible link
between interpolation abilities and learning useful representations.
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods
Authors: Laurence Aitchison
Date: 2018-07-19 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Bayesian Inference

We formulate the problem of neural network optimization as Bayesian filtering, where the observations are the backpropagated gradients. While neural network optimization has previously been studied using natural gradient methods which are closely related to Bayesian inference, they were unable to recover standard optimizers such as Adam and RMSprop with a root-mean-square gradient normalizer, instead getting a mean-square normalizer. To recover the root-mean-square normalizer, we find it necessary to account for the temporal dynamics of all the other parameters as they are geing optimized. The resulting optimizer, AdaBayes, adaptively transitions between SGD-like and Adam-like behaviour, automatically recovers AdamW, a state of the art variant of Adam with decoupled weight decay, and has generalisation performance competitive with SGD.
Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs
Authors: Yu-Xing Tang, Xiaosong Wang, Adam P. Harrison, Le Lu, Jing Xiao, Ronald M. Summers
Date: 2018-07-19 00:00:00
Tasks: Classification, General Classification, Weakly Supervised Classification

In this work, we exploit the task of joint classification and weakly
supervised localization of thoracic diseases from chest radiographs, with only
image-level disease labels coupled with disease severity-level (DSL)
information of a subset. A convolutional neural network (CNN) based
attention-guided curriculum learning (AGCL) framework is presented, which
leverages the severity-level attributes mined from radiology reports. Images in
order of difficulty (grouped by different severity-levels) are fed to CNN to
boost the learning gradually. In addition, highly confident samples (measured
by classification probabilities) and their corresponding class-conditional
heatmaps (generated by the CNN) are extracted and further fed into the AGCL
framework to guide the learning of more distinctive convolutional features in
the next iteration. A two-path network architecture is designed to regress the
heatmaps from selected seed samples in addition to the original classification
task. The joint learning scheme can improve the classification and localization
performance along with more seed samples for the next iteration. We demonstrate
the effectiveness of this iterative refinement framework via extensive
experimental evaluations on the publicly available ChestXray14 dataset. AGCL
achieves over 5.7\% (averaged over 14 diseases) increase in classification AUC
and 7%/11% increases in Recall/Precision for the localization task compared to
the state of the art.
Self-Organizing Maps as a Storage and Transfer Mechanism in Reinforcement Learning
Authors: Thommen George Karimpanal, Roland Bouffanais
Date: 2018-07-19 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The idea of reusing information from previously learned tasks (source tasks)
for the learning of new tasks (target tasks) has the potential to significantly
improve the sample efficiency reinforcement learning agents. In this work, we
describe an approach to concisely store and represent learned task knowledge,
and reuse it by allowing it to guide the exploration of an agent while it
learns new tasks. In order to do so, we use a measure of similarity that is
defined directly in the space of parameterized representations of the value
functions. This similarity measure is also used as a basis for a variant of the
growing self-organizing map algorithm, which is simultaneously used to enable
the storage of previously acquired task knowledge in an adaptive and scalable
manner.We empirically validate our approach in a simulated navigation
environment and discuss possible extensions to this approach along with
potential applications where it could be particularly useful.
Bio-Measurements Estimation and Support in Knee Recovery through Machine Learning
Authors: João Bernardino, Luís Filipe Teixeira, Hugo Sereno Ferreira
Date: 2018-07-19 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation, Transfer Learning

Knee injuries are frequent, varied and often require the patient to undergo
intensive rehabilitation for several months. Treatment protocols usually
contemplate some recurrent measurements in order to assess progress, such as
goniometry. The need for specific equipment or the complexity and duration of
these tasks cause them to often be neglected. A novel deep learning based
solution is presented, supported by the generation of a synthetic image
dataset. A 3D human-body model was used for this purpose, simulating a
recovering patient. For each image, the coordinates of three key points were
registered: the centers of the thigh, the knee and the lower leg. These values
are sufficient to estimate the flexion angle. Convolutional neural networks
were then trained for predicting these six coordinates. Transfer learning was
used with the VGG16 and InceptionV3 models pre-trained on the ImageNet dataset,
being an additional custom model trained from scratch. All models were tested
with different combinations of data augmentation techniques applied on the
training sets. InceptionV3 achieved the best overall results, producing
considerably good predictions even on real unedited pictures.
Statistical Model Compression for Small-Footprint Natural Language Understanding
Authors: Grant P. Strimel, Kanthashree Mysore Sathyendra, Stanislav Peshterliev
Date: 2018-07-19 00:00:00
Tasks: Model Compression, Natural Language Understanding, Quantization

In this paper we investigate statistical model compression applied to natural
language understanding (NLU) models. Small-footprint NLU models are important
for enabling offline systems on hardware restricted devices, and for decreasing
on-demand model loading latency in cloud-based systems. To compress NLU models,
we present two main techniques, parameter quantization and perfect feature
hashing. These techniques are complementary to existing model pruning
strategies such as L1 regularization. We performed experiments on a large scale
NLU system. The results show that our approach achieves 14-fold reduction in
memory usage compared to the original models with minimal predictive
performance impact.
Using Deep Neural Networks to Translate Multi-lingual Threat Intelligence
Authors: Priyanka Ranade, Sudip Mittal, Anupam Joshi, Karuna Joshi
Date: 2018-07-19 00:00:00
Tasks: Translation

The multilingual nature of the Internet increases complications in the
cybersecurity community's ongoing efforts to strategically mine threat
intelligence from OSINT data on the web. OSINT sources such as social media,
blogs, and dark web vulnerability markets exist in diverse languages and hinder
security analysts, who are unable to draw conclusions from intelligence in
languages they don't understand. Although third party translation engines are
growing stronger, they are unsuited for private security environments. First,
sensitive intelligence is not a permitted input to third party engines due to
privacy and confidentiality policies. In addition, third party engines produce
generalized translations that tend to lack exclusive cybersecurity terminology.
In this paper, we address these issues and describe our system that enables
threat intelligence understanding across unfamiliar languages. We create a
neural network based system that takes in cybersecurity data in a different
language and outputs the respective English translation. The English
translation can then be understood by an analyst, and can also serve as input
to an AI based cyber-defense system that can take mitigative action. As a proof
of concept, we have created a pipeline which takes Russian threats and
generates its corresponding English, RDF, and vectorized representations. Our
network optimizes translations on specifically, cybersecurity data.
Hybrid Scene Compression for Visual Localization
Authors: Federico Camposeco, Andrea Cohen, Marc Pollefeys, Torsten Sattler
Date: 2018-07-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Quantization, Self-Driving Cars, Visual Localization

Localizing an image wrt. a 3D scene model represents a core task for many
computer vision applications. An increasing number of real-world applications
of visual localization on mobile devices, e.g., Augmented Reality or autonomous
robots such as drones or self-driving cars, demand localization approaches to
minimize storage and bandwidth requirements. Compressing the 3D models used for
localization thus becomes a practical necessity. In this work, we introduce a
new hybrid compression algorithm that uses a given memory limit in a more
effective way. Rather than treating all 3D points equally, it represents a
small set of points with full appearance information and an additional, larger
set of points with compressed information. This enables our approach to obtain
a more complete scene representation without increasing the memory
requirements, leading to a superior performance compared to previous
compression schemes. As part of our contribution, we show how to handle
ambiguous matches arising from point compression during RANSAC. Besides
outperforming previous compression techniques in terms of pose accuracy under
the same memory constraints, our compression scheme itself is also more
efficient. Furthermore, the localization rates and accuracy obtained with our
approach are comparable to state-of-the-art feature-based methods, while using
a small fraction of the memory.
A Strategy of MR Brain Tissue Images' Suggestive Annotation Based on Modified U-Net
Authors: Yang Deng, Yao Sun, Yongpei Zhu, Mingwang Zhu, Wei Han, Kehong Yuan
Date: 2018-07-19 00:00:00
Tasks: Segmentation

Accurate segmentation of MR brain tissue is a crucial step for
diagnosis,surgical planning, and treatment of brain abnormalities. However,it
is a time-consuming task to be performed by medical experts. So, automatic and
reliable segmentation methods are required. How to choose appropriate training
dataset from limited labeled dataset rather than the whole also has great
significance in saving training time. In addition, medical data labeled is too
rare and expensive to obtain extensively, so choosing appropriate unlabeled
dataset instead of all the datasets to annotate, which can attain at least same
performance, is also very meaningful. To solve the problem above, we design an
automatic segmentation method based on U-shaped deep convolutional network and
obtain excellent result with average DSC metric of 0.8610, 0.9131, 0.9003 for
Cerebrospinal Fluid (CSF), Gray Matter (GM) and White Matter (WM) respectively
on the well-known IBSR18 dataset. We use bootstrapping algorithm for selecting
the most effective training data and get more state-of-the-art segmentation
performance by using only 50% of training data. Moreover, we propose a strategy
of MR brain tissue images' suggestive annotation for unlabeled medical data
based on the modified U-net. The proposed method performs fast and can be used
in clinical.
Improving Simple Models with Confidence Profiles
Authors: Amit Dhurandhar, Karthikeyan Shanmugam, Ronny Luss, Peder Olsen
Date: 2018-07-19 00:00:00
Proceeding: NeurIPS 2018 12

In this paper, we propose a new method called ProfWeight for transferring
information from a pre-trained deep neural network that has a high test
accuracy to a simpler interpretable model or a very shallow network of low
complexity and a priori low test accuracy. We are motivated by applications in
interpretability and model deployment in severely memory constrained
environments (like sensors). Our method uses linear probes to generate
confidence scores through flattened intermediate representations. Our transfer
method involves a theoretically justified weighting of samples during the
training of the simple model using confidence scores of these intermediate
layers. The value of our method is first demonstrated on CIFAR-10, where our
weighting method significantly improves (3-4%) networks with only a fraction of
the number of Resnet blocks of a complex Resnet model. We further demonstrate
operationally significant results on a real manufacturing problem, where we
dramatically increase the test accuracy of a CART model (the domain standard)
by roughly 13%.
FuzzerGym: A Competitive Framework for Fuzzing and Learning
Authors: William Drozd, Michael D. Wagner
Date: 2018-07-19 00:00:00
Tasks: Deep Reinforcement Learning, Fault Detection, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Fuzzing is a commonly used technique designed to test software by
automatically crafting program inputs. Currently, the most successful fuzzing
algorithms emphasize simple, low-overhead strategies with the ability to
efficiently monitor program state during execution. Through compile-time
instrumentation, these approaches have access to numerous aspects of program
state including coverage, data flow, and heterogeneous fault detection and
classification. However, existing approaches utilize blind random mutation
strategies when generating test inputs. We present a different approach that
uses this state information to optimize mutation operators using reinforcement
learning (RL). By integrating OpenAI Gym with libFuzzer we are able to
simultaneously leverage advancements in reinforcement learning as well as
fuzzing to achieve deeper coverage across several varied benchmarks. Our
technique connects the rich, efficient program monitors provided by LLVM
Santizers with a deep neural net to learn mutation selection strategies
directly from the input data. The cross-language, asynchronous architecture we
developed enables us to apply any OpenAI Gym compatible deep reinforcement
learning algorithm to any fuzzing problem with minimal slowdown.