pwc-archive/papers-with-abstracts huggingface dataset:
Enhancing Semantic Word Representations by Embedding Deeper Word Relationships
Authors: Anupiya Nugaliyadde, Kok Wai Wong, Ferdous Sohel, Hong Xie
Date: 2019-01-22 00:00:00
Tasks: Natural Language Understanding

Word representations are created using analogy context-based statistics and
lexical relations on words. Word representations are inputs for the learning
models in Natural Language Understanding (NLU) tasks. However, to understand
language, knowing only the context is not sufficient. Reading between the lines
is a key component of NLU. Embedding deeper word relationships which are not
represented in the context enhances the word representation. This paper
presents a word embedding which combines an analogy, context-based statistics
using Word2Vec, and deeper word relationships using Conceptnet, to create an
expanded word representation. In order to fine-tune the word representation,
Self-Organizing Map is used to optimize it. The proposed word representation is
compared with semantic word representations using Simlex 999. Furthermore, the
use of 3D visual representations has shown to be capable of representing the
similarity and association between words. The proposed word representation
shows a Spearman correlation score of 0.886 and provided the best results when
compared to the current state-of-the-art methods, and exceed the human
performance of 0.78.
Efficient Image Splicing Localization via Contrastive Feature Extraction
Authors: Ronald Salloum, C. -C. Jay Kuo
Date: 2019-01-22 00:00:00
Tasks: Clustering, Data Visualization

In this work, we propose a new data visualization and clustering technique
for discovering discriminative structures in high-dimensional data. This
technique, referred to as cPCA++, utilizes the fact that the interesting
features of a "target" dataset may be obscured by high variance components
during traditional PCA. By analyzing what is referred to as a "background"
dataset (i.e., one that exhibits the high variance principal components but not
the interesting structures), our technique is capable of efficiently
highlighting the structure that is unique to the "target" dataset. Similar to
another recently proposed algorithm called "contrastive PCA" (cPCA), the
proposed cPCA++ method identifies important dataset specific patterns that are
not detected by traditional PCA in a wide variety of settings. However, the
proposed cPCA++ method is significantly more efficient than cPCA, because it
does not require the parameter sweep in the latter approach. We applied the
cPCA++ method to the problem of image splicing localization. In this
application, we utilize authentic edges as the background dataset and the
spliced edges as the target dataset. The proposed method is significantly more
efficient than state-of-the-art methods, as the former does not require
iterative updates of filter weights via stochastic gradient descent and
backpropagation, nor the training of a classifier. Furthermore, the cPCA++
method is shown to provide performance scores comparable to the
state-of-the-art Multi-task Fully Convolutional Network (MFCN).
Energy Confused Adversarial Metric Learning for Zero-Shot Image Retrieval and Clustering
Authors: Binghui Chen, Weihong Deng
Date: 2019-01-22 00:00:00
Tasks: Clustering, Image Retrieval, Metric Learning, Retrieval, Zero-shot Image Retrieval

Deep metric learning has been widely applied in many computer vision tasks,
and recently, it is more attractive in \emph{zero-shot image retrieval and
clustering}(ZSRC) where a good embedding is requested such that the unseen
classes can be distinguished well. Most existing works deem this 'good'
embedding just to be the discriminative one and thus race to devise powerful
metric objectives or hard-sample mining strategies for leaning discriminative
embedding. However, in this paper, we first emphasize that the generalization
ability is a core ingredient of this 'good' embedding as well and largely
affects the metric performance in zero-shot settings as a matter of fact. Then,
we propose the Energy Confused Adversarial Metric Learning(ECAML) framework to
explicitly optimize a robust metric. It is mainly achieved by introducing an
interesting Energy Confusion regularization term, which daringly breaks away
from the traditional metric learning idea of discriminative objective devising,
and seeks to 'confuse' the learned model so as to encourage its generalization
ability by reducing overfitting on the seen classes. We train this confusion
term together with the conventional metric objective in an adversarial manner.
Although it seems weird to 'confuse' the network, we show that our ECAML indeed
serves as an efficient regularization technique for metric learning and is
applicable to various conventional metric methods. This paper empirically and
experimentally demonstrates the importance of learning embedding with good
generalization, achieving state-of-the-art performances on the popular CUB,
CARS, Stanford Online Products and In-Shop datasets for ZSRC tasks.
\textcolor[rgb]{1, 0, 0}{Code available at http://www.bhchen.cn/}.
Generation High resolution 3D model from natural language by Generative Adversarial Network
Authors: Kentaro Fukamizu, Masaaki Kondo, Ryuichi Sakamoto
Date: 2019-01-22 00:00:00
Tasks: Generative Adversarial Network

We present a method of generating high resolution 3D shapes from natural
language descriptions. To achieve this goal, we propose two steps that
generating low resolution shapes which roughly reflect texts and generating
high resolution shapes which reflect the detail of texts. In a previous paper,
the authors have shown a method of generating low resolution shapes. We improve
it to generate 3D shapes more faithful to natural language and test the
effectiveness of the method. To generate high resolution 3D shapes, we use the
framework of Conditional Wasserstein GAN. We propose two roles of Critic
separately, which calculate the Wasserstein distance between two probability
distribution, so that we achieve generating high quality shapes or acceleration
of learning speed of model. To evaluate our approach, we performed quantitive
evaluation with several numerical metrics for Critic models. Our method is
first to realize the generation of high quality model by propagating text
embedding information to high resolution task when generating 3D model.
Sensitivity Analysis of Deep Neural Networks
Authors: Hai Shu, Hongtu Zhu
Date: 2019-01-22 00:00:00
Tasks: Sensitivity

Deep neural networks (DNNs) have achieved superior performance in various
prediction tasks, but can be very vulnerable to adversarial examples or
perturbations. Therefore, it is crucial to measure the sensitivity of DNNs to
various forms of perturbations in real applications. We introduce a novel
perturbation manifold and its associated influence measure to quantify the
effects of various perturbations on DNN classifiers. Such perturbations include
various external and internal perturbations to input samples and network
parameters. The proposed measure is motivated by information geometry and
provides desirable invariance properties. We demonstrate that our influence
measure is useful for four model building tasks: detecting potential
'outliers', analyzing the sensitivity of model architectures, comparing network
sensitivity between training and test sets, and locating vulnerable areas.
Experiments show reasonably good performance of the proposed measure for the
popular DNN models ResNet50 and DenseNet121 on CIFAR10 and MNIST datasets.
Universal Rules for Fooling Deep Neural Networks based Text Classification
Authors: Di Li, Danilo Vasconcellos Vargas, Sakurai Kouichi
Date: 2019-01-22 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

Recently, deep learning based natural language processing techniques are
being extensively used to deal with spam mail, censorship evaluation in social
networks, among others. However, there is only a couple of works evaluating the
vulnerabilities of such deep neural networks. Here, we go beyond attacks to
investigate, for the first time, universal rules, i.e., rules that are sample
agnostic and therefore could turn any text sample in an adversarial one. In
fact, the universal rules do not use any information from the method itself (no
information from the method, gradient information or training dataset
information is used), making them black-box universal attacks. In other words,
the universal rules are sample and method agnostic. By proposing a
coevolutionary optimization algorithm we show that it is possible to create
universal rules that can automatically craft imperceptible adversarial samples
(only less than five perturbations which are close to misspelling are inserted
in the text sample). A comparison with a random search algorithm further
justifies the strength of the method. Thus, universal rules for fooling
networks are here shown to exist. Hopefully, the results from this work will
impact the development of yet more sample and model agnostic attacks as well as
their defenses, culminating in perhaps a new age for artificial intelligence.
An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation
Authors: Xiang Kong, Bohan Li, Graham Neubig, Eduard Hovy, Yiming Yang
Date: 2019-01-22 00:00:00
Tasks: Dialogue Generation, Response Generation, Vocal Bursts Intensity Prediction

In this work, we propose a method for neural dialogue response generation
that allows not only generating semantically reasonable responses according to
the dialogue history, but also explicitly controlling the sentiment of the
response via sentiment labels. Our proposed model is based on the paradigm of
conditional adversarial learning; the training of a sentiment-controlled
dialogue generator is assisted by an adversarial discriminator which assesses
the fluency and feasibility of the response generating from the dialogue
history and a given sentiment label. Because of the flexibility of our
framework, the generator could be a standard sequence-to-sequence (SEQ2SEQ)
model or a more complicated one such as a conditional variational
autoencoder-based SEQ2SEQ model. Experimental results using automatic and human
evaluation both demonstrate that our proposed framework is able to generate
both semantically reasonable and sentiment-controlled dialogue responses.
Linearized Multi-Sampling for Differentiable Image Transformation
Authors: Wei Jiang, Weiwei Sun, Andrea Tagliasacchi, Eduard Trulls, Kwang Moo Yi
Date: 2019-01-22 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Registration

We propose a novel image sampling method for differentiable image transformation in deep neural networks. The sampling schemes currently used in deep learning, such as Spatial Transformer Networks, rely on bilinear interpolation, which performs poorly under severe scale changes, and more importantly, results in poor gradient propagation. This is due to their strict reliance on direct neighbors. Instead, we propose to generate random auxiliary samples in the vicinity of each pixel in the sampled image, and create a linear approximation with their intensity values. We then use this approximation as a differentiable formula for the transformed image. We demonstrate that our approach produces more representative gradients with a wider basin of convergence for image alignment, which leads to considerable performance improvements when training networks for classification tasks. This is not only true under large downsampling, but also when there are no scale changes. We compare our approach with multi-scale sampling and show that we outperform it. We then demonstrate that our improvements to the sampler are compatible with other tangential improvements to Spatial Transformer Networks and that it further improves their performance.
Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits
Authors: Xialiang Dou, Tengyuan Liang
Date: 2019-01-21 00:00:00

Consider the problem: given the data pair $(\mathbf{x}, \mathbf{y})$ drawn from a population with $f_*(x) = \mathbf{E}[\mathbf{y} | \mathbf{x} = x]$, specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does $f_t$, the function computed by the neural network at time $t$, relate to $f_*$, in terms of approximation and representation? What are the provable benefits of the adaptive representation by neural networks compared to the pre-specified fixed basis representation in the classical nonparametric literature? We answer the above questions via a dynamic reproducing kernel Hilbert space (RKHS) approach indexed by the training process of neural networks. Firstly, we show that when reaching any local stationarity, gradient flow learns an adaptive RKHS representation and performs the global least-squares projection onto the adaptive RKHS, simultaneously. Secondly, we prove that as the RKHS is data-adaptive and task-specific, the residual for $f_*$ lies in a subspace that is potentially much smaller than the orthogonal complement of the RKHS. The result formalizes the representation and approximation benefits of neural networks. Lastly, we show that the neural network function computed by gradient flow converges to the kernel ridgeless regression with an adaptive kernel, in the limit of vanishing regularization. The adaptive kernel viewpoint provides new angles of studying the approximation, representation, generalization, and optimization advantages of neural networks.
Robust Angular Local Descriptor Learning
Authors: Yanwu Xu, Mingming Gong, Tongliang Liu, Kayhan Batmanghelich, Chaohui Wang
Date: 2019-01-21 00:00:00
Tasks: Metric Learning, Triplet

In recent years, the learned local descriptors have outperformed handcrafted
ones by a large margin, due to the powerful deep convolutional neural network
architectures such as L2-Net [1] and triplet based metric learning [2].
However, there are two problems in the current methods, which hinders the
overall performance. Firstly, the widely-used margin loss is sensitive to
incorrect correspondences, which are prevalent in the existing local descriptor
learning datasets. Second, the L2 distance ignores the fact that the feature
vectors have been normalized to unit norm. To tackle these two problems and
further boost the performance, we propose a robust angular loss which 1) uses
cosine similarity instead of L2 distance to compare descriptors and 2) relies
on a robust loss function that gives smaller penalty to triplets with negative
relative similarity. The resulting descriptor shows robustness on different
datasets, reaching the state-of-the-art result on Brown dataset , as well as
demonstrating excellent generalization ability on the Hpatches dataset and a
Wide Baseline Stereo dataset.
Development of verification system of socio-demographic data of virtual community member
Authors: S. S. Fedushko
Date: 2019-01-21 00:00:00

The important task of developing verification system of data of virtual
community member on the basis of computer-linguistic analysis of the content of
a large sample of Ukrainian virtual communities is solved. The subject of
research is methods and tools for verification of web-members socio-demographic
characteristics based on computer-linguistic analysis of their communicative
interaction results. The aim of paper is to verifying web-user personal data on
the basis of computer-linguistic analysis of web-members information tracks.
The structure of verification software for web-user profile is designed for a
practical implementation of assigned tasks. The method of personal data
verification of web-members by analyzing information track of virtual community
member is conducted. For the first time the method for checking the
authenticity of web members personal data, which helped to design of
verification tool for socio-demographic characteristics of web-member is
developed. The verification system of data of web-members, which forms the
verified socio-demographic profiles of web-members, is developed as a result of
conducted experiments. Also the user interface of the developed verification
system web-members data is presented. Effectiveness and efficiency of use of
the developed methods and means for solving tasks in web-communities
administration is proved by their approbation. The number of false results of
verification system is 18%.
On Compression of Unsupervised Neural Nets by Pruning Weak Connections
Authors: Zhiwen Zuo, Lei Zhao, Liwen Zuo, Feng Jiang, Wei Xing, Dongming Lu
Date: 2019-01-21 00:00:00
Tasks: Density Estimation

Unsupervised neural nets such as Restricted Boltzmann Machines(RBMs) and Deep Belif Networks(DBNs), are powerful in automatic feature extraction,unsupervised weight initialization and density estimation. In this paper,we demonstrate that the parameters of these neural nets can be dramatically reduced without affecting their performance. We describe a method to reduce the parameters required by RBM which is the basic building block for deep architectures. Further we propose an unsupervised sparse deep architectures selection algorithm to form sparse deep neural networks.Experimental results show that there is virtually no loss in either generative or discriminative performance.
Prior Information Guided Regularized Deep Learning for Cell Nucleus Detection
Authors: Mohammad Tofighi, Tiantong Guo, Jairam K. P. Vanamala, Vishal Monga
Date: 2019-01-21 00:00:00
Tasks: Deep Learning

Cell nuclei detection is a challenging research topic because of limitations
in cellular image quality and diversity of nuclear morphology, i.e. varying
nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been
a topic of enduring interest with promising recent success shown by deep
learning methods. These methods train Convolutional Neural Networks (CNNs) with
a training set of input images and known, labeled nuclei locations. Many such
methods are supplemented by spatial or morphological processing. Using a set of
canonical cell nuclei shapes, prepared with the help of a domain expert, we
develop a new approach that we call Shape Priors with Convolutional Neural
Networks (SP-CNN). We further extend the network to introduce a shape prior
(SP) layer and then allowing it to become trainable (i.e. optimizable). We call
this network tunable SP-CNN (TSP-CNN). In summary, we present new network
structures that can incorporate 'expected behavior' of nucleus shapes via two
components: learnable layers that perform the nucleus detection and a fixed
processing part that guides the learning with prior information. Analytically,
we formulate two new regularization terms that are targeted at: 1) learning the
shapes, 2) reducing false positives while simultaneously encouraging detection
inside the cell nucleus boundary. Experimental results on two challenging
datasets reveal that the proposed SP-CNN and TSP-CNN can outperform
state-of-the-art alternatives.
MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
Authors: Alistair E. W. Johnson, Tom J. Pollard, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G. Mark, Seth J. Berkowitz, Steven Horng
Date: 2019-01-21 00:00:00

Chest radiography is an extremely powerful imaging modality, allowing for a detailed inspection of a patient's thorax, but requiring specialized training for proper interpretation. With the advent of high performance general purpose computer vision algorithms, the accurate automated analysis of chest radiographs is becoming increasingly of interest to researchers. However, a key challenge in the development of these techniques is the lack of sufficient data. Here we describe MIMIC-CXR-JPG v2.0.0, a large dataset of 377,110 chest x-rays associated with 227,827 imaging studies sourced from the Beth Israel Deaconess Medical Center between 2011 - 2016. Images are provided with 14 labels derived from two natural language processing tools applied to the corresponding free-text radiology reports. MIMIC-CXR-JPG is derived entirely from the MIMIC-CXR database, and aims to provide a convenient processed version of MIMIC-CXR, as well as to provide a standard reference for data splits and image labels. All images have been de-identified to protect patient privacy. The dataset is made freely available to facilitate and encourage a wide range of research in medical computer vision.
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Authors: Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, Andrew Y. Ng
Date: 2019-01-21 00:00:00
Tasks: Lung Disease Classification

Large, labeled datasets have driven deep learning methods to achieve
expert-level performance on a variety of medical imaging tasks. We present
CheXpert, a large dataset that contains 224,316 chest radiographs of 65,240
patients. We design a labeler to automatically detect the presence of 14
observations in radiology reports, capturing uncertainties inherent in
radiograph interpretation. We investigate different approaches to using the
uncertainty labels for training convolutional neural networks that output the
probability of these observations given the available frontal and lateral
radiographs. On a validation set of 200 chest radiographic studies which were
manually annotated by 3 board-certified radiologists, we find that different
uncertainty approaches are useful for different pathologies. We then evaluate
our best model on a test set composed of 500 chest radiographic studies
annotated by a consensus of 5 board-certified radiologists, and compare the
performance of our model to that of 3 additional radiologists in the detection
of 5 selected pathologies. On Cardiomegaly, Edema, and Pleural Effusion, the
model ROC and PR curves lie above all 3 radiologist operating points. We
release the dataset to the public as a standard benchmark to evaluate
performance of chest radiograph interpretation models.
  The dataset is freely available at
https://stanfordmlgroup.github.io/competitions/chexpert .
Automatic Synthesis of Totally Self-Checking Circuits
Authors: Michael Garvie, Phil Husbands
Date: 2019-01-21 00:00:00

Totally self-checking (TSC) circuits are synthesised with a grid of computers
running a distributed population based stochastic optimisation algorithm. The
presented method is the first to automatically synthesise TSC circuits from
arbitrary logic as all previous methods fail to guarantee the checker is
self-testing (ST) for circuits with limited output codespaces. The circuits
synthesised by the presented method have significantly lower overhead than the
previously reported best for every one of a set of 11 frequently used
benchmarks. Average overhead across the entire set is 23% of duplication and
comparison overhead, compared with an average of 69% for the previous best
reported values across the set. The methodology presented represents a
breakthrough in concurrent error detection (CED). The highly efficient, novel
designs produced are tailored to each circuit's function, rather than being
constrained by a particular modular CED design methodology. Results are
synthesised using two-input gates and are TSC with respect to all gate input
and output stuck-at faults. The method can be used to add CED with or without
modifications to the original logic, and can be generalised to any
implementation technology and fault model. An example circuit is analysed and
rigorously proven to be TSC.