pwc-archive/papers-with-abstracts huggingface dataset:
Deep Semantic Role Labeling with Self-Attention
Authors: Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, Xiaodong Shi
Date: 2017-12-05 00:00:00
Tasks: GPU, Natural Language Understanding, Semantic Role Labeling

Semantic Role Labeling (SRL) is believed to be a crucial step towards natural
language understanding and has been widely studied. Recent years, end-to-end
SRL with recurrent neural networks (RNN) has gained increasing attention.
However, it remains a major challenge for RNNs to handle structural information
and long range dependencies. In this paper, we present a simple and effective
architecture for SRL which aims to address these problems. Our model is based
on self-attention which can directly capture the relationships between two
tokens regardless of their distance. Our single model achieves F$_1=83.4$ on
the CoNLL-2005 shared task dataset and F$_1=82.7$ on the CoNLL-2012 shared task
dataset, which outperforms the previous state-of-the-art results by $1.8$ and
$1.0$ F$_1$ score respectively. Besides, our model is computationally
efficient, and the parsing speed is 50K tokens per second on a single Titan X
GPU.
Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification
Authors: Jakub M. Tomczak, Maximilian Ilse, Max Welling
Date: 2017-12-01 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

The computer-aided analysis of medical scans is a longstanding goal in the
medical imaging field. Currently, deep learning has became a dominant
methodology for supporting pathologists and radiologist. Deep learning
algorithms have been successfully applied to digital pathology and radiology,
nevertheless, there are still practical issues that prevent these tools to be
widely used in practice. The main obstacles are low number of available cases
and large size of images (a.k.a. the small n, large p problem in machine
learning), and a very limited access to annotation at a pixel level that can
lead to severe overfitting and large computational requirements. We propose to
handle these issues by introducing a framework that processes a medical image
as a collection of small patches using a single, shared neural network. The
final diagnosis is provided by combining scores of individual patches using a
permutation-invariant operator (combination). In machine learning community
such approach is called a multi-instance learning (MIL).
EmTaggeR: A Word Embedding Based Novel Method for Hashtag Recommendation on Twitter
Authors: Kuntal Dey, Ritvik Shrivastava, Saroj Kaushik, L. Venkata Subramaniam
Date: 2017-12-05 00:00:00
Tasks: TAG

The hashtag recommendation problem addresses recommending (suggesting) one or
more hashtags to explicitly tag a post made on a given social network platform,
based upon the content and context of the post. In this work, we propose a
novel methodology for hashtag recommendation for microblog posts, specifically
Twitter. The methodology, EmTaggeR, is built upon a training-testing framework
that builds on the top of the concept of word embedding. The training phase
comprises of learning word vectors associated with each hashtag, and deriving a
word embedding for each hashtag. We provide two training procedures, one in
which each hashtag is trained with a separate word embedding model applicable
in the context of that hashtag, and another in which each hashtag obtains its
embedding from a global context. The testing phase constitutes computing the
average word embedding of the test post, and finding the similarity of this
embedding with the known embeddings of the hashtags. The tweets that contain
the most-similar hashtag are extracted, and all the hashtags that appear in
these tweets are ranked in terms of embedding similarity scores. The top-K
hashtags that appear in this ranked list, are recommended for the given test
post. Our system produces F1 score of 50.83%, improving over the LDA baseline
by around 6.53 times, outperforming the best-performing system known in the
literature that provides a lift of 6.42 times. EmTaggeR is a fast, scalable and
lightweight system, which makes it practical to deploy in real-life
applications.
Parameter Reference Loss for Unsupervised Domain Adaptation
Authors: Jiren Jin, Richard G. Calland, Takeru Miyato, Brian K. Vogel, Hideki Nakayama
Date: 2017-11-20 00:00:00
Tasks: Domain Adaptation, Model Selection, Unsupervised Domain Adaptation

The success of deep learning in computer vision is mainly attributed to an
abundance of data. However, collecting large-scale data is not always possible,
especially for the supervised labels. Unsupervised domain adaptation (UDA) aims
to utilize labeled data from a source domain to learn a model that generalizes
to a target domain of unlabeled data. A large amount of existing work uses
Siamese network-based models, where two streams of neural networks process the
source and the target domain data respectively. Nevertheless, most of these
approaches focus on minimizing the domain discrepancy, overlooking the
importance of preserving the discriminative ability for target domain features.
Another important problem in UDA research is how to evaluate the methods
properly. Common evaluation procedures require target domain labels for
hyper-parameter tuning and model selection, contradicting the definition of the
UDA task. Hence we propose a more reasonable evaluation principle that avoids
this contradiction by simply adopting the latest snapshot of a model for
evaluation. This adds an extra requirement for UDA methods besides the main
performance criteria: the stability during training. We design a novel method
that connects the target domain stream to the source domain stream with a
Parameter Reference Loss (PRL) to solve these problems simultaneously.
Experiments on various datasets show that the proposed PRL not only improves
the performance on the target domain, but also stabilizes the training
procedure. As a result, PRL based models do not need the contradictory model
selection, and thus are more suitable for practical applications.
Manifold-valued Image Generation with Wasserstein Generative Adversarial Nets
Authors: Zhiwu Huang, Jiqing Wu, Luc van Gool
Date: 2017-12-05 00:00:00
Tasks: Image Generation

Generative modeling over natural images is one of the most fundamental
machine learning problems. However, few modern generative models, including
Wasserstein Generative Adversarial Nets (WGANs), are studied on manifold-valued
images that are frequently encountered in real-world applications. To fill the
gap, this paper first formulates the problem of generating manifold-valued
images and exploits three typical instances: hue-saturation-value (HSV) color
image generation, chromaticity-brightness (CB) color image generation, and
diffusion-tensor (DT) image generation. For the proposed generative modeling
problem, we then introduce a theorem of optimal transport to derive a new
Wasserstein distance of data distributions on complete manifolds, enabling us
to achieve a tractable objective under the WGAN framework. In addition, we
recommend three benchmark datasets that are CIFAR-10 HSV/CB color images,
ImageNet HSV/CB color images, UCL DT image datasets. On the three datasets, we
experimentally demonstrate the proposed manifold-aware WGAN model can generate
more plausible manifold-valued images than its competitors.
Clustering with feature selection using alternating minimization, Application to computational biology
Authors: Cyprien Gilet, Marie Deprez, Jean-Baptiste Caillau, Michel Barlaud
Date: 2017-11-08 00:00:00
Tasks: Clustering, feature selection

This paper deals with unsupervised clustering with feature selection. The problem is to estimate both labels and a sparse projection matrix of weights. To address this combinatorial non-convex problem maintaining a strict control on the sparsity of the matrix of weights, we propose an alternating minimization of the Frobenius norm criterion. We provide a new efficient algorithm named K-sparse which alternates k-means with projection-gradient minimization. The projection-gradient step is a method of splitting type, with exact projection on the $\ell^1$ ball to promote sparsity. The convergence of the gradient-projection step is addressed, and a preliminary analysis of the alternating minimization is made. The Frobenius norm criterion converges as the number of iterates in Algorithm K-sparse goes to infinity. Experiments on Single Cell RNA sequencing datasets show that our method significantly improves the results of PCA k-means, spectral clustering, SIMLR, and Sparcl methods, and achieves a relevant selection of genes. The complexity of K-sparse is linear in the number of samples (cells), so that the method scales up to large datasets.
O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
Authors: Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, Xin Tong
Date: 2017-12-05 00:00:00
Tasks: 3D Object Classification, GPU, Retrieval, Semantic Segmentation

We present O-CNN, an Octree-based Convolutional Neural Network (CNN) for 3D
shape analysis. Built upon the octree representation of 3D shapes, our method
takes the average normal vectors of a 3D model sampled in the finest leaf
octants as input and performs 3D CNN operations on the octants occupied by the
3D shape surface. We design a novel octree data structure to efficiently store
the octant information and CNN features into the graphics memory and execute
the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN
structures and works for 3D shapes in different representations. By restraining
the computations on the octants occupied by 3D surfaces, the memory and
computational costs of the O-CNN grow quadratically as the depth of the octree
increases, which makes the 3D CNN feasible for high-resolution 3D models. We
compare the performance of the O-CNN with other existing 3D CNN solutions and
demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks,
including object classification, shape retrieval, and shape segmentation.
Human Perception of Performance
Authors: Luca Pappalardo, Paolo Cintia, Dino Pedreschi, Fosca Giannotti, Albert-Laszlo Barabasi
Date: 2017-12-05 00:00:00

Humans are routinely asked to evaluate the performance of other individuals,
separating success from failure and affecting outcomes from science to
education and sports. Yet, in many contexts, the metrics driving the human
evaluation process remain unclear. Here we analyse a massive dataset capturing
players' evaluations by human judges to explore human perception of performance
in soccer, the world's most popular sport. We use machine learning to design an
artificial judge which accurately reproduces human evaluation, allowing us to
demonstrate how human observers are biased towards diverse contextual features.
By investigating the structure of the artificial judge, we uncover the aspects
of the players' behavior which attract the attention of human judges,
demonstrating that human evaluation is based on a noticeability heuristic where
only feature values far from the norm are considered to rate an individual's
performance.
R2-D2: ColoR-inspired Convolutional NeuRal Network (CNN)-based AndroiD Malware Detections
Authors: TonTon Hsien-De Huang, Hung-Yu Kao
Date: 2017-05-12 00:00:00
Tasks: Android Malware Detection, Feature Engineering, Malware Detection

The influence of Deep Learning on image identification and natural language
processing has attracted enormous attention globally. The convolution neural
network that can learn without prior extraction of features fits well in
response to the rapid iteration of Android malware. The traditional solution
for detecting Android malware requires continuous learning through
pre-extracted features to maintain high performance of identifying the malware.
In order to reduce the manpower of feature engineering prior to the condition
of not to extract pre-selected features, we have developed a coloR-inspired
convolutional neuRal networks (CNN)-based AndroiD malware Detection (R2-D2)
system. The system can convert the bytecode of classes.dex from Android archive
file to rgb color code and store it as a color image with fixed size. The color
image is input to the convolutional neural network for automatic feature
extraction and training. The data was collected from Jan. 2017 to Aug 2017.
During the period of time, we have collected approximately 2 million of benign
and malicious Android apps for our experiments with the help from our research
partner Leopard Mobile Inc. Our experiment results demonstrate that the
proposed system has accurate security analysis on contracts. Furthermore, we
keep our research results and experiment materials on http://R2D2.TWMAN.ORG.
An Online Algorithm for Nonparametric Correlations
Authors: Wei Xiao
Date: 2017-12-05 00:00:00
Tasks: Data Summarization, Sequential Correlation Estimation

Nonparametric correlations such as Spearman's rank correlation and Kendall's
tau correlation are widely applied in scientific and engineering fields. This
paper investigates the problem of computing nonparametric correlations on the
fly for streaming data. Standard batch algorithms are generally too slow to
handle real-world big data applications. They also require too much memory
because all the data need to be stored in the memory before processing. This
paper proposes a novel online algorithm for computing nonparametric
correlations. The algorithm has O(1) time complexity and O(1) memory cost and
is quite suitable for edge devices, where only limited memory and processing
power are available. You can seek a balance between speed and accuracy by
changing the number of cutpoints specified in the algorithm. The online
algorithm can compute the nonparametric correlations 10 to 1,000 times faster
than the corresponding batch algorithm, and it can compute them based either on
all past observations or on fixed-size sliding windows.
Cooperative Training of Deep Aggregation Networks for RGB-D Action Recognition
Authors: Pichao Wang, Wanqing Li, Jun Wan, Philip Ogunbona, Xinwang Liu
Date: 2017-12-05 00:00:00
Tasks: Action Recognition, Temporal Action Localization, Triplet

A novel deep neural network training paradigm that exploits the conjoint
information in multiple heterogeneous sources is proposed. Specifically, in a
RGB-D based action recognition task, it cooperatively trains a single
convolutional neural network (named c-ConvNet) on both RGB visual features and
depth features, and deeply aggregates the two kinds of features for action
recognition. Differently from the conventional ConvNet that learns the deep
separable features for homogeneous modality-based classification with only one
softmax loss function, the c-ConvNet enhances the discriminative power of the
deeply learned features and weakens the undesired modality discrepancy by
jointly optimizing a ranking loss and a softmax loss for both homogeneous and
heterogeneous modalities. The ranking loss consists of intra-modality and
cross-modality triplet losses, and it reduces both the intra-modality and
cross-modality feature variations. Furthermore, the correlations between RGB
and depth data are embedded in the c-ConvNet, and can be retrieved by either of
the modalities and contribute to the recognition in the case even only one of
the modalities is available. The proposed method was extensively evaluated on
two large RGB-D action recognition datasets, ChaLearn LAP IsoGD and NTU RGB+D
datasets, and one small dataset, SYSU 3D HOI, and achieved state-of-the-art
results.
Fully Automatic Segmentation of Lumbar Vertebrae from CT Images using Cascaded 3D Fully Convolutional Networks
Authors: Rens Janssens, Guodong Zeng, Guoyan Zheng
Date: 2017-12-05 00:00:00
Tasks: Segmentation

We present a method to address the challenging problem of segmentation of
lumbar vertebrae from CT images acquired with varying fields of view. Our
method is based on cascaded 3D Fully Convolutional Networks (FCNs) consisting
of a localization FCN and a segmentation FCN. More specifically, in the first
step we train a regression 3D FCN (we call it "LocalizationNet") to find the
bounding box of the lumbar region. After that, a 3D U-net like FCN (we call it
"SegmentationNet") is then developed, which after training, can perform a
pixel-wise multi-class segmentation to map a cropped lumber region volumetric
data to its volume-wise labels. Evaluated on publicly available datasets, our
method achieved an average Dice coefficient of 95.77 $\pm$ 0.81% and an average
symmetric surface distance of 0.37 $\pm$ 0.06 mm.
PHOCNet: A Deep Convolutional Neural Network for Word Spotting in Handwritten Documents
Authors: Sebastian Sudholt, Gernot A. Fink
Date: 2016-04-01 00:00:00
Tasks: General Classification, Word Spotting In Handwritten Documents

In recent years, deep convolutional neural networks have achieved state of
the art performance in various computer vision task such as classification,
detection or segmentation. Due to their outstanding performance, CNNs are more
and more used in the field of document image analysis as well. In this work, we
present a CNN architecture that is trained with the recently proposed PHOC
representation. We show empirically that our CNN architecture is able to
outperform state of the art results for various word spotting benchmarks while
exhibiting short training and test times.
ssEMnet: Serial-section Electron Microscopy Image Registration using a Spatial Transformer Network with Learned Features
Authors: Inwan Yoo, David G. C. Hildebrand, Willie F. Tobin, Wei-Chung Allen Lee, Won-Ki Jeong
Date: 2017-07-25 00:00:00
Tasks: Image Registration

The alignment of serial-section electron microscopy (ssEM) images is critical
for efforts in neuroscience that seek to reconstruct neuronal circuits.
However, each ssEM plane contains densely packed structures that vary from one
section to the next, which makes matching features across images a challenge.
Advances in deep learning has resulted in unprecedented performance in similar
computer vision problems, but to our knowledge, they have not been successfully
applied to ssEM image co-registration. In this paper, we introduce a novel deep
network model that combines a spatial transformer for image deformation and a
convolutional autoencoder for unsupervised feature learning for robust ssEM
image alignment. This results in improved accuracy and robustness while
requiring substantially less user intervention than conventional methods. We
evaluate our method by comparing registration quality across several datasets.
Automatic chemical design using a data-driven continuous representation of molecules
Authors: Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, Alán Aspuru-Guzik
Date: 2016-10-07 00:00:00
Tasks: Decoder, Efficient Exploration

We report a method to convert discrete representations of molecules to and
from a multidimensional continuous representation. This model allows us to
generate new molecules for efficient exploration and optimization through
open-ended spaces of chemical compounds. A deep neural network was trained on
hundreds of thousands of existing chemical structures to construct three
coupled functions: an encoder, a decoder and a predictor. The encoder converts
the discrete representation of a molecule into a real-valued continuous vector,
and the decoder converts these continuous vectors back to discrete molecular
representations. The predictor estimates chemical properties from the latent
continuous vector representation of the molecule. Continuous representations
allow us to automatically generate novel chemical structures by performing
simple operations in the latent space, such as decoding random vectors,
perturbing known chemical structures, or interpolating between molecules.
Continuous representations also allow the use of powerful gradient-based
optimization to efficiently guide the search for optimized functional
compounds. We demonstrate our method in the domain of drug-like molecules and
also in the set of molecules with fewer that nine heavy atoms.
Spectrally-normalized margin bounds for neural networks
Authors: Peter Bartlett, Dylan J. Foster, Matus Telgarsky
Date: 2017-06-26 00:00:00
Proceeding: NeurIPS 2017 12

This paper presents a margin-based multiclass generalization bound for neural
networks that scales with their margin-normalized "spectral complexity": their
Lipschitz constant, meaning the product of the spectral norms of the weight
matrices, times a certain correction factor. This bound is empirically
investigated for a standard AlexNet network trained with SGD on the mnist and
cifar10 datasets, with both original and random labels; the bound, the
Lipschitz constants, and the excess risks are all in direct correlation,
suggesting both that SGD selects predictors whose complexity scales with the
difficulty of the learning task, and secondly that the presented bound is
sensitive to this complexity.
Determinism in the Certification of UNSAT Proofs
Authors: Tomer Libal, Xaviera Steele
Date: 2017-12-05 00:00:00

The search for increased trustworthiness of SAT solvers is very active and
uses various methods. Some of these methods obtain a proof from the provers
then check it, normally by replicating the search based on the proof's
information. Because the certification process involves another nontrivial
proof search, the trust we can place in it is decreased. Some attempts to amend
this use certifiers which have been verified by proofs assistants such as
Isabelle/HOL and Coq. Our approach is different because it is based on an
extremely simplified certifier. This certifier enjoys a very high level of
trust but is very inefficient. In this paper, we experiment with this approach
and conclude that by placing some restrictions on the formats, one can mostly
eliminate the need for search and in principle, can certify proofs of arbitrary
size.
A Probabilistic Quality Representation Approach to Deep Blind Image Quality Prediction
Authors: Hui Zeng, Lei Zhang, Alan C. Bovik
Date: 2017-08-28 00:00:00
Tasks: Image Quality Assessment, No-Reference Image Quality Assessment

Blind image quality assessment (BIQA) remains a very challenging problem due
to the unavailability of a reference image. Deep learning based BIQA methods
have been attracting increasing attention in recent years, yet it remains a
difficult task to train a robust deep BIQA model because of the very limited
number of training samples with human subjective scores. Most existing methods
learn a regression network to minimize the prediction error of a scalar image
quality score. However, such a scheme ignores the fact that an image will
receive divergent subjective scores from different subjects, which cannot be
adequately represented by a single scalar number. This is particularly true on
complex, real-world distorted images. Moreover, images may broadly differ in
their distributions of assigned subjective scores. Recognizing this, we propose
a new representation of perceptual image quality, called probabilistic quality
representation (PQR), to describe the image subjective score distribution,
whereby a more robust loss function can be employed to train a deep BIQA model.
The proposed PQR method is shown to not only speed up the convergence of deep
model training, but to also greatly improve the achievable level of quality
prediction accuracy relative to scalar quality score regression methods. The
source code is available at https://github.com/HuiZeng/BIQA_Toolbox.