pwc-archive/papers-with-abstracts huggingface dataset:
Generalized Canonical Polyadic Tensor Decomposition
Authors: David Hong, Tamara G. Kolda, Jed A. Duersch
Date: 2018-08-22 00:00:00
Tasks: Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in
data science, with applications including network analysis and sensor data
processing. This work develops a generalized canonical polyadic (GCP) low-rank
tensor decomposition that allows other loss functions besides squared error.
For instance, we can use logistic loss or Kullback-Leibler divergence, enabling
tensor decomposition for binary or count data. We present a variety
statistically-motivated loss functions for various scenarios. We provide a
generalized framework for computing gradients and handling missing data that
enables the use of standard optimization methods for fitting the model. We
demonstrate the flexibility of GCP on several real-world examples including
interactions in a social network, neural activity in a mouse, and monthly
rainfall measurements in India.
3D Topology Optimization using Convolutional Neural Networks
Authors: Saurabh Banga, Harsh Gehani, Sanket Bhilare, Sagar Patel, Levent Kara
Date: 2018-08-22 00:00:00
Tasks: Decoder

Topology optimization is computationally demanding that requires the assembly
and solution to a finite element problem for each material distribution
hypothesis. As a complementary alternative to the traditional physics-based
topology optimization, we explore a data-driven approach that can quickly
generate accurate solutions. To this end, we propose a deep learning approach
based on a 3D encoder-decoder Convolutional Neural Network architecture for
accelerating 3D topology optimization and to determine the optimal
computational strategy for its deployment. Analysis of iteration-wise progress
of the Solid Isotropic Material with Penalization process is used as a
guideline to study how the earlier steps of the conventional topology
optimization can be used as input for our approach to predict the final
optimized output structure directly from this input. We conduct a comparative
study between multiple strategies for training the neural network and assess
the effect of using various input combinations for the CNN to finalize the
strategy with the highest accuracy in predictions for practical deployment. For
the best performing network, we achieved about 40% reduction in overall
computation time while also attaining structural accuracies in the order of
96%.
Deep Boosted Regression for MR to CT Synthesis
Authors: Kerstin Kläser, Pawel Markiewicz, Marta Ranzini, Wenqi Li, Marc Modat, Brian F. Hutton, David Atkinson, Kris Thielemans, M. Jorge Cardoso, Sebastien Ourselin
Date: 2018-08-22 00:00:00
Tasks: Computed Tomography (CT), Image Reconstruction, regression

Attenuation correction is an essential requirement of positron emission
tomography (PET) image reconstruction to allow for accurate quantification.
However, attenuation correction is particularly challenging for PET-MRI as
neither PET nor magnetic resonance imaging (MRI) can directly image tissue
attenuation properties. MRI-based computed tomography (CT) synthesis has been
proposed as an alternative to physics based and segmentation-based approaches
that assign a population-based tissue density value in order to generate an
attenuation map. We propose a novel deep fully convolutional neural network
that generates synthetic CTs in a recursive manner by gradually reducing the
residuals of the previous network, increasing the overall accuracy and
generalisability, while keeping the number of trainable parameters within
reasonable limits. The model is trained on a database of 20 pre-acquired MRI/CT
pairs and a four-fold random bootstrapped validation with a 80:20 split is
performed. Quantitative results show that the proposed framework outperforms a
state-of-the-art atlas-based approach decreasing the Mean Absolute Error (MAE)
from 131HU to 68HU for the synthetic CTs and reducing the PET reconstruction
error from 14.3% to 7.2%.
Joint Coarse-And-Fine Reasoning for Deep Optical Flow
Authors: Victor Vaquero, German Ros, Francesc Moreno-Noguer, Antonio M. Lopez, Alberto Sanfeliu
Date: 2018-08-22 00:00:00
Tasks: Optical Flow Estimation

We propose a novel representation for dense pixel-wise estimation tasks using
CNNs that boosts accuracy and reduces training time, by explicitly exploiting
joint coarse-and-fine reasoning. The coarse reasoning is performed over a
discrete classification space to obtain a general rough solution, while the
fine details of the solution are obtained over a continuous regression space.
In our approach both components are jointly estimated, which proved to be
beneficial for improving estimation accuracy. Additionally, we propose a new
network architecture, which combines coarse and fine components by treating the
fine estimation as a refinement built on top of the coarse solution, and
therefore adding details to the general prediction. We apply our approach to
the challenging problem of optical flow estimation and empirically validate it
against state-of-the-art CNN-based solutions trained from scratch and tested on
large optical flow datasets.
Manipulating Attributes of Natural Scenes via Hallucination
Authors: Levent Karacan, Zeynep Akata, Aykut Erdem, Erkut Erdem
Date: 2018-08-22 00:00:00
Tasks: Hallucination, Style Transfer, Translation

In this study, we explore building a two-stage framework for enabling users to directly manipulate high-level attributes of a natural scene. The key to our approach is a deep generative network which can hallucinate images of a scene as if they were taken at a different season (e.g. during winter), weather condition (e.g. in a cloudy day) or time of the day (e.g. at sunset). Once the scene is hallucinated with the given attributes, the corresponding look is then transferred to the input image while preserving the semantic details intact, giving a photo-realistic manipulation result. As the proposed framework hallucinates what the scene will look like, it does not require any reference style image as commonly utilized in most of the appearance or style transfer approaches. Moreover, it allows to simultaneously manipulate a given scene according to a diverse set of transient attributes within a single model, eliminating the need of training multiple networks per each translation task. Our comprehensive set of qualitative and quantitative results demonstrate the effectiveness of our approach against the competing methods.
An Explicit Neural Network Construction for Piecewise Constant Function Approximation
Authors: Kailiang Wu, Dongbin Xiu
Date: 2018-08-22 00:00:00

We present an explicit construction for feedforward neural network (FNN),
which provides a piecewise constant approximation for multivariate functions.
The proposed FNN has two hidden layers, where the weights and thresholds are
explicitly defined and do not require numerical optimization for training.
Unlike most of the existing work on explicit FNN construction, the proposed FNN
does not rely on tensor structure in multiple dimensions. Instead, it
automatically creates Voronoi tessellation of the domain, based on the given
data of the target function, and piecewise constant approximation of the
function. This makes the construction more practical for applications. We
present both theoretical analysis and numerical examples to demonstrate its
properties.
A Note on Inexact Condition for Cubic Regularized Newton's Method
Authors: Zhe Wang, Yi Zhou, Yingbin Liang, Guanghui Lan
Date: 2018-08-22 00:00:00

This note considers the inexact cubic-regularized Newton's method (CR), which
has been shown in \cite{Cartis2011a} to achieve the same order-level
convergence rate to a secondary stationary point as the exact CR
\citep{Nesterov2006}. However, the inexactness condition in \cite{Cartis2011a}
is not implementable due to its dependence on future iterates variable. This
note fixes such an issue by proving the same convergence rate for nonconvex
optimization under an inexact adaptive condition that depends on only the
current iterate. Our proof controls the sufficient decrease of the function
value over the total iterations rather than each iteration as used in the
previous studies, which can be of independent interest in other contexts.
Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding
Authors: Deunsol Yoon, Dongbok Lee, SangKeun Lee
Date: 2018-08-22 00:00:00
Tasks: Natural Language Inference, Sentence, Sentence Embedding, Sentence-Embedding

In this paper, we propose Dynamic Self-Attention (DSA), a new self-attention
mechanism for sentence embedding. We design DSA by modifying dynamic routing in
capsule network (Sabouretal.,2017) for natural language processing. DSA attends
to informative words with a dynamic weight vector. We achieve new
state-of-the-art results among sentence encoding methods in Stanford Natural
Language Inference (SNLI) dataset with the least number of parameters, while
showing comparative results in Stanford Sentiment Treebank (SST) dataset.
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property
Authors: Yi Zhou, Zhe Wang, Yingbin Liang
Date: 2018-08-22 00:00:00
Proceeding: NeurIPS 2018 12

Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees
to produce a second-order stationary solution for solving nonconvex
optimization problems. However, existing understandings of the convergence rate
of CR are conditioned on special types of geometrical properties of the
objective function. In this paper, we explore the asymptotic convergence rate
of CR by exploiting the ubiquitous Kurdyka-Lojasiewicz (KL) property of
nonconvex objective functions. In specific, we characterize the asymptotic
convergence rate of various types of optimality measures for CR including
function value gap, variable distance gap, gradient norm and least eigenvalue
of the Hessian matrix. Our results fully characterize the diverse convergence
behaviors of these optimality measures in the full parameter regime of the KL
property. Moreover, we show that the obtained asymptotic convergence rates of
CR are order-wise faster than those of first-order gradient descent algorithms
under the KL property.
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation
Authors: Junyang Lin, Xu sun, Xuancheng Ren, Muyu Li, Qi Su
Date: 2018-08-22 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Machine Translation, NMT, Translation

Most of the Neural Machine Translation (NMT) models are based on the
sequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped
with the attention mechanism. However, the conventional attention mechanism
treats the decoding at each time step equally with the same matrix, which is
problematic since the softness of the attention for different types of words
(e.g. content words and function words) should differ. Therefore, we propose a
new model with a mechanism called Self-Adaptive Control of Temperature (SACT)
to control the softness of attention by means of an attention temperature.
Experimental results on the Chinese-English translation and English-Vietnamese
translation demonstrate that our model outperforms the baseline models, and the
analysis and the case study show that our model can attend to the most relevant
elements in the source-side contexts and generate the translation of high
quality.
Neural Named Entity Recognition from Subword Units
Authors: Abdalghani Abujabal, Judith Gaspers
Date: 2018-08-22 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Spoken Language Understanding

Named entity recognition (NER) is a vital task in spoken language understanding, which aims to identify mentions of named entities in text e.g., from transcribed speech. Existing neural models for NER rely mostly on dedicated word-level representations, which suffer from two main shortcomings. First, the vocabulary size is large, yielding large memory requirements and training time. Second, these models are not able to learn morphological or phonological representations. To remedy the above shortcomings, we adopt a neural solution based on bidirectional LSTMs and conditional random fields, where we rely on subword units, namely characters, phonemes, and bytes. For each word in an utterance, our model learns a representation from each of the subword units. We conducted experiments in a real-world large-scale setting for the use case of a voice-controlled device covering four languages with up to 5.5M utterances per language. Our experiments show that (1) with increasing training data, performance of models trained solely on subword units becomes closer to that of models with dedicated word-level embeddings (91.35 vs 93.92 F1 for English), while using a much smaller vocabulary size (332 vs 74K), (2) subword units enhance models with dedicated word-level embeddings, and (3) combining different subword units improves performance.
Multi-Branch Siamese Networks with Online Selection for Object Tracking
Authors: Zhenxi Li, Guillaume-Alexandre Bilodeau, Wassim Bouachir
Date: 2018-08-22 00:00:00
Tasks: Object, Object Tracking

In this paper, we propose a robust object tracking algorithm based on a
branch selection mechanism to choose the most efficient object representations
from multi-branch siamese networks. While most deep learning trackers use a
single CNN for target representation, the proposed Multi-Branch Siamese Tracker
(MBST) employs multiple branches of CNNs pre-trained for different tasks, and
used for various target representations in our tracking method. With our branch
selection mechanism, the appropriate CNN branch is selected depending on the
target characteristics in an online manner. By using the most adequate target
representation with respect to the tracked object, our method achieves
real-time tracking, while obtaining improved performance compared to standard
Siamese network trackers on object tracking benchmarks.
Expansional Retrofitting for Word Vector Enrichment
Authors: Hwiyeol Jo
Date: 2018-08-22 00:00:00
Tasks: General Classification, text-classification, Text Classification, Word Similarity

Retrofitting techniques, which inject external resources into word
representations, have compensated the weakness of distributed representations
in semantic and relational knowledge between words. Implicitly retrofitting
word vectors by expansional technique outperforms retrofitting in word
similarity tasks with word vector generalization. In this paper, we propose
unsupervised extrofitting: expansional retrofitting (extrofitting) without
external semantic lexicons. We also propose deep extrofitting: in-depth
stacking of extrofitting and further combinations of extrofitting with
retrofitting. When experimenting with GloVe, we show that our methods
outperform the previous methods on most of word similarity tasks while
requiring only synonyms as an external resource. Lastly, we show the effect of
word vector enrichment on text classification task, as a downstream task.
A method for automatic forensic facial reconstruction based on dense statistics of soft tissue thickness
Authors: Thomas Gietzen, Robert Brylka, Jascha Achenbach, Katja zum Hebel, Elmar Schömer, Mario Botsch, Ulrich Schwanecke, Ralf Schulze
Date: 2018-08-22 00:00:00
Tasks: Computed Tomography (CT)

In this paper, we present a method for automated estimation of a human face
given a skull remain. The proposed method is based on three statistical models.
A volumetric (tetrahedral) skull model encoding the variations of different
skulls, a surface head model encoding the head variations, and a dense
statistic of facial soft tissue thickness (FSTT). All data are automatically
derived from computed tomography (CT) head scans and optical face scans. In
order to obtain a proper dense FSTT statistic, we register a skull model to
each skull extracted from a CT scan and determine the FSTT value for each
vertex of the skull model towards the associated extracted skin surface. The
FSTT values at predefined landmarks from our statistic are well in agreement
with data from the literature.
  To recover a face from a skull remain, we first fit our skull model to the
given skull. Next, we generate spheres with radius of the respective FSTT value
obtained from our statistic at each vertex of the registered skull. Finally, we
fit a head model to the union of all spheres. The proposed automated method
enables a probabilistic face-estimation that facilitates forensic recovery even
from incomplete skull remains. The FSTT statistic allows the generation of
plausible head variants, which can be adjusted intuitively using principal
component analysis. We validate our face recovery process using an anonymized
head CT scan. The estimation generated from the given skull visually compares
well with the skin surface extracted from the CT scan itself.
Multidomain Document Layout Understanding using Few Shot Object Detection
Authors: Pranaydeep Singh, Srikrishna Varadarajan, Ankit Narayan Singh, Muktabh Mayank Srivastava
Date: 2018-08-22 00:00:00
Tasks: Few-Shot Object Detection, Object, object-detection, Object Detection, Transfer Learning

We try to address the problem of document layout understanding using a simple
algorithm which generalizes across multiple domains while training on just few
examples per domain. We approach this problem via supervised object detection
method and propose a methodology to overcome the requirement of large datasets.
We use the concept of transfer learning by pre-training our object detector on
a simple artificial (source) dataset and fine-tuning it on a tiny domain
specific (target) dataset. We show that this methodology works for multiple
domains with training samples as less as 10 documents. We demonstrate the
effect of each component of the methodology in the end result and show the
superiority of this methodology over simple object detectors.
An Attention-Gated Convolutional Neural Network for Sentence Classification
Authors: Yang Liu, Lixin Ji, Ruiyang Huang, Tuosiyu Ming, Chao GAO, Jianpeng Zhang
Date: 2018-08-22 00:00:00
Tasks: Classification, General Classification, Sentence, Sentence Classification

The classification of sentences is very challenging, since sentences contain
the limited contextual information. In this paper, we proposed an
Attention-Gated Convolutional Neural Network (AGCNN) for sentence
classification, which generates attention weights from the feature's context
windows of different sizes by using specialized convolution encoders. It makes
full use of limited contextual information to extract and enhance the influence
of important features in predicting the sentence's category. Experimental
results demonstrated that our model can achieve up to 3.1% higher accuracy than
standard CNN models, and gain competitive results over the baselines on four
out of the six tasks. Besides, we designed an activation function, namely,
Natural Logarithm rescaled Rectified Linear Unit (NLReLU). Experiments showed
that NLReLU can outperform ReLU and is comparable to other well-known
activation functions on AGCNN.
Predicting Musical Sophistication from Music Listening Behaviors: A Preliminary Study
Authors: Bruce Ferwerda, Mark Graus
Date: 2018-08-22 00:00:00

Psychological models are increasingly being used to explain online behavioral
traces. Aside from the commonly used personality traits as a general user
model, more domain dependent models are gaining attention. The use of domain
dependent psychological models allows for more fine-grained identification of
behaviors and provide a deeper understanding behind the occurrence of those
behaviors. Understanding behaviors based on psychological models can provide an
advantage over data-driven approaches. For example, relying on psychological
models allow for ways to personalize when data is scarce. In this preliminary
work we look at the relation between users' musical sophistication and their
online music listening behaviors and to what extent we can successfully predict
musical sophistication. An analysis of data from a study with 61 participants
shows that listening behaviors can successfully be used to infer users' musical
sophistication.
Learning Sentiment Memories for Sentiment Modification without Parallel Data
Authors: Yi Zhang, Jingjing Xu, Pengcheng Yang, Xu sun
Date: 2018-08-22 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Text Style Transfer

The task of sentiment modification requires reversing the sentiment of the
input and preserving the sentiment-independent content. However, aligned
sentences with the same content but different sentiments are usually
unavailable. Due to the lack of such parallel data, it is hard to extract
sentiment independent content and reverse the sentiment in an unsupervised way.
Previous work usually can not reconcile sentiment transformation and content
preservation. In this paper, motivated by the fact the non-emotional context
(e.g., "staff") provides strong cues for the occurrence of emotional words
(e.g., "friendly"), we propose a novel method that automatically extracts
appropriate sentiment information from learned sentiment memories according to
specific context. Experiments show that our method substantially improves the
content preservation degree and achieves the state-of-the-art performance.
Hybrid ASP-based Approach to Pattern Mining
Authors: Sergey Paramonov, Daria Stepanova, Pauli Miettinen
Date: 2018-08-22 00:00:00
Tasks: Graph Mining

Detecting small sets of relevant patterns from a given dataset is a central
challenge in data mining. The relevance of a pattern is based on user-provided
criteria; typically, all patterns that satisfy certain criteria are considered
relevant. Rule-based languages like Answer Set Programming (ASP) seem
well-suited for specifying such criteria in a form of constraints. Although
progress has been made, on the one hand, on solving individual mining problems
and, on the other hand, developing generic mining systems, the existing methods
either focus on scalability or on generality. In this paper we make steps
towards combining local (frequency, size, cost) and global (various condensed
representations like maximal, closed, skyline) constraints in a generic and
efficient way. We present a hybrid approach for itemset, sequence and graph
mining which exploits dedicated highly optimized mining systems to detect
frequent patterns and then filters the results using declarative ASP. To
further demonstrate the generic nature of our hybrid framework we apply it to a
problem of approximately tiling a database. Experiments on real-world datasets
show the effectiveness of the proposed method and computational gains for
itemset, sequence and graph mining, as well as approximate tiling.
  Under consideration in Theory and Practice of Logic Programming (TPLP).
Deep Association Learning for Unsupervised Video Person Re-identification
Authors: Yanbei Chen, Xiatian Zhu, Shaogang Gong
Date: 2018-08-22 00:00:00
Tasks: Person Re-Identification, Unsupervised Person Re-Identification, Video-Based Person Re-Identification

Deep learning methods have started to dominate the research progress of
video-based person re-identification (re-id). However, existing methods mostly
consider supervised learning, which requires exhaustive manual efforts for
labelling cross-view pairwise data. Therefore, they severely lack scalability
and practicality in real-world video surveillance applications. In this work,
to address the video person re-id task, we formulate a novel Deep Association
Learning (DAL) scheme, the first end-to-end deep learning method using none of
the identity labels in model initialisation and training. DAL learns a deep
re-id matching model by jointly optimising two margin-based association losses
in an end-to-end manner, which effectively constrains the association of each
frame to the best-matched intra-camera representation and cross-camera
representation. Existing standard CNNs can be readily employed within our DAL
scheme. Experiment results demonstrate that our proposed DAL significantly
outperforms current state-of-the-art unsupervised video person re-id methods on
three benchmarks: PRID 2011, iLIDS-VID and MARS.