pwc-archive/papers-with-abstracts huggingface dataset:
Data-driven Modelling of Dynamical Systems Using Tree Adjoining Grammar and Genetic Programming
Authors: Dhruv Khandelwal, Maarten Schoukens, Roland Tóth
Date: 2019-04-05 00:00:00

State-of-the-art methods for data-driven modelling of non-linear dynamical
systems typically involve interactions with an expert user. In order to
partially automate the process of modelling physical systems from data, many
EA-based approaches have been proposed for model-structure selection, with
special focus on non-linear systems. Recently, an approach for data-driven
modelling of non-linear dynamical systems using Genetic Programming (GP) was
proposed. The novelty of the method was the modelling of noise and the use of
Tree Adjoining Grammar to shape the search-space explored by GP. In this paper,
we report results achieved by the proposed method on three case studies. Each
of the case studies considered here is based on real physical systems. The case
studies pose a variety of challenges. In particular, these challenges range
over varying amounts of prior knowledge of the true system, amount of data
available, the complexity of the dynamics of the system, and the nature of
non-linearities in the system. Based on the results achieved for the case
studies, we critically analyse the performance of the proposed method.
Unsupervised Image Matching and Object Discovery as Optimization
Authors: Huy V. Vo, Francis Bach, Minsu Cho, Kai Han, Yann Lecun, Patrick Perez, Jean Ponce
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object, Object Discovery, Single-object colocalization, Single-object discovery

Learning with complete or partial supervision is powerful but relies on
ever-growing human annotation efforts. As a way to mitigate this serious
problem, as well as to serve specific applications, unsupervised learning has
emerged as an important field of research. In computer vision, unsupervised
learning comes in various guises. We focus here on the unsupervised discovery
and matching of object categories among images in a collection, following the
work of Cho et al. 2015. We show that the original approach can be reformulated
and solved as a proper optimization problem. Experiments on several benchmarks
establish the merit of our approach.
Spatial Shortcut Network for Human Pose Estimation
Authors: Te Qi, Bayram Bayramli, Usman Ali, Qinchuan Zhang, Hongtao Lu
Date: 2019-04-05 00:00:00
Tasks: Pose Estimation

Like many computer vision problems, human pose estimation is a challenging
problem in that recognizing a body part requires not only information from
local area but also from areas with large spatial distance. In order to
spatially pass information, large convolutional kernels and deep layers have
been normally used, introducing high computation cost and large parameter
space. Luckily for pose estimation, human body is geometrically structured in
images, enabling modeling of spatial dependency. In this paper, we propose a
spatial shortcut network for pose estimation task, where information is easier
to flow spatially. We evaluate our model with detailed analyses and present its
outstanding performance with smaller structure.
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
Authors: Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jähnichen, Moin Nabi
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Continual Learning

Models trained in the context of continual learning (CL) should be able to learn from a stream of data over an undefined period of time. The main challenges herein are: 1) maintaining old knowledge while simultaneously benefiting from it when learning new tasks, and 2) guaranteeing model scalability with a growing amount of data to learn from. In order to tackle these challenges, we introduce Dynamic Generative Memory (DGM) - a synaptic plasticity driven framework for continual learning. DGM relies on conditional generative adversarial networks with learnable connection plasticity realized with neural masking. Specifically, we evaluate two variants of neural masking: applied to (i) layer activations and (ii) to connection weights directly. Furthermore, we propose a dynamic network expansion mechanism that ensures sufficient model capacity to accommodate for continually incoming tasks. The amount of added capacity is determined dynamically from the learned binary mask. We evaluate DGM in the continual class-incremental setup on visual classification tasks.
Estimation of Monge Matrices
Authors: Jan-Christian Hütter, Cheng Mao, Philippe Rigollet, Elina Robeva
Date: 2019-04-05 00:00:00

Monge matrices and their permuted versions known as pre-Monge matrices
naturally appear in many domains across science and engineering. While the rich
structural properties of such matrices have long been leveraged for algorithmic
purposes, little is known about their impact on statistical estimation. In this
work, we propose to view this structure as a shape constraint and study the
problem of estimating a Monge matrix subject to additive random noise. More
specifically, we establish the minimax rates of estimation of Monge and
pre-Monge matrices. In the case of pre-Monge matrices, the minimax-optimal
least-squares estimator is not efficiently computable, and we propose two
efficient estimators and establish their rates of convergence. Our theoretical
findings are supported by numerical experiments.
Deep Learning Under the Microscope: Improving the Interpretability of Medical Imaging Neural Networks
Authors: Magdalini Paschali, Muhammad Ferjad Naeem, Walter Simson, Katja Steiger, Martin Mollenhauer, Nassir Navab
Date: 2019-04-05 00:00:00
Tasks: Decision Making, General Classification, Multiple Instance Learning

In this paper, we propose a novel interpretation method tailored to
histological Whole Slide Image (WSI) processing. A Deep Neural Network (DNN),
inspired by Bag-of-Features models is equipped with a Multiple Instance
Learning (MIL) branch and trained with weak supervision for WSI classification.
MIL avoids label ambiguity and enhances our model's expressive power without
guiding its attention. We utilize a fine-grained logit heatmap of the models
activations to interpret its decision-making process. The proposed method is
quantitatively and qualitatively evaluated on two challenging histology
datasets, outperforming a variety of baselines. In addition, two expert
pathologists were consulted regarding the interpretability provided by our
method and acknowledged its potential for integration into several clinical
applications.
Leaf segmentation through the classification of edges
Authors: Jonathan Bell, Hannah M. Dee
Date: 2019-04-05 00:00:00
Tasks: Classification, Edge Classification, General Classification, Segmentation

We present an approach to leaf level segmentation of images of Arabidopsis
thaliana plants based upon detected edges. We introduce a novel approach to
edge classification, which forms an important part of a method to both count
the leaves and establish the leaf area of a growing plant from images obtained
in a high-throughput phenotyping system. Our technique uses a relatively
shallow convolutional neural network to classify image edges as background,
plant edge, leaf-on-leaf edge or internal leaf noise. The edges themselves were
found using the Canny edge detector and the classified edges can be used with
simple image processing techniques to generate a region-based segmentation in
which the leaves are distinct. This approach is strong at distinguishing
occluding pairs of leaves where one leaf is largely hidden, a situation which
has proved troublesome for plant image analysis systems in the past. In
addition, we introduce the publicly available plant image dataset that was used
for this work.
Outlier Detection for Improved Data Quality and Diversity in Dialog Systems
Authors: Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Diversity, intent-classification, Intent Classification, Intent Classification and Slot Filling, Outlier Detection, Sentence, Sentence Embeddings, slot-filling, Slot Filling

In a corpus of data, outliers are either errors: mistakes in the data that
are counterproductive, or are unique: informative samples that improve model
robustness. Identifying outliers can lead to better datasets by (1) removing
noise in datasets and (2) guiding collection of additional data to fill gaps.
However, the problem of detecting both outlier types has received relatively
little attention in NLP, particularly for dialog systems. We introduce a simple
and effective technique for detecting both erroneous and unique samples in a
corpus of short texts using neural sentence embeddings combined with
distance-based outlier detection. We also present a novel data collection
pipeline built atop our detection technique to automatically and iteratively
mine unique data samples while discarding erroneous samples. Experiments show
that our outlier detection technique is effective at finding errors while our
data collection pipeline yields highly diverse corpora that in turn produce
more robust intent classification and slot-filling models.
Fast Weakly Supervised Action Segmentation Using Mutual Consistency
Authors: Yaser Souri, Mohsen Fayyaz, Luca Minciullo, Gianpiero Francesca, Juergen Gall
Date: 2019-04-05 00:00:00
Tasks: Action Segmentation, Segmentation, Weakly Supervised Action Segmentation (Transcript)

Action segmentation is the task of predicting the actions for each frame of a video. As obtaining the full annotation of videos for action segmentation is expensive, weakly supervised approaches that can learn only from transcripts are appealing. In this paper, we propose a novel end-to-end approach for weakly supervised action segmentation based on a two-branch neural network. The two branches of our network predict two redundant but different representations for action segmentation and we propose a novel mutual consistency (MuCon) loss that enforces the consistency of the two redundant representations. Using the MuCon loss together with a loss for transcript prediction, our proposed approach achieves the accuracy of state-of-the-art approaches while being $14$ times faster to train and $20$ times faster during inference. The MuCon loss proves beneficial even in the fully supervised setting.
PoMo: Generating Entity-Specific Post-Modifiers in Context
Authors: Jun Seok Kang, Robert L. Logan IV, Zewei Chu, Yang Chen, Dheeru Dua, Kevin Gimpel, Sameer Singh, Niranjan Balasubramanian
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Articles, Sentence

We introduce entity post-modifier generation as an instance of a
collaborative writing task. Given a sentence about a target entity, the task is
to automatically generate a post-modifier phrase that provides contextually
relevant information about the entity. For example, for the sentence, "Barack
Obama, _______, supported the #MeToo movement.", the phrase "a father of two
girls" is a contextually relevant post-modifier. To this end, we build PoMo, a
post-modifier dataset created automatically from news articles reflecting a
journalistic need for incorporating entity information that is relevant to a
particular news event. PoMo consists of more than 231K sentences with
post-modifiers and associated facts extracted from Wikidata for around 57K
unique entities. We use crowdsourcing to show that modeling contextual
relevance is necessary for accurate post-modifier generation. We adapt a number
of existing generation approaches as baselines for this dataset. Our results
show there is large room for improvement in terms of both identifying relevant
facts to include (knowing which claims are relevant gives a >20% improvement in
BLEU score), and generating appropriate post-modifier text for the context
(providing relevant claims is not sufficient for accurate generation). We
conduct an error analysis that suggests promising directions for future
research.
3DQ: Compact Quantized Neural Networks for Volumetric Whole Brain Segmentation
Authors: Magdalini Paschali, Stefano Gasperini, Abhijit Guha Roy, Michael Y. -S. Fang, Nassir Navab
Date: 2019-04-05 00:00:00
Tasks: Brain Segmentation, Model Compression, Quantization

Model architectures have been dramatically increasing in size, improving performance at the cost of resource requirements. In this paper we propose 3DQ, a ternary quantization method, applied for the first time to 3D Fully Convolutional Neural Networks (F-CNNs), enabling 16x model compression while maintaining performance on par with full precision models. We extensively evaluate 3DQ on two datasets for the challenging task of whole brain segmentation. Additionally, we showcase our method's ability to generalize on two common 3D architectures, namely 3D U-Net and V-Net. Outperforming a variety of baselines, the proposed method is capable of compressing large 3D models to a few MBytes, alleviating the storage needs in space critical applications.
Convolutional Self-Attention Networks
Authors: Baosong Yang, Long-Yue Wang, Derek Wong, Lidia S. Chao, Zhaopeng Tu
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Translation

Self-attention networks (SANs) have drawn increasing interest due to their
high parallelization in computation and flexibility in modeling dependencies.
SANs can be further enhanced with multi-head attention by allowing the model to
attend to information from different representation subspaces. In this work, we
propose novel convolutional self-attention networks, which offer SANs the
abilities to 1) strengthen dependencies among neighboring elements, and 2)
model the interaction between features extracted by multiple attention heads.
Experimental results of machine translation on different language pairs and
model settings show that our approach outperforms both the strong Transformer
baseline and other existing models on enhancing the locality of SANs. Comparing
with prior studies, the proposed model is parameter free in terms of
introducing no more parameters.
Information Aggregation for Multi-Head Attention with Routing-by-Agreement
Authors: Jian Li, Baosong Yang, Zi-Yi Dou, Xing Wang, Michael R. Lyu, Zhaopeng Tu
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Translation

Multi-head attention is appealing for its ability to jointly extract
different types of information from multiple representation subspaces.
Concerning the information aggregation, a common practice is to use a
concatenation followed by a linear transformation, which may not fully exploit
the expressiveness of multi-head attention. In this work, we propose to improve
the information aggregation for multi-head attention with a more powerful
routing-by-agreement algorithm. Specifically, the routing algorithm iteratively
updates the proportion of how much a part (i.e. the distinct information
learned from a specific subspace) should be assigned to a whole (i.e. the final
output representation), based on the agreement between parts and wholes.
Experimental results on linguistic probing tasks and machine translation tasks
prove the superiority of the advanced information aggregation over the standard
linear transformation.
Modeling Recurrence for Transformer
Authors: Jie Hao, Xing Wang, Baosong Yang, Long-Yue Wang, Jinfeng Zhang, Zhaopeng Tu
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Translation

Recently, the Transformer model that is based solely on attention mechanisms,
has advanced the state-of-the-art on various machine translation tasks.
However, recent studies reveal that the lack of recurrence hinders its further
improvement of translation capacity. In response to this problem, we propose to
directly model recurrence for Transformer with an additional recurrence
encoder. In addition to the standard recurrent neural network, we introduce a
novel attentive recurrent network to leverage the strengths of both attention
and recurrent networks. Experimental results on the widely-used WMT14
English-German and WMT17 Chinese-English translation tasks demonstrate the
effectiveness of the proposed approach. Our studies also reveal that the
proposed model benefits from a short-cut that bridges the source and target
sequences with a single recurrent layer, which outperforms its deep
counterpart.
Eigenvalue distribution of nonlinear models of random matrices
Authors: Lucas Benigni, Sandrine Péché
Date: 2019-04-05 00:00:00

This paper is concerned with the asymptotic empirical eigenvalue distribution of a non linear random matrix ensemble. More precisely we consider $M= \frac{1}{m} YY^*$ with $Y=f(WX)$ where $W$ and $X$ are random rectangular matrices with i.i.d. centered entries. The function $f$ is applied pointwise and can be seen as an activation function in (random) neural networks. We compute the asymptotic empirical distribution of this ensemble in the case where $W$ and $X$ have sub-Gaussian tails and $f$ is real analytic. This extends a previous result where the case of Gaussian matrices $W$ and $X$ is considered. We also investigate the same questions in the multi-layer case, regarding neural network applications.
Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
Authors: Chun-Hung Chao, Yen-Chi Cheng, Hsien-Tzu Cheng, Chi-Wen Huang, Tsung-Ying Ho, Chen-Kan Tseng, Le Lu, Min Sun
Date: 2019-04-05 00:00:00
Tasks: Graph Neural Network

Tomography medical imaging is essential in the clinical workflow of modern
cancer radiotherapy. Radiation oncologists identify cancerous tissues, applying
delineation on treatment regions throughout all image slices. This kind of task
is often formulated as a volumetric segmentation task by means of 3D
convolutional networks with considerable computational cost. Instead, inspired
by the treating methodology of considering meaningful information across
slices, we used Gated Graph Neural Network to frame this problem more
efficiently. More specifically, we propose convolutional recurrent Gated Graph
Propagator (GGP) to propagate high-level information through image slices, with
learnable adjacency weighted matrix. Furthermore, as physicians often
investigate a few specific slices to refine their decision, we model this
slice-wise interaction procedure to further improve our segmentation result.
This can be set by editing any slice effortlessly as updating predictions of
other slices using GGP. To evaluate our method, we collect an Esophageal Cancer
Radiotherapy Target Treatment Contouring dataset of 81 patients which includes
tomography images with radiotherapy target. On this dataset, our convolutional
graph network produces state-of-the-art results and outperforms the baselines.
With the addition of interactive setting, performance is improved even further.
Our method has the potential to be easily applied to diverse kinds of medical
tasks with volumetric images. Incorporating both the ability to make a feasible
prediction and to consider the human interactive input, the proposed method is
suitable for clinical scenarios.
CLEARumor at SemEval-2019 Task 7: ConvoLving ELMo Against Rumors
Authors: Ipek Baris, Lukas Schmelzeisen, Steffen Staab
Date: 2019-04-05 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Rumour Detection

This paper describes our submission to SemEval-2019 Task 7: RumourEval:
Determining Rumor Veracity and Support for Rumors. We participated in both
subtasks. The goal of subtask A is to classify the type of interaction between
a rumorous social media post and a reply post as support, query, deny, or
comment. The goal of subtask B is to predict the veracity of a given rumor. For
subtask A, we implement a CNN-based neural architecture using ELMo embeddings
of post text combined with auxiliary features and achieve a F1-score of 44.6%.
For subtask B, we employ a MLP neural network leveraging our estimates for
subtask A and achieve a F1-score of 30.1% (second place in the competition). We
provide results and analysis of our system performance and present ablation
experiments.
Controlling Neural Networks via Energy Dissipation
Authors: Michael Moeller, Thomas Möllenhoff, Daniel Cremers
Date: 2019-04-05 00:00:00
Proceeding: ICCV 2019 10
Tasks: Computed Tomography (CT), CT Reconstruction, Deblurring, Image Reconstruction, Image Super-Resolution, Super-Resolution

The last decade has shown a tremendous success in solving various computer vision problems with the help of deep learning techniques. Lately, many works have demonstrated that learning-based approaches with suitable network architectures even exhibit superior performance for the solution of (ill-posed) image reconstruction problems such as deblurring, super-resolution, or medical image reconstruction. The drawback of purely learning-based methods, however, is that they cannot provide provable guarantees for the trained network to follow a given data formation process during inference. In this work we propose energy dissipating networks that iteratively compute a descent direction with respect to a given cost function or energy at the currently estimated reconstruction. Therefore, an adaptive step size rule such as a line-search, along with a suitable number of iterations can guarantee the reconstruction to follow a given data formation model encoded in the energy to arbitrary precision, and hence control the model's behavior even during test time. We prove that under standard assumptions, descent using the direction predicted by the network converges (linearly) to the global minimum of the energy. We illustrate the effectiveness of the proposed approach in experiments on single image super resolution and computed tomography (CT) reconstruction, and further illustrate extensions to convex feasibility problems.
SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks
Authors: René Schuster, Oliver Wasenmüller, Christian Unger, Didier Stricker
Date: 2019-04-05 00:00:00
Tasks: Optical Flow Estimation, Stereo Matching, Stereo Matching Hand

Dense pixel matching is important for many computer vision tasks such as
disparity and flow estimation. We present a robust, unified descriptor network
that considers a large context region with high spatial variance. Our network
has a very large receptive field and avoids striding layers to maintain spatial
resolution. These properties are achieved by creating a novel neural network
layer that consists of multiple, parallel, stacked dilated convolutions (SDC).
Several of these layers are combined to form our SDC descriptor network. In our
experiments, we show that our SDC features outperform state-of-the-art feature
descriptors in terms of accuracy and robustness. In addition, we demonstrate
the superior performance of SDC in state-of-the-art stereo matching, optical
flow and scene flow algorithms on several famous public benchmarks.
Comparative Analysis of Automatic Skin Lesion Segmentation with Two Different Implementations
Authors: Md. Kamrul Hasan, Basel Alyafi, Fakrul Islam Tushar
Date: 2019-04-05 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation, Vocal Bursts Valence Prediction

Lesion segmentation from the surrounding skin is the first task for
developing automatic Computer-Aided Diagnosis of skin cancer. Variant features
of lesion like uneven distribution of color, irregular shape, border and
texture make this task challenging. The contribution of this paper is to
present and compare two different approaches to skin lesion segmentation. The
first approach uses watershed, while the second approach uses mean-shift.
Pre-processing steps were performed in both approaches for removing hair and
dark borders of microscopic images. The Evaluation of the proposed approaches
was performed using Jaccard Index (Intersection over Union or IoU). An
additional contribution of this paper is to present pipelines for performing
pre-processing and segmentation applying existing segmentation and
morphological algorithms which led to promising results. On average, the first
approach showed better performance than the second one with average Jaccard
Index over 200 ISIC-2017 challenge images are 89.16% and 76.94% respectively.