pwc-archive/papers-with-abstracts huggingface dataset:
Probing the Need for Visual Context in Multimodal Machine Translation
Authors: Ozan Caglayan, Pranava Madhyastha, Lucia Specia, Loïc Barrault
Date: 2019-03-20 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Multimodal Machine Translation, Translation

Current work on multimodal machine translation (MMT) has suggested that the visual modality is either unnecessary or only marginally beneficial. We posit that this is a consequence of the very simple, short and repetitive sentences used in the only available dataset for the task (Multi30K), rendering the source text sufficient as context. In the general case, however, we believe that it is possible to combine visual and textual information in order to ground translations. In this paper we probe the contribution of the visual modality to state-of-the-art MMT models by conducting a systematic analysis where we partially deprive the models from source-side textual context. Our results show that under limited textual context, models are capable of leveraging the visual input to generate better translations. This contradicts the current belief that MMT models disregard the visual modality because of either the quality of the image features or the way they are integrated into the model.
Performance Measurement for Deep Bayesian Neural Network
Authors: Yikuan Li, Yajie Zhu
Date: 2019-03-20 00:00:00

Deep Bayesian neural network has aroused a great attention in recent years
since it combines the benefits of deep neural network and probability theory.
Because of this, the network can make predictions and quantify the uncertainty
of the predictions at the same time, which is important in many
life-threatening areas. However, most of the recent researches are mainly
focusing on making the Bayesian neural network easier to train, and proposing
methods to estimate the uncertainty. I notice there are very few works that
properly discuss the ways to measure the performance of the Bayesian neural
network. Although accuracy and average uncertainty are commonly used for now,
they are too general to provide any insight information about the model. In
this paper, we would like to introduce more specific criteria and propose
several metrics to measure the model performance from different perspectives,
which include model calibration measurement, data rejection ability and
uncertainty divergence for samples from the same and different distributions.
Gradient based sample selection for online continual learning
Authors: Rahaf Aljundi, Min Lin, Baptiste Goujaud, Yoshua Bengio
Date: 2019-03-20 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: class-incremental learning, Class Incremental Learning, Continual Learning, Diversity

A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i.d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.
Dilated deeply supervised networks for hippocampus segmentation in MRI
Authors: Lukas Folle, Sulaiman Vesal, Nishant Ravikumar, Andreas Maier
Date: 2019-03-20 00:00:00
Tasks: Hippocampus, Prognosis, Segmentation, Semantic Segmentation

Tissue loss in the hippocampi has been heavily correlated with the
progression of Alzheimer's Disease (AD). The shape and structure of the
hippocampus are important factors in terms of early AD diagnosis and prognosis
by clinicians. However, manual segmentation of such subcortical structures in
MR studies is a challenging and subjective task. In this paper, we investigate
variants of the well known 3D U-Net, a type of convolution neural network (CNN)
for semantic segmentation tasks. We propose an alternative form of the 3D
U-Net, which uses dilated convolutions and deep supervision to incorporate
multi-scale information into the model. The proposed method is evaluated on the
task of hippocampus head and body segmentation in an MRI dataset, provided as
part of the MICCAI 2018 segmentation decathlon challenge. The experimental
results show that our approach outperforms other conventional methods in terms
of different segmentation accuracy metrics.
A Novel Monocular Disparity Estimation Network with Domain Transformation and Ambiguity Learning
Authors: Juan Luis Gonzalez Bello, Munchurl Kim
Date: 2019-03-20 00:00:00
Tasks: Decoder, Depth Estimation, Disparity Estimation, Monocular Depth Estimation, Unsupervised Monocular Depth Estimation

Convolutional neural networks (CNN) have shown state-of-the-art results for
low-level computer vision problems such as stereo and monocular disparity
estimations, but still, have much room to further improve their performance in
terms of accuracy, numbers of parameters, etc. Recent works have uncovered the
advantages of using an unsupervised scheme to train CNN's to estimate monocular
disparity, where only the relatively-easy-to-obtain stereo images are needed
for training. We propose a novel encoder-decoder architecture that outperforms
previous unsupervised monocular depth estimation networks by (i) taking into
account ambiguities, (ii) efficient fusion between encoder and decoder features
with rectangular convolutions and (iii) domain transformations between encoder
and decoder. Our architecture outperforms the Monodepth baseline in all
metrics, even with a considerable reduction of parameters. Furthermore, our
architecture is capable of estimating a full disparity map in a single forward
pass, whereas the baseline needs two passes. We perform extensive experiments
to verify the effectiveness of our method on the KITTI dataset.
Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction
Authors: Lu-chen Liu, Haoran Li, Zhiting Hu, Haoran Shi, Zichang Wang, Jian Tang, Ming Zhang
Date: 2019-03-20 00:00:00
Tasks: ICU Admission

Clinical outcome prediction based on the Electronic Health Record (EHR) plays a crucial role in improving the quality of healthcare. Conventional deep sequential models fail to capture the rich temporal patterns encoded in the longand irregular clinical event sequences. We make the observation that clinical events at a long time scale exhibit strongtemporal patterns, while events within a short time period tend to be disordered co-occurrence. We thus propose differentiated mechanisms to model clinical events at different time scales. Our model learns hierarchical representationsof event sequences, to adaptively distinguish between short-range and long-range events, and accurately capture coretemporal dependencies. Experimental results on real clinical data show that our model greatly improves over previous state-of-the-art models, achieving AUC scores of 0.94 and 0.90 for predicting death and ICU admission respectively, Our model also successfully identifies important events for different clinical outcome prediction tasks
Face Detection in Repeated Settings
Authors: Mohammad Nayeem Teli, Bruce A. Draper, J. Ross Beveridge
Date: 2019-03-20 00:00:00
Tasks: Face Detection, Face Verification

Face detection is an important first step before face verification and
recognition. In unconstrained settings it is still an open challenge because of
the variation in pose, lighting, scale, background and location. However, for
the purposes of verification we can have a control on background and location.
Images are primarily captured in places such as the entrance to a sensitive
building, in front of a door or some location where the background does not
change. We present a correlation based face detection algorithm to detect faces
in such settings, where we control the location, and leave lighting, pose, and
scale uncontrolled. In these scenarios the results indicate that our algorithm
is easy and fast to train, outperforms Viola and Jones face detection accuracy
and is faster to test.
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Authors: Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, Ritesh Kumar
Date: 2019-03-19 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification

We present the results and the main findings of SemEval-2019 Task 6 on
Identifying and Categorizing Offensive Language in Social Media (OffensEval).
The task was based on a new dataset, the Offensive Language Identification
Dataset (OLID), which contains over 14,000 English tweets. It featured three
sub-tasks. In sub-task A, the goal was to discriminate between offensive and
non-offensive posts. In sub-task B, the focus was on the type of offensive
content in the post. Finally, in sub-task C, systems had to detect the target
of the offensive posts. OffensEval attracted a large number of participants and
it was one of the most popular tasks in SemEval-2019. In total, about 800 teams
signed up to participate in the task, and 115 of them submitted results, which
we present and analyze in this report.
Bidirectional Recurrent Models for Offensive Tweet Classification
Authors: Aleix Cambray, Norbert Podsadowski
Date: 2019-03-19 00:00:00
Tasks: Classification, General Classification

In this paper we propose four deep recurrent architectures to tackle the task
of offensive tweet detection as well as further classification into targeting
and subject of said targeting. Our architectures are based on LSTMs and GRUs,
we present a simple bidirectional LSTM as a baseline system and then further
increase the complexity of the models by adding convolutional layers and
implementing a split-process-merge architecture with LSTM and GRU as
processors. Multiple pre-processing techniques were also investigated. The
validation F1-score results from each model are presented for the three
subtasks as well as the final F1-score performance on the private competition
test set. It was found that model complexity did not necessarily yield better
results. Our best-performing model was also the simplest, a bidirectional LSTM;
closely followed by a two-branch bidirectional LSTM and GRU architecture.
On-line Search History-assisted Restart Strategy for Covariance Matrix Adaptation Evolution Strategy
Authors: Yang Lou, Shiu Yin Yuen, Guanrong Chen, Xin Zhang
Date: 2019-03-16 00:00:00

Restart strategy helps the covariance matrix adaptation evolution strategy
(CMA-ES) to increase the probability of finding the global optimum in
optimization, while a single run CMA-ES is easy to be trapped in local optima.
In this paper, the continuous non-revisiting genetic algorithm (cNrGA) is used
to help CMA-ES to achieve multiple restarts from different sub-regions of the
search space. The CMA-ES with on-line search history-assisted restart strategy
(HR-CMA-ES) is proposed. The entire on-line search history of cNrGA is stored
in a binary space partitioning (BSP) tree, which is effective for performing
local search. The frequently sampled sub-region is reflected by a deep position
in the BSP tree. When leaf nodes are located deeper than a threshold, the
corresponding sub-region is considered a region of interest (ROI). In
HR-CMA-ES, cNrGA is responsible for global exploration and suggesting ROI for
CMA-ES to perform an exploitation within or around the ROI. CMA-ES restarts
independently in each suggested ROI. The non-revisiting mechanism of cNrGA
avoids to suggest the same ROI for a second time. Experimental results on the
CEC 2013 and 2017 benchmark suites show that HR-CMA-ES performs better than
both CMA-ES and cNrGA. A positive synergy is observed by the memetic
cooperation of the two algorithms.
Combination of multiple Deep Learning architectures for Offensive Language Detection in Tweets
Authors: Nicolò Frisiani, Alexis Laignelet, Batuhan Güler
Date: 2019-03-16 00:00:00
Tasks: Language Identification

This report contains the details regarding our submission to the OffensEval
2019 (SemEval 2019 - Task 6). The competition was based on the Offensive
Language Identification Dataset. We first discuss the details of the classifier
implemented and the type of input data used and pre-processing performed. We
then move onto critically evaluating our performance. We have achieved a
macro-average F1-score of 0.76, 0.68, 0.54, respectively for Task a, Task b,
and Task c, which we believe reflects on the level of sophistication of the
models implemented. Finally, we will be discussing the difficulties encountered
and possible improvements for the future.
Russian Language Datasets in the Digitial Humanities Domain and Their Evaluation with Word Embeddings
Authors: Gerhard Wohlgenannt, Artemii Babushkin, Denis Romashov, Igor Ukrainets, Anton Maskaykin, Ilya Shutov
Date: 2019-03-04 00:00:00
Tasks: Language Modeling, Language Modelling, Word Embeddings

In this paper, we present Russian language datasets in the digital humanities
domain for the evaluation of word embedding techniques or similar language
modeling and feature learning algorithms. The datasets are split into two task
types, word intrusion and word analogy, and contain 31362 task units in total.
The characteristics of the tasks and datasets are that they build upon small,
domain-specific corpora, and that the datasets contain a high number of named
entities. The datasets were created manually for two fantasy novel book series
("A Song of Ice and Fire" and "Harry Potter"). We provide baseline evaluations
with popular word embedding models trained on the book corpora for the given
tasks, both for the Russian and English language versions of the datasets.
Finally, we compare and analyze the results and discuss specifics of Russian
language with regards to the problem setting.
CRAFT: A multifunction online platform for speech prosody visualisation
Authors: Dafydd Gibbon
Date: 2019-03-18 00:00:00
Tasks: Edge Detection, Rhythm

There are many research tools which are also used for teaching the acoustic
phonetics of speech rhythm and speech melody. But they were not
purpose-designed for teaching-learning situations, and some have a steep
learning curve. CRAFT (Creation and Recovery of Amplitude and Frequency Tracks)
is custom-designed as a novel flexible online tool for visualisation and
critical comparison of functions and transforms, with implementations of the
Reaper, RAPT, PyRapt, YAAPT, YIN and PySWIPE F0 estimators, three Praat
configurations, and two purpose-built estimators, PyAMDF, S0FT. Visualisations
of amplitude and frequency envelope spectra, spectral edge detection of rhythm
zones, and a parametrised spectrogram are included. A selection of audio clips
from tone and intonation languages is provided for demonstration purposes. The
main advantages of online tools are consistency (users have the same version
and the same data selection), interoperability over different platforms, and
ease of maintenance. The code is available on GitHub.
A Markov-Switching Model Approach to Heart Sound Segmentation and Classification
Authors: Fuad Noman, Sh-Hussain Salleh, Chee-Ming Ting, S. Balqis Samdin, Hernando Ombao, Hadri Hussain
Date: 2018-09-10 00:00:00
Tasks: General Classification

Objective: This paper considers challenges in developing algorithms for
accurate segmentation and classification of heart sound (HS) signals. Methods:
We propose an approach based on Markov switching autoregressive model (MSAR) to
segmenting the HS into four fundamental components each with distinct
second-order structure. The identified boundaries are then utilized for
automated classification of pathological HS using the continuous density hidden
Markov model (CD-HMM). The MSAR formulated in a state-space form is able to
capture simultaneously both the continuous hidden dynamics in HS, and the
regime switching in the dynamics using a discrete Markov chain. This overcomes
the limitation of HMM which uses a single-layer of discrete states. We
introduce three schemes for model estimation: (1.) switching Kalman filter
(SKF); (2.) refined SKF; (3.) fusion of SKF and the duration-dependent Viterbi
algorithm (SKF-Viterbi). Results: The proposed methods are evaluated on
Physionet/CinC Challenge 2016 database. The SKF-Viterbi significantly
outperforms SKF by improvement of segmentation accuracy from 71% to 84.2%. The
use of CD-HMM as a classifier and Mel-frequency cepstral coefficients (MFCCs)
as features can characterize not only the normal and abnormal morphologies of
HS signals but also morphologies considered as unclassifiable (denoted as
X-Factor). It gives classification rates with best gross F1 score of 90.19
(without X-Factor) and 82.7 (with X-Factor) for abnormal beats. Conclusion: The
proposed MSAR approach for automatic localization and detection of pathological
HS shows a noticeable performance on large HS dataset. Significance: It has
potential applications in heart monitoring systems to assist cardiologists for
pre-screening of heart pathologies.
Profile-Based Privacy for Locally Private Computations
Authors: Joseph Geumlek, Kamalika Chaudhuri
Date: 2019-01-21 00:00:00
Tasks: Privacy Preserving

Differential privacy has emerged as a gold standard in privacy-preserving data analysis. A popular variant is local differential privacy, where the data holder is the trusted curator. A major barrier, however, towards a wider adoption of this model is that it offers a poor privacy-utility tradeoff. In this work, we address this problem by introducing a new variant of local privacy called profile-based privacy. The central idea is that the problem setting comes with a graph G of data generating distributions, whose edges encode sensitive pairs of distributions that should be made indistinguishable. This provides higher utility because unlike local differential privacy, we no longer need to make every pair of private values in the domain indistinguishable, and instead only protect the identity of the underlying distribution. We establish privacy properties of the profile-based privacy definition, such as post-processing invariance and graceful composition. Finally, we provide mechanisms that are private in this framework, and show via simulations that they achieve higher utility than the corresponding local differential privacy mechanisms.
TextRank: Bringing Order into Texts
Authors: Rada Mihalcea, Paul Tarau
Date: 2004-07-25 00:00:00
Proceeding: Conference 2004 7
Tasks: Sentence

In this paper, we introduce TextRank – a graph-based ranking model for text processing and show how this model can be successfully used in natural language applications. In particular, we propose two innovative unsupervised methods for keyword and sentence extraction and show that the results obtained compare favorably with previously published results on established benchmarks.
The Neural Hype and Comparisons Against Weak Baselines
Authors: Jimmy Lin
Date: 2018-12-01 00:00:00
Proceeding: ACM SIGIR Forum, Volume 52 Issue 2 2018 12
Tasks: Ad-Hoc Information Retrieval, Cultural Vocal Bursts Intensity Prediction, Hyperparameter Optimization

Recently, the machine learning community paused in a moment of self-reflection. In a widely discussed paper at ICLR 2018, Sculley et al. wrote: "We observe that the rate of empirical advancement may not have been matched by consistent increase in the level of empirical rigor across the field as a whole." Their primary complaint is the development of a "research and publication culture that emphasizes wins" (emphasis in original), which typically means "demonstrating that a new method beats previous methods on a given task or benchmark". An apt description might be "leaderboard chasing"-and for many vision and NLP tasks, this isn't a metaphor. There are literally centralized leaderboards1 that track incremental progress, down to the fifth decimal point, some persisting over years, accumulating dozens of entries.

Sculley et al. remind us that "the goal of science is not wins, but knowledge". The structure of the scientific enterprise today (pressure to publish, pace of progress, etc.) means that "winning" and "doing good science" are often not fully aligned. To wit, they cite a number of papers showing that recent advances in neural networks could very well be attributed to mundane issues like better hyperparameter optimization. Many results can't be reproduced, and some observed improvements might just be noise.
Explainable Reasoning over Knowledge Graphs for Recommendation
Authors: Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, Tat-Seng Chua
Date: 2018-11-12 00:00:00
Tasks: Knowledge Graphs, Recommendation Systems

Incorporating knowledge graph into recommender systems has attracted
increasing attention in recent years. By exploring the interlinks within a
knowledge graph, the connectivity between users and items can be discovered as
paths, which provide rich and complementary information to user-item
interactions. Such connectivity not only reveals the semantics of entities and
relations, but also helps to comprehend a user's interest. However, existing
efforts have not fully explored this connectivity to infer user preferences,
especially in terms of modeling the sequential dependencies within and holistic
semantics of a path. In this paper, we contribute a new model named
Knowledge-aware Path Recurrent Network (KPRN) to exploit knowledge graph for
recommendation. KPRN can generate path representations by composing the
semantics of both entities and relations. By leveraging the sequential
dependencies within a path, we allow effective reasoning on paths to infer the
underlying rationale of a user-item interaction. Furthermore, we design a new
weighted pooling operation to discriminate the strengths of different paths in
connecting a user with an item, endowing our model with a certain level of
explainability. We conduct extensive experiments on two datasets about movie
and music, demonstrating significant improvements over state-of-the-art
solutions Collaborative Knowledge Base Embedding and Neural Factorization
Machine.
Learned Convolutional Sparse Coding
Authors: Hillel Sreter, Raja Giryes
Date: 2017-11-01 00:00:00
Tasks: Decoder, Denoising, Image Denoising, Image Inpainting

We propose a convolutional recurrent sparse auto-encoder model. The model
consists of a sparse encoder, which is a convolutional extension of the learned
ISTA (LISTA) method, and a linear convolutional decoder. Our strategy offers a
simple method for learning a task-driven sparse convolutional dictionary (CD),
and producing an approximate convolutional sparse code (CSC) over the learned
dictionary. We trained the model to minimize reconstruction loss via gradient
decent with back-propagation and have achieved competitive results to KSVD
image denoising and to leading CSC methods in image inpainting requiring only a
small fraction of their run-time.