pwc-archive/papers-with-abstracts huggingface dataset:
Automatic text summarization: What has been done and what has to be done
Authors: Abdelkrime Aries, Djamel eddine Zegour, Walid Khaled Hidouci
Date: 2019-04-01 00:00:00
Tasks: Text Summarization

Summaries are important when it comes to process huge amounts of information.
Their most important benefit is saving time, which we do not have much
nowadays. Therefore, a summary must be short, representative and readable.
Generating summaries automatically can be beneficial for humans, since it can
save time and help selecting relevant documents. Automatic summarization and,
in particular, Automatic text summarization (ATS) is not a new research field;
It was known since the 50s. Since then, researchers have been active to find
the perfect summarization method. In this article, we will discuss different
works in automatic summarization, especially the recent ones. We will present
some problems and limits which prevent works to move forward. Most of these
challenges are much more related to the nature of processed languages. These
challenges are interesting for academics and developers, as a path to follow in
this field.
Question Embeddings Based on Shannon Entropy: Solving intent classification task in goal-oriented dialogue system
Authors: Aleksandr Perevalov, Daniil Kurushin, Rustam Faizrakhmanov, Farida Khabibrakhmanova
Date: 2019-03-25 00:00:00
Tasks: General Classification, intent-classification, Intent Classification, Intent Detection, Natural Language Understanding, Question Answering, Word Embeddings

Question-answering systems and voice assistants are becoming major part of
client service departments of many organizations, helping them to reduce the
labor costs of staff. In many such systems, there is always natural language
understanding module that solves intent classification task. This task is
complicated because of its case-dependency - every subject area has its own
semantic kernel. The state of art approaches for intent classification are
different machine learning and deep learning methods that use text vector
representations as input. The basic vector representation models such as Bag of
words and TF-IDF generate sparse matrixes, which are becoming very big as the
amount of input data grows. Modern methods such as word2vec and FastText use
neural networks to evaluate word embeddings with fixed dimension size. As we
are developing a question-answering system for students and enrollees of the
Perm National Research Polytechnic University, we have faced the problem of
user's intent detection. The subject area of our system is very specific, that
is why there is a lack of training data. This aspect makes intent
classification task more challenging for using state of the art deep learning
methods. In this paper, we propose an approach of the questions embeddings
representation based on calculation of Shannon entropy.The goal of the approach
is to produce low dimensional question vectors as neural approaches do and to
outperform related methods, described above in condition of small dataset. We
evaluate and compare our model with existing ones using logistic regression and
dataset that contains questions asked by students and enrollees. The data is
labeled into six classes. Experimental comparison of proposed approach and
other models revealed that proposed model performed better in the given task.
Making Neural Machine Reading Comprehension Faster
Authors: Debajyoti Chatterjee
Date: 2019-03-29 00:00:00
Tasks: Knowledge Distillation, Machine Reading Comprehension, Natural Language Understanding, Reading Comprehension

This study aims at solving the Machine Reading Comprehension problem where
questions have to be answered given a context passage. The challenge is to
develop a computationally faster model which will have improved inference time.
State of the art in many natural language understanding tasks, BERT model, has
been used and knowledge distillation method has been applied to train two
smaller models. The developed models are compared with other models which have
been developed with the same intention.
Polysemy and brevity versus frequency in language
Authors: Bernardino Casas, Antoni Hernández-Fernández, Neus Català, Ramon Ferrer-i-Cancho, Jaume Baixeries
Date: 2019-03-27 00:00:00

The pioneering research of G. K. Zipf on the relationship between word
frequency and other word features led to the formulation of various linguistic
laws. The most popular is Zipf's law for word frequencies. Here we focus on two
laws that have been studied less intensively: the meaning-frequency law, i.e.
the tendency of more frequent words to be more polysemous, and the law of
abbreviation, i.e. the tendency of more frequent words to be shorter. In a
previous work, we tested the robustness of these Zipfian laws for English,
roughly measuring word length in number of characters and distinguishing adult
from child speech. In the present article, we extend our study to other
languages (Dutch and Spanish) and introduce two additional measures of length:
syllabic length and phonemic length. Our correlation analysis indicates that
both the meaning-frequency law and the law of abbreviation hold overall in all
the analyzed languages.
Unsupervised Abbreviation Disambiguation Contextual disambiguation using word embeddings
Authors: Manuel Ciosici, Tobias Sommer, Ira Assent
Date: 2019-04-01 00:00:00
Tasks: Question Answering, Reading Comprehension, Word Embeddings

Abbreviations often have several distinct meanings, often making their use in text ambiguous. Expanding them to their intended meaning in context is important for Machine Reading tasks such as document search, recommendation and question answering. Existing approaches mostly rely on manually labeled examples of abbreviations and their correct long-forms. Such data sets are costly to create and result in trained models with limited applicability and flexibility. Importantly, most current methods must be subjected to a full empirical evaluation in order to understand their limitations, which is cumbersome in practice. In this paper, we present an entirely unsupervised abbreviation disambiguation method (called UAD) that picks up abbreviation definitions from unstructured text. Creating distinct tokens per meaning, we learn context representations as word vectors. We demonstrate how to further boost abbreviation disambiguation performance by obtaining better context representations using additional unstructured text. Our method is the first abbreviation disambiguation approach with a transparent model that allows performance analysis without requiring full-scale evaluation, making it highly relevant for real-world deployments. In our thorough empirical evaluation, UAD achieves high performance on large real-world data sets from different domains and outperforms both baseline and state-of-the-art methods. UAD scales well and supports thousands of abbreviations with multiple different meanings within a single model. In order to spur more research into abbreviation disambiguation, we publish a new data set, that we also use in our experiments.
Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation
Authors: Nikolai Vogler, Craig Stewart, Graham Neubig
Date: 2019-04-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Translation

Simultaneous interpretation, the translation of speech from one language to
another in real-time, is an inherently difficult and strenuous task. One of the
greatest challenges faced by interpreters is the accurate translation of
difficult terminology like proper names, numbers, or other entities.
Intelligent computer-assisted interpreting (CAI) tools that could analyze the
spoken word and detect terms likely to be untranslated by an interpreter could
reduce translation error and improve interpreter performance. In this paper, we
propose a task of predicting which terminology simultaneous interpreters will
leave untranslated, and examine methods that perform this task using supervised
sequence taggers. We describe a number of task-specific features explicitly
designed to indicate when an interpreter may struggle with translating a word.
Experimental results on a newly-annotated version of the NAIST Simultaneous
Translation Corpus (Shimizu et al., 2014) indicate the promise of our proposed
method.
Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora
Authors: Hieu-Thi Luong, Xin Wang, Junichi Yamagishi, Nobuyuki Nishizawa
Date: 2019-04-01 00:00:00
Tasks: text-to-speech, Text to Speech

When the available data of a target speaker is insufficient to train a high
quality speaker-dependent neural text-to-speech (TTS) system, we can combine
data from multiple speakers and train a multi-speaker TTS model instead. Many
studies have shown that neural multi-speaker TTS model trained with a small
amount data from multiple speakers combined can generate synthetic speech with
better quality and stability than a speaker-dependent one. However when the
amount of data from each speaker is highly unbalanced, the best approach to
make use of the excessive data remains unknown. Our experiments showed that
simply combining all available data from every speaker to train a multi-speaker
model produces better than or at least similar performance to its
speaker-dependent counterpart. Moreover by using an ensemble multi-speaker
model, in which each subsystem is trained on a subset of available data, we can
further improve the quality of the synthetic speech especially for
underrepresented speakers whose training data is limited.
Decomposing information into copying versus transformation
Authors: Artemy Kolchinsky, Bernat Corominas-Murtra
Date: 2019-03-21 00:00:00

In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation. Copying occurs when messages are transmitted without modification, e.g., when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are modified systematically during transmission, e.g., when mutational biases occur during genetic replication. Standard information-theoretic measures do not distinguish these two modes of information transfer, although they may reflect different mechanisms and have different functional consequences. Starting from a few simple axioms, we derive a decomposition of mutual information into the information transmitted by copying versus the information transmitted by transformation. We begin with a decomposition that applies when the source and destination of the channel have the same set of messages and a notion of message identity exists. We then generalize our decomposition to other kinds of channels, which can involve different source and destination sets and broader notions of similarity. In addition, we show that copy information can be interpreted as the minimal work needed by a physical copying process, which is relevant for understanding the physics of replication. We use the proposed decomposition to explore a model of amino acid substitution rates. Our results apply to any system in which the fidelity of copying, rather than simple predictability, is of critical relevance.
A multi-device dataset for urban acoustic scene classification
Authors: Annamaria Mesaros, Toni Heittola, Tuomas Virtanen
Date: 2018-07-25 00:00:00
Tasks: Acoustic Scene Classification, Classification, Scene Classification

This paper introduces the acoustic scene classification task of DCASE 2018
Challenge and the TUT Urban Acoustic Scenes 2018 dataset provided for the task,
and evaluates the performance of a baseline system in the task. As in previous
years of the challenge, the task is defined for classification of short audio
samples into one of predefined acoustic scene classes, using a supervised,
closed-set classification setup. The newly recorded TUT Urban Acoustic Scenes
2018 dataset consists of ten different acoustic scenes and was recorded in six
large European cities, therefore it has a higher acoustic variability than the
previous datasets used for this task, and in addition to high-quality binaural
recordings, it also includes data recorded with mobile devices. We also present
the baseline system consisting of a convolutional neural network and its
performance in the subtasks using the recommended cross-validation setup.
Keyphrase Cloud Generation of Broadcast News
Authors: Luis Marujo, Márcio Viveiros, João Paulo da Silva Neto
Date: 2013-06-19 00:00:00
Tasks: Keyphrase Extraction, speech-recognition, Speech Recognition, TAG

This paper describes an enhanced automatic keyphrase extraction method
applied to Broadcast News. The keyphrase extraction process is used to create a
concept level for each news. On top of words resulting from a speech
recognition system output and news indexation and it contributes to the
generation of a tag/keyphrase cloud of the top news included in a Multimedia
Monitoring Solution system for TV and Radio news/programs, running daily, and
monitoring 12 TV channels and 4 Radios.
Exploring Randomly Wired Neural Networks for Image Recognition
Authors: Saining Xie, Alexander Kirillov, Ross Girshick, Kaiming He
Date: 2019-04-02 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Classification, Neural Architecture Search

Neural networks for image recognition have evolved through extensive manual
design from simple chain-like models to structures with multiple wiring paths.
The success of ResNets and DenseNets is due in large part to their innovative
wiring plans. Now, neural architecture search (NAS) studies are exploring the
joint optimization of wiring and operation types, however, the space of
possible wirings is constrained and still driven by manual design despite being
searched. In this paper, we explore a more diverse set of connectivity patterns
through the lens of randomly wired neural networks. To do this, we first define
the concept of a stochastic network generator that encapsulates the entire
network generation process. Encapsulation provides a unified view of NAS and
randomly wired networks. Then, we use three classical random graph models to
generate randomly wired graphs for networks. The results are surprising:
several variants of these random generators yield network instances that have
competitive accuracy on the ImageNet benchmark. These results suggest that new
efforts focusing on designing better network generators may lead to new
breakthroughs by exploring less constrained search spaces with more room for
novel design.
FCOS: Fully Convolutional One-Stage Object Detection
Authors: Zhi Tian, Chunhua Shen, Hao Chen, Tong He
Date: 2019-04-02 00:00:00
Proceeding: ICCV 2019 10
Tasks: 2D Object Detection, Object, Object Detection, Pedestrian Detection, Semantic Segmentation

We propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the predefined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating overlapping during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often very sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), FCOS with ResNeXt-64x4d-101 achieves 44.7% in AP with single-model and single-scale testing, surpassing previous one-stage detectors with the advantage of being much simpler. For the first time, we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code is available at:Code is available at: https://tinyurl.com/FCOSv1
Res2Net: A New Multi-scale Backbone Architecture
Authors: Shang-Hua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, Philip Torr
Date: 2019-04-02 00:00:00
Tasks: Image Classification, Instance Segmentation, Medical Image Classification, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Representing features at multiple scales is of great importance for numerous vision tasks. Recent advances in backbone convolutional neural networks (CNNs) continually demonstrate stronger multi-scale representation ability, leading to consistent performance gains on a wide range of applications. However, most existing methods represent the multi-scale features in a layer-wise manner. In this paper, we propose a novel building block for CNNs, namely Res2Net, by constructing hierarchical residual-like connections within one single residual block. The Res2Net represents multi-scale features at a granular level and increases the range of receptive fields for each network layer. The proposed Res2Net block can be plugged into the state-of-the-art backbone CNN models, e.g., ResNet, ResNeXt, and DLA. We evaluate the Res2Net block on all these models and demonstrate consistent performance gains over baseline models on widely-used datasets, e.g., CIFAR-100 and ImageNet. Further ablation studies and experimental results on representative computer vision tasks, i.e., object detection, class activation mapping, and salient object detection, further verify the superiority of the Res2Net over the state-of-the-art baseline methods. The source code and trained models are available on https://mmcheng.net/res2net/.
GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments
Authors: Xiaotong Chen, Rui Chen, Zhiqiang Sui, Zhefan Ye, Yanqi Liu, R. Iris Bahar, Odest Chadwicke Jenkins
Date: 2019-03-20 00:00:00
Tasks: Object, object-detection, Object Detection, Pose Estimation, Robot Manipulation

Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer overfitting during training process and are less robust within unseen conditions, which are especially vulnerable to adversarial scenarios. In this work, we propose Generative Robust Inference and Perception (GRIP) as a two-stage object detection and pose estimation system that aims to combine relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of sample-based generative inference is able to recover from false object detection by CNNs, and produce robust estimations in adversarial conditions. We demonstrate the efficacy of GRIP robustness through comparison with state-of-the-art learning-based pose estimators and pick-and-place manipulation in dark and cluttered environments.
Natural Image Stitching with the Global Similarity Prior
Authors: Yu-Sheng Chen; Yung-Yu Chuang
Date: 2016-10-01 00:00:00
Proceeding: European Conference on Computer Vision 2016 10
Tasks: Image Stitching, Local Distortion

This paper proposes a method for stitching multiple images together so that the stitched image looks as natural as possible. Our method adopts the local warp model and guides the warping of each image with a grid mesh. An objective function is designed for specifying the desired characteristics of the warps. In addition to good alignment and minimal local distortion, we add a global similarity prior in the objective function. This prior constrains the warp of each image so that it resembles a similarity transformation as a whole. The selection of the similarity transformation is crucial to the naturalness of the results. We propose methods for selecting the proper scale and rotation for each image. The warps of all images are solved together for minimizing the distortion globally. A comprehensive evaluation shows that the proposed method consistently outperforms several state-of-the-art methods, including AutoStitch, APAP, SPHP and ANNAP.
Analyzing Learned Molecular Representations for Property Prediction
Authors: Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi Jaakkola, Klavs Jensen, Regina Barzilay
Date: 2019-04-02 00:00:00
Tasks: Molecular Property Prediction, molecular representation, Prediction, Property Prediction

Advancements in neural machinery have led to a wide range of algorithmic solutions for molecular property prediction. Two classes of models in particular have yielded promising results: neural networks applied to computed molecular fingerprints or expert-crafted descriptors, and graph convolutional neural networks that construct a learned molecular representation by operating on the graph structure of the molecule. However, recent literature has yet to clearly determine which of these two methods is superior when generalizing to new chemical space. Furthermore, prior research has rarely examined these new models in industry research settings in comparison to existing employed models. In this paper, we benchmark models extensively on 19 public and 16 proprietary industrial datasets spanning a wide variety of chemical endpoints. In addition, we introduce a graph convolutional model that consistently matches or outperforms models using fixed molecular descriptors as well as previous graph neural architectures on both public and proprietary datasets. Our empirical findings indicate that while approaches based on these representations have yet to reach the level of experimental reproducibility, our proposed model nevertheless offers significant improvements over models currently used in industrial workflows.
Analysing Mathematical Reasoning Abilities of Neural Models
Authors: David Saxton, Edward Grefenstette, Felix Hill, Pushmeet Kohli
Date: 2019-04-02 00:00:00
Proceeding: ICLR 2019 5
Tasks: Mathematical Question Answering, Mathematical Reasoning, Math Word Problem Solving, Question Answering

Mathematical reasoning---a core ability within human intelligence---presents
some unique challenges as a domain: we do not come to understand and solve
mathematical problems primarily on the back of experience and evidence, but on
the basis of inferring, learning, and exploiting laws, axioms, and symbol
manipulation rules. In this paper, we present a new challenge for the
evaluation (and eventually the design) of neural architectures and similar
system, developing a task suite of mathematics problems involving sequential
questions and answers in a free-form textual input/output format. The
structured nature of the mathematics domain, covering arithmetic, algebra,
probability and calculus, enables the construction of training and test splits
designed to clearly illuminate the capabilities and failure-modes of different
architectures, as well as evaluate their ability to compose and relate
knowledge and learned processes. Having described the data generation process
and its potential future expansions, we conduct a comprehensive analysis of
models from two broad classes of the most powerful sequence-to-sequence
architectures and find notable differences in their ability to resolve
mathematical problems and generalize their knowledge.