pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for Low-Dose CT Denoising
Authors: Maryam Gholizadeh-Ansari, Javad Alirezaie, Paul Babyn
Date: 2019-02-25 00:00:00
Tasks: Deep Learning, Denoising, Edge Detection

Low-dose CT denoising is a challenging task that has been studied by many
researchers. Some studies have used deep neural networks to improve the quality
of low-dose CT images and achieved fruitful results. In this paper, we propose
a deep neural network that uses dilated convolutions with different dilation
rates instead of standard convolution helping to capture more contextual
information in fewer layers. Also, we have employed residual learning by
creating shortcut connections to transmit image information from the early
layers to later ones. To further improve the performance of the network, we
have introduced a non-trainable edge detection layer that extracts edges in
horizontal, vertical, and diagonal directions. Finally, we demonstrate that
optimizing the network by a combination of mean-square error loss and
perceptual loss preserves many structural details in the CT image. This
objective function does not suffer from over smoothing and blurring effects
caused by per-pixel loss and grid-like artifacts resulting from perceptual
loss. The experiments show that each modification to the network improves the
outcome while only minimally changing the complexity of the network.
BUT-FIT at SemEval-2019 Task 7: Determining the Rumour Stance with Pre-Trained Deep Bidirectional Transformers
Authors: Martin Fajcik, Lukáš Burget, Pavel Smrz
Date: 2019-02-25 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification, Rumour Detection, Stance Classification

This paper describes our system submitted to SemEval 2019 Task 7: RumourEval
2019: Determining Rumour Veracity and Support for Rumours, Subtask A (Gorrell
et al., 2019). The challenge focused on classifying whether posts from Twitter
and Reddit support, deny, query, or comment a hidden rumour, truthfulness of
which is the topic of an underlying discussion thread. We formulate the problem
as a stance classification, determining the rumour stance of a post with
respect to the previous thread post and the source thread post. The recent BERT
architecture was employed to build an end-to-end system which has reached the
F1 score of 61.67% on the provided test data. It finished at the 2nd place in
the competition, without any hand-crafted features, only 0.2% behind the
winner.
Learning to Optimize Multigrid PDE Solvers
Authors: Daniel Greenfeld, Meirav Galun, Ron Kimmel, Irad Yavneh, Ronen Basri
Date: 2019-02-25 00:00:00

Constructing fast numerical solvers for partial differential equations (PDEs) is crucial for many scientific disciplines. A leading technique for solving large-scale PDEs is using multigrid methods. At the core of a multigrid solver is the prolongation matrix, which relates between different scales of the problem. This matrix is strongly problem-dependent, and its optimal construction is critical to the efficiency of the solver. In practice, however, devising multigrid algorithms for new problems often poses formidable challenges. In this paper we propose a framework for learning multigrid solvers. Our method learns a (single) mapping from a family of parameterized PDEs to prolongation operators. We train a neural network once for the entire class of PDEs, using an efficient and unsupervised loss function. Experiments on a broad class of 2D diffusion problems demonstrate improved convergence rates compared to the widely used Black-Box multigrid scheme, suggesting that our method successfully learned rules for constructing prolongation matrices.
Leveraging Deep Graph-Based Text Representation for Sentiment Polarity Applications
Authors: Kayvan Bijari, Hadi Zare, Emad Kebriaei, Hadi Veisi
Date: 2019-02-23 00:00:00
Tasks: BIG-bench Machine Learning, Document Classification, Event Detection, Information Retrieval, News Recommendation, Opinion Mining, Representation Learning, Sentence, Sentiment Analysis, Sentiment Classification, Word Embeddings

Over the last few years, machine learning over graph structures has manifested a significant enhancement in text mining applications such as event detection, opinion mining, and news recommendation. One of the primary challenges in this regard is structuring a graph that encodes and encompasses the features of textual data for the effective machine learning algorithm. Besides, exploration and exploiting of semantic relations is regarded as a principal step in text mining applications. However, most of the traditional text mining methods perform somewhat poor in terms of employing such relations. In this paper, we propose a sentence-level graph-based text representation which includes stop words to consider semantic and term relations. Then, we employ a representation learning approach on the combined graphs of sentences to extract the latent and continuous features of the documents. Eventually, the learned features of the documents are fed into a deep neural network for the sentiment classification task. The experimental results demonstrate that the proposed method substantially outperforms the related sentiment analysis approaches based on several benchmark datasets. Furthermore, our method can be generalized on different datasets without any dependency on pre-trained word embeddings.
Fixed-Size Ordinally Forgetting Encoding Based Word Sense Disambiguation
Authors: Xi Zhu, MingBin Xu, Hui Jiang
Date: 2019-02-23 00:00:00
Tasks: Language Modeling, Language Modelling, Word Sense Disambiguation

In this paper, we present our method of using fixed-size ordinally forgetting
encoding (FOFE) to solve the word sense disambiguation (WSD) problem. FOFE
enables us to encode variable-length sequence of words into a theoretically
unique fixed-size representation that can be fed into a feed forward neural
network (FFNN), while keeping the positional information between words. In our
method, a FOFE-based FFNN is used to train a pseudo language model over
unlabelled corpus, then the pre-trained language model is capable of
abstracting the surrounding context of polyseme instances in labelled corpus
into context embeddings. Next, we take advantage of these context embeddings
towards WSD classification. We conducted experiments on several WSD data sets,
which demonstrates that our proposed method can achieve comparable performance
to that of the state-of-the-art approach at the expense of much lower
computational cost.
Fast Multi-language LSTM-based Online Handwriting Recognition
Authors: Victor Carbune, Pedro Gonnet, Thomas Deselaers, Henry A. Rowley, Alexander Daryin, Marcos Calvo, Li-Lun Wang, Daniel Keysers, Sandro Feuz, Philippe Gervais
Date: 2019-02-22 00:00:00
Tasks: Handwriting Recognition

We describe an online handwriting system that is able to support 102 languages using a deep neural network architecture. This new system has completely replaced our previous Segment-and-Decode-based system and reduced the error rate by 20%-40% relative for most languages. Further, we report new state-of-the-art results on IAM-OnDB for both the open and closed dataset setting. The system combines methods from sequence recognition with a new input encoding using B\'ezier curves. This leads to up to 10x faster recognition times compared to our previous system. Through a series of experiments we determine the optimal configuration of our models and report the results of our setup on a number of additional public datasets.
Non-Autoregressive Machine Translation with Auxiliary Regularization
Authors: Yiren Wang, Fei Tian, Di He, Tao Qin, ChengXiang Zhai, Tie-Yan Liu
Date: 2019-02-22 00:00:00
Tasks: Decoder, Machine Translation, Sentence, Translation

As a new neural machine translation approach, Non-Autoregressive machine
Translation (NAT) has attracted attention recently due to its high efficiency
in inference. However, the high efficiency has come at the cost of not
capturing the sequential dependency on the target side of translation, which
causes NAT to suffer from two kinds of translation errors: 1) repeated
translations (due to indistinguishable adjacent decoder hidden states), and 2)
incomplete translations (due to incomplete transfer of source side information
via the decoder hidden states).
  In this paper, we propose to address these two problems by improving the
quality of decoder hidden representations via two auxiliary regularization
terms in the training process of an NAT model. First, to make the hidden states
more distinguishable, we regularize the similarity between consecutive hidden
states based on the corresponding target tokens. Second, to force the hidden
states to contain all the information in the source sentence, we leverage the
dual nature of translation tasks (e.g., English to German and German to
English) and minimize a backward reconstruction error to ensure that the hidden
states of the NAT decoder are able to recover the source side sentence.
Extensive experiments conducted on several benchmark datasets show that both
regularization strategies are effective and can alleviate the issues of
repeated translations and incomplete translations in NAT models. The accuracy
of NAT models is therefore improved significantly over the state-of-the-art NAT
models with even better efficiency for inference.
ROMANet: Fine-Grained Reuse-Driven Off-Chip Memory Access Management and Data Organization for Deep Neural Network Accelerators
Authors: Rachmad Vidya Wicaksana Putra, Muhammad Abdullah Hanif, Muhammad Shafique
Date: 2019-02-04 00:00:00
Tasks: Management, Scheduling

Enabling high energy efficiency is crucial for embedded implementations of deep learning. Several studies have shown that the DRAM-based off-chip memory accesses are one of the most energy-consuming operations in deep neural network (DNN) accelerators, and thereby limit the designs from achieving efficiency gains at the full potential. DRAM access energy varies depending upon the number of accesses required as well as the energy consumed per-access. Therefore, searching for a solution towards the minimum DRAM access energy is an important optimization problem. Towards this, we propose the ROMANet methodology that aims at reducing the number of memory accesses, by searching for the appropriate data partitioning and scheduling for each layer of a network using a design space exploration, based on the knowledge of the available on-chip memory and the data reuse factors. Moreover, ROMANet also targets decreasing the number of DRAM row buffer conflicts and misses, by exploiting the DRAM multi-bank burst feature to improve the energy-per-access. Besides providing the energy benefits, our proposed DRAM data mapping also results in an increased effective DRAM throughput, which is useful for latency-constraint scenarios. Our experimental results show that the ROMANet saves DRAM access energy by 12% for the AlexNet, by 36% for the VGG-16, and by 46% for the MobileNet, while also improving the DRAM throughput by 10%, as compared to the state-of-the-art.
Survey of Higher Order Rigid Body Motion Interpolation Methods for Keyframe Animation and Continuous-Time Trajectory Estimation
Authors: Adrian Haarbach, Tolga Birdal, Slobodan Ilic
Date: 2018-09-05 00:00:00
Proceeding: 3D Vision 2018 2018 9
Tasks: Motion Interpolation

In this survey we carefully analyze the characteristics of higher order rigid body motion interpolation methods to obtain a continuous trajectory from a discrete set of poses. We first discuss the tradeoff between continuity, local control and approximation of classical Euclidean interpolation schemes such as Bezier and B-splines. The benefits of the manifold of unit quaternions SU(2), a double-cover of rotation matrices SO(3), as rotation parameterization are presented, which allow for an elegant formulation of higher order orientation interpolation with easy analytic derivatives, made possible through the Lie Algebra su(2) of pure quaternions and the cumulative form of cubic B-splines. The same construction scheme is then applied for joint interpolation in the full rigid body pose space, which had previously been done for the matrix representation SE(3) and its twists, but not for the more efficient unit dual quaternion DH1 and its screw motions. Both suffer from the effects of coupling translation and rotation that have mostly been ignored by previous work. We thus conclude that split interpolation in R3 × SU(2) is preferable for most applications. Our final runtime experiments show that joint interpolation in SE(3) is 2 times and in DH1 1.3 times slower - which furthermore justifies our suggestion from a practical point of view.
PyCM: Multiclass confusion matrix library in Python
Authors: Sepand Haghighi, Masoomeh Jasemi, Shaahin Hessabi, Alireza Zolanvari
Date: 2018-05-29 00:00:00
Proceeding: The Journal of Open Source Software 2018 5
Tasks: General Classification

PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters. PyCM is the swiss-army knife of confusion matrices, targeted mainly at data scientists that need a broad array of metrics for predictive models and an accurate evaluation of large variety of classifiers.
EpistAid: Interactive Interface for Document Filtering in Evidence-based Health Care
Authors: Ivania Donoso, Denis Parra
Date: 2016-11-07 00:00:00
Tasks: Articles

Evidence-based health care (EBHC) is an important practice of medicine which
attempts to provide systematic scientific evidence to answer clinical
questions. In this context, Epistemonikos (www.epistemonikos.org) is one of the
first and most important online systems in the field, providing an interface
that supports users on searching and filtering scientific articles for
practicing EBHC. The system nowadays requires a large amount of expert human
effort, where close to 500 physicians manually curate articles to be utilized
in the platform. In order to scale up the large and continuous amount of data
to keep the system updated, we introduce EpistAid, an interactive intelligent
interface which supports clinicians in the process of curating documents for
Epistemonikos within lists of papers called evidence matrices. We introduce the
characteristics, design and algorithms of our solution, as well as a prototype
implementation and a case study to show how our solution addresses the
information overload problem in this area.
Predictive Inequity in Object Detection
Authors: Benjamin Wilson, Judy Hoffman, Jamie Morgenstern
Date: 2019-02-21 00:00:00
Tasks: Object, object-detection, Object Detection

In this work, we investigate whether state-of-the-art object detection
systems have equitable predictive performance on pedestrians with different
skin tones. This work is motivated by many recent examples of ML and vision
systems displaying higher error rates for certain demographic groups than
others. We annotate an existing large scale dataset which contains pedestrians,
BDD100K, with Fitzpatrick skin tones in ranges [1-3] or [4-6]. We then provide
an in-depth comparative analysis of performance between these two skin tone
groupings, finding that neither time of day nor occlusion explain this
behavior, suggesting this disparity is not merely the result of pedestrians in
the 4-6 range appearing in more difficult scenes for detection. We investigate
to what extent time of day, occlusion, and reweighting the supervised loss
during training affect this predictive bias.
A lattice-based approach to the expressivity of deep ReLU neural networks
Authors: Vincent Corlay, Joseph J. Boutros, Philippe Ciblat, Loic Brunel
Date: 2019-02-28 00:00:00

We present new families of continuous piecewise linear (CPWL) functions in Rn having a number of affine pieces growing exponentially in $n$. We show that these functions can be seen as the high-dimensional generalization of the triangle wave function used by Telgarsky in 2016. We prove that they can be computed by ReLU networks with quadratic depth and linear width in the space dimension. We also investigate the approximation error of one of these functions by shallower networks and prove a separation result. The main difference between our functions and other constructions is their practical interest: they arise in the scope of channel coding. Hence, computing such functions amounts to performing a decoding operation.
FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
Authors: Felix Wu, Boyi Li, Lequn Wang, Ni Lao, John Blitzer, Kilian Q. Weinberger
Date: 2019-02-28 00:00:00
Tasks: Reading Comprehension, Retrieval

In this technical report, we introduce FastFusionNet, an efficient variant of
FusionNet [12]. FusionNet is a high performing reading comprehension
architecture, which was designed primarily for maximum retrieval accuracy with
less regard towards computational requirements. For FastFusionNets we remove
the expensive CoVe layers [21] and substitute the BiLSTMs with far more
efficient SRU layers [19]. The resulting architecture obtains state-of-the-art
results on DAWNBench [5] while achieving the lowest training and inference time
on SQuAD [25] to-date. The code is available at
https://github.com/felixgwu/FastFusionNet.
Multi-Criteria Dimensionality Reduction with Applications to Fairness
Authors: Uthaipon Tantipongpipat, Samira Samadi, Mohit Singh, Jamie Morgenstern, Santosh Vempala
Date: 2019-02-28 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Dimensionality Reduction, Fairness

Dimensionality reduction is a classical technique widely used for data analysis. One foundational instantiation is Principal Component Analysis (PCA), which minimizes the average reconstruction error. In this paper, we introduce the "multi-criteria dimensionality reduction" problem where we are given multiple objectives that need to be optimized simultaneously. As an application, our model captures several fairness criteria for dimensionality reduction such as our novel Fair-PCA problem and the Nash Social Welfare (NSW) problem. In Fair-PCA, the input data is divided into $k$ groups, and the goal is to find a single $d$-dimensional representation for all groups for which the minimum variance of any one group is maximized. In NSW, the goal is to maximize the product of the individual variances of the groups achieved by the common low-dimensional space. Our main result is an exact polynomial-time algorithm for the two-criterion dimensionality reduction problem when the two criteria are increasing concave functions. As an application of this result, we obtain a polynomial time algorithm for Fair-PCA for $k=2$ groups and a polynomial time algorithm for NSW objective for $k=2$ groups. We also give approximation algorithms for $k>2$. Our technical contribution in the above results is to prove new low-rank properties of extreme point solutions to semi-definite programs. We conclude with experiments indicating the effectiveness of algorithms based on extreme point solutions of semi-definite programs on several real-world data sets.
From Visual to Acoustic Question Answering
Authors: Jerome Abdelnour, Giampiero Salvi, Jean Rouat
Date: 2019-02-28 00:00:00
Tasks: Acoustic Question Answering, Position, Question Answering, Visual Reasoning

We introduce the new task of Acoustic Question Answering (AQA) to promote
research in acoustic reasoning. The AQA task consists of analyzing an acoustic
scene composed by a combination of elementary sounds and answering questions
that relate the position and properties of these sounds. The kind of relational
questions asked, require that the models perform non-trivial reasoning in order
to answer correctly. Although similar problems have been extensively studied in
the domain of visual reasoning, we are not aware of any previous studies
addressing the problem in the acoustic domain. We propose a method for
generating the acoustic scenes from elementary sounds and a number of relevant
questions for each scene using templates. We also present preliminary results
obtained with two models (FiLM and MAC) that have been shown to work for visual
reasoning.
A Novel Multi-Attention Driven System For Multi-Label Remote Sensing Image Classification
Authors: Gencer Sumbul, Begüm Demir
Date: 2019-02-28 00:00:00
Tasks: General Classification, image-classification, Image Classification, Remote Sensing Image Classification

This paper presents a novel multi-attention driven system that jointly exploits Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) in the context of multi-label remote sensing (RS) image classification. The proposed system consists of four main modules. The first module aims to extract preliminary local descriptors of RS image bands that can be associated to different spatial resolutions. To this end, we introduce a K-Branch CNN, in which each branch extracts descriptors of image bands that have the same spatial resolution. The second module aims to model spatial relationship among local descriptors. This is achieved by a bidirectional RNN architecture, in which Long Short-Term Memory nodes enrich local descriptors by considering spatial relationships of local areas (image patches). The third module aims to define multiple attention scores for local descriptors. This is achieved by a novel patch-based multi-attention mechanism that takes into account the joint occurrence of multiple land-cover classes and provides the attention-based local descriptors. The last module exploits these descriptors for multi-label RS image classification. Experimental results obtained on the BigEarthNet that is a large-scale Sentinel-2 benchmark archive show the effectiveness of the proposed method compared to a state of the art method.
Efficient Contextual Representation Learning Without Softmax Layer
Authors: Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang
Date: 2019-02-28 00:00:00
Tasks: Dimensionality Reduction, Language Modeling, Language Modelling, Representation Learning, Word Embeddings

Contextual representation models have achieved great success in improving
various downstream tasks. However, these language-model-based encoders are
difficult to train due to the large parameter sizes and high computational
complexity. By carefully examining the training procedure, we find that the
softmax layer (the output layer) causes significant inefficiency due to the
large vocabulary size. Therefore, we redesign the learning objective and
propose an efficient framework for training contextual representation models.
Specifically, the proposed approach bypasses the softmax layer by performing
language modeling with dimension reduction, and allows the models to leverage
pre-trained word embeddings. Our framework reduces the time spent on the output
layer to a negligible level, eliminates almost all the trainable parameters of
the softmax layer and performs language modeling without truncating the
vocabulary. When applied to ELMo, our method achieves a 4 times speedup and
eliminates 80% trainable parameters while achieving competitive performance on
downstream tasks.
CircConv: A Structured Convolution with Low Complexity
Authors: Siyu Liao, Zhe Li, Liang Zhao, Qinru Qiu, Yanzhi Wang, Bo Yuan
Date: 2019-02-28 00:00:00

Deep neural networks (DNNs), especially deep convolutional neural networks
(CNNs), have emerged as the powerful technique in various machine learning
applications. However, the large model sizes of DNNs yield high demands on
computation resource and weight storage, thereby limiting the practical
deployment of DNNs. To overcome these limitations, this paper proposes to
impose the circulant structure to the construction of convolutional layers, and
hence leads to circulant convolutional layers (CircConvs) and circulant CNNs.
The circulant structure and models can be either trained from scratch or
re-trained from a pre-trained non-circulant model, thereby making it very
flexible for different training environments. Through extensive experiments,
such strong structure-imposing approach is proved to be able to substantially
reduce the number of parameters of convolutional layers and enable significant
saving of computational cost by using fast multiplication of the circulant
tensor.