pwc-archive/papers-with-abstracts huggingface dataset:
Combining SMT and NMT Back-Translated Data for Efficient NMT
Authors: Alberto Poncelas, Maja Popovic, Dimitar Shterionov, Gideon Maillette de Buy Wenniger, Andy Way
Date: 2019-09-09 00:00:00
Tasks: Machine Translation, NMT, Translation

Neural Machine Translation (NMT) models achieve their best performance when large sets of parallel data are used for training. Consequently, techniques for augmenting the training set have become popular recently. One of these methods is back-translation (Sennrich et al., 2016), which consists on generating synthetic sentences by translating a set of monolingual, target-language sentences using a Machine Translation (MT) model. Generally, NMT models are used for back-translation. In this work, we analyze the performance of models when the training data is extended with synthetic data using different MT approaches. In particular we investigate back-translated data generated not only by NMT but also by Statistical Machine Translation (SMT) models and combinations of both. The results reveal that the models achieve the best performances when the training set is augmented with back-translated data created by merging different MT approaches.
Learning Visual Dynamics Models of Rigid Objects using Relational Inductive Biases
Authors: Fabio Ferreira, Lin Shao, Tamim Asfour, Jeannette Bohg
Date: 2019-09-09 00:00:00
Tasks: Inductive Bias

Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work, we learn action-conditional forward dynamics models of a simulated manipulation task from visual observations involving cluttered and irregularly shaped objects. We investigate two GNN approaches and empirically assess their capability to generalize to scenarios with novel and an increasing number of objects. The first, Graph Networks (GN) based approach, considers explicitly defined edge attributes and not only does it consistently underperform an auto-encoder baseline that we modified to predict future states, our results indicate how different edge attributes can significantly influence the predictions. Consequently, we develop the Auto-Predictor that does not rely on explicitly defined edge attributes. It outperforms the baseline and the GN-based models. Overall, our results show the sensitivity of GNN-based approaches to the task representation, the efficacy of relational inductive biases and advocate choosing lightweight approaches that implicitly reason about relations over ones that leave these decisions to human designers.
Deep Super-Resolution Network for Single Image Super-Resolution with Realistic Degradations
Authors: Rao Muhammad Umer, Gian Luca Foresti, Christian Micheloni
Date: 2019-09-09 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Single Image Super-Resolution (SISR) aims to generate a high-resolution (HR) image of a given low-resolution (LR) image. The most of existing convolutional neural network (CNN) based SISR methods usually take an assumption that a LR image is only bicubicly down-sampled version of an HR image. However, the true degradation (i.e. the LR image is a bicubicly downsampled, blurred and noisy version of an HR image) of a LR image goes beyond the widely used bicubic assumption, which makes the SISR problem highly ill-posed nature of inverse problems. To address this issue, we propose a deep SISR network that works for blur kernels of different sizes, and different noise levels in an unified residual CNN-based denoiser network, which significantly improves a practical CNN-based super-resolver for real applications. Extensive experimental results on synthetic LR datasets and real images demonstrate that our proposed method not only can produce better results on more realistic degradation but also computational efficient to practical SISR applications.
Reasoning Over Semantic-Level Graph for Fact Checking
Authors: Wanjun Zhong, Jingjing Xu, Duyu Tang, Zenan Xu, Nan Duan, Ming Zhou, Jiahai Wang, Jian Yin
Date: 2019-09-09 00:00:00
Proceeding: ACL 2020 6
Tasks: Claim Verification, Fact Checking, Fact Verification, Graph Attention, Graph Neural Network, Representation Learning, Semantic Role Labeling

Fact checking is a challenging task because verifying the truthfulness of a claim requires reasoning about multiple retrievable evidence. In this work, we present a method suitable for reasoning about the semantic-level structure of evidence. Unlike most previous works, which typically represent evidence sentences with either string concatenation or fusing the features of isolated evidence sentences, our approach operates on rich semantic structures of evidence obtained by semantic role labeling. We propose two mechanisms to exploit the structure of evidence while leveraging the advances of pre-trained models like BERT, GPT or XLNet. Specifically, using XLNet as the backbone, we first utilize the graph structure to re-define the relative distances of words, with the intuition that semantically related words should have short distances. Then, we adopt graph convolutional network and graph attention network to propagate and aggregate information from neighboring nodes on the graph. We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy. Our model is the state-of-the-art system in terms of both official evaluation metrics, namely claim verification accuracy and FEVER score.
Efficient Continual Learning in Neural Networks with Embedding Regularization
Authors: Jary Pomponi, Simone Scardapane, Vincenzo Lomonaco, Aurelio Uncini
Date: 2019-09-09 00:00:00
Tasks: Continual Learning, Lifelong learning

Continual learning of deep neural networks is a key requirement for scaling them up to more complex applicative scenarios and for achieving real lifelong learning of these architectures. Previous approaches to the problem have considered either the progressive increase in the size of the networks, or have tried to regularize the network behavior to equalize it with respect to previously observed tasks. In the latter case, it is essential to understand what type of information best represents this past behavior. Common techniques include regularizing the past outputs, gradients, or individual weights. In this work, we propose a new, relatively simple and efficient method to perform continual learning by regularizing instead the network internal embeddings. To make the approach scalable, we also propose a dynamic sampling strategy to reduce the memory footprint of the required external storage. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, while requiring significantly less space in memory and computational time. In addition, inspired inspired by to recent works, we evaluate the impact of selecting a more flexible model for the activation functions inside the network, evaluating the impact of catastrophic forgetting on the activation functions themselves.
Off-Policy Evaluation in Partially Observable Environments
Authors: Guy Tennenholtz, Shie Mannor, Uri Shalit
Date: 2019-09-09 00:00:00
Tasks: Off-policy evaluation, Reinforcement Learning, Reinforcement Learning (RL)

This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments. Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors. We define the problem of off-policy evaluation for Partially Observable Markov Decision Processes (POMDPs) and establish what we believe is the first off-policy evaluation result for POMDPs. In addition, we formulate a model in which observed and unobserved variables are decoupled into two dynamic processes, called a Decoupled POMDP. We show how off-policy evaluation can be performed under this new model, mitigating estimation errors inherent to general POMDPs. We demonstrate the pitfalls of off-policy evaluation in POMDPs using a well-known off-policy method, Importance Sampling, and compare it with our result on synthetic medical data.
Toward a Knowledge-based Personalised Recommender System for Mobile App Development
Authors: Bilal Abu-Salih, Hamad Alsawalqah, Basima Elshqeirat, Tomayess Issa, Pornpit Wongthongtham
Date: 2019-09-09 00:00:00
Tasks: Recommendation Systems

Over the last few years, the arena of mobile application development has expanded considerably beyond the balance of the world\'s software markets. With the growing number of mobile software companies, and the mounting sophistication of smartphones\' technology, developers have been building several categories of applications on dissimilar platforms. However, developers confront several challenges through the implementation of mobile application projects. In particular, there is a lack of consolidated systems that are competent to provide developers with personalised services promptly and efficiently. Hence, it is essential to develop tailored systems which can recommend appropriate tools, IDEs, platforms, software components and other correlated artifacts to mobile application developers. This paper proposes a new recommender system framework comprising a fortified set of techniques that are designed to provide mobile app developers with a distinctive platform to browse and search for the personalised artifacts. The proposed system make use of ontology and semantic web technology as well as machine learning techniques. In particular, the new RS framework comprises the following components; (i) domain knowledge inference module: including various semantic web technologies and lightweight ontologies; (ii) profiling and preferencing: a new proposed time-aware multidimensional user modelling; (iii) query expansion: to improve and enhance the retrieved results by semantically augmenting users\' query; and (iv) recommendation and information filtration: to make use of the aforementioned components to provide personalised services to the designated users and to answer a user\'s query with the minimum mismatches.
Optimal Function Approximation with Relu Neural Networks
Authors: Bo Liu, Yi Liang
Date: 2019-09-09 00:00:00

We consider in this paper the optimal approximations of convex univariate functions with feed-forward Relu neural networks. We are interested in the following question: what is the minimal approximation error given the number of approximating linear pieces? We establish the necessary and sufficient conditions and uniqueness of optimal approximations, and give lower and upper bounds of the optimal approximation errors. Relu neural network architectures are then presented to generate these optimal approximations. Finally, we propose an algorithm to find the optimal approximations, as well as prove its convergence and validate it with experimental results.
Machine learning for automatic construction of pseudo-realistic pediatric abdominal phantoms
Authors: Marco Virgolin, Ziyuan Wang, Tanja Alderliesten, Peter A. N. Bosman
Date: 2019-09-09 00:00:00
Tasks: Anatomy, BIG-bench Machine Learning, Computed Tomography (CT)

Machine Learning (ML) is proving extremely beneficial in many healthcare applications. In pediatric oncology, retrospective studies that investigate the relationship between treatment and late adverse effects still rely on simple heuristics. To assess the effects of radiation therapy, treatment plans are typically simulated on phantoms, i.e., virtual surrogates of patient anatomy. Currently, phantoms are built according to reasonable, yet simple, human-designed criteria. This often results in a lack of individualization. We present a novel approach that combines imaging and ML to build individualized phantoms automatically. Given the features of a patient treated historically (only 2D radiographs available), and a database of 3D Computed Tomography (CT) imaging with organ segmentations and relative patient features, our approach uses ML to predict how to assemble a patient-specific phantom automatically. Experiments on 60 abdominal CTs of pediatric patients show that our approach constructs significantly more representative phantoms than using current phantom building criteria, in terms of location and shape of the abdomen and of two considered organs, the liver and the spleen. Among several ML algorithms considered, the Gene-pool Optimal Mixing Evolutionary Algorithm for Genetic Programming (GP-GOMEA) is found to deliver the best performing models, which are, moreover, transparent and interpretable mathematical expressions.
Improving Neural Question Generation using World Knowledge
Authors: Deepak Gupta, Kaheer Suleman, Mahmoud Adada, Andrew McNamara, Justin Harris
Date: 2019-09-09 00:00:00
Tasks: Question Generation, Question-Generation, World Knowledge

In this paper, we propose a method for incorporating world knowledge (linked entities and fine-grained entity types) into a neural question generation model. This world knowledge helps to encode additional information related to the entities present in the passage required to generate human-like questions. We evaluate our models on both SQuAD and MS MARCO to demonstrate the usefulness of the world knowledge features. The proposed world knowledge enriched question generation model is able to outperform the vanilla neural question generation model by 1.37 and 1.59 absolute BLEU 4 score on SQuAD and MS MARCO test dataset respectively.
Latent Multi-view Semi-Supervised Classification
Authors: Xiaofan Bo, Zhao Kang, Zhitong Zhao, Yuanzhang Su, Wenyu Chen
Date: 2019-09-09 00:00:00
Tasks: Classification, General Classification, graph construction, Graph Learning, Representation Learning

To explore underlying complementary information from multiple views, in this paper, we propose a novel Latent Multi-view Semi-Supervised Classification (LMSSC) method. Unlike most existing multi-view semi-supervised classification methods that learn the graph using original features, our method seeks an underlying latent representation and performs graph learning and label propagation based on the learned latent representation. With the complementarity of multiple views, the latent representation could depict the data more comprehensively than every single view individually, accordingly making the graph more accurate and robust as well. Finally, LMSSC integrates latent representation learning, graph construction, and label propagation into a unified framework, which makes each subtask optimized. Experimental results on real-world benchmark datasets validate the effectiveness of our proposed method.
Sparse linear regression with compressed and low-precision data via concave quadratic programming
Authors: Vito Cerone, Sophie M. Fosson, Diego Regruto
Date: 2019-09-09 00:00:00
Tasks: Quantization, regression

We consider the problem of the recovery of a k-sparse vector from compressed linear measurements when data are corrupted by a quantization noise. When the number of measurements is not sufficiently large, different $k$-sparse solutions may be present in the feasible set, and the classical l1 approach may be unsuccessful. For this motivation, we propose a non-convex quadratic programming method, which exploits prior information on the magnitude of the non-zero parameters. This results in a more efficient support recovery. We provide sufficient conditions for successful recovery and numerical simulations to illustrate the practical feasibility of the proposed method.
Estimating Granger Causality with Unobserved Confounders via Deep Latent-Variable Recurrent Neural Network
Authors: Yuan Meng
Date: 2019-09-09 00:00:00
Tasks: Time Series Analysis

Granger causality analysis, as one of the most popular time series causality methods, has been widely used in the economics, neuroscience. However, unobserved confounders is a fundamental problem in the observational studies, which is still not solved for the non-linear Granger causality. The application works often deal with this problem in virtue of the proxy variables, who can be treated as a measure of the confounder with noise. But the proxy variables has been proved to be unreliable, because of the bias it may induce. In this paper, we try to "recover" the unobserved confounders for the Granger causality. We use a generative model with latent variable to build the relationship between the unobserved confounders and the observed variables(tested variable and the proxy variables). The posterior distribution of the latent variable is adopted to represent the confounders distribution, which can be sampled to get the estimated confounders. We adopt the variational autoencoder to estimate the intractable posterior distribution. The recurrent neural network is applied to build the temporal relationship in the data. We evaluate our method in the synthetic and semi-synthetic dataset. The result shows our estimated confounders has a better performance than the proxy variables in the non-linear Granger causality with multiple proxies in the semi-synthetic dataset. But the performances of the synthetic dataset and the different noise level of proxy seem terrible. Any advice can really help.
Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Authors: Christopher Clark, Mark Yatskar, Luke Zettlemoyer
Date: 2019-09-09 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Natural Language Inference, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

State-of-the-art models often make use of superficial patterns in the data that do not generalize well to out-of-domain or adversarial settings. For example, textual entailment models often learn that particular key words imply entailment, irrespective of context, and visual question answering models learn to predict prototypical answers, without considering evidence in the image. In this paper, we show that if we have prior knowledge of such biases, we can train a model to be more robust to domain shift. Our method has two stages: we (1) train a naive model that makes predictions exclusively based on dataset biases, and (2) train a robust model as part of an ensemble with the naive one in order to encourage it to focus on other patterns in the data that are more likely to generalize. Experiments on five datasets with out-of-domain test sets show significantly improved robustness in all settings, including a 12 point gain on a changing priors visual question answering dataset and a 9 point gain on an adversarial question answering test set.
Outlier Detection in High Dimensional Data
Authors: Firuz Kamalov, Ho Hon Leung
Date: 2019-09-09 00:00:00
Tasks: Density Estimation, Outlier Detection, Vocal Bursts Intensity Prediction

High-dimensional data poses unique challenges in outlier detection process. Most of the existing algorithms fail to properly address the issues stemming from a large number of features. In particular, outlier detection algorithms perform poorly on data set of small size with a large number of features. In this paper, we propose a novel outlier detection algorithm based on principal component analysis and kernel density estimation. The proposed method is designed to address the challenges of dealing with high-dimensional data by projecting the original data onto a smaller space and using the innate structure of the data to calculate anomaly scores for each data point. Numerical experiments on synthetic and real-life data show that our method performs well on high-dimensional data. In particular, the proposed method outperforms the benchmark methods as measured by the $F_1$-score. Our method also produces better-than-average execution times compared to the benchmark methods.
Machine Learning Approach for Air Shower Recognition in EUSO-SPB Data
Authors: Michal Vrábel, Ján Genči, Pavol Bobik, Francesca Bisconti
Date: 2019-09-09 00:00:00
Tasks: BIG-bench Machine Learning

The main goal of The Extreme Universe Space Observatory on a Super Pressure Balloon (EUSO-SPB1) was to observe from above extensive air showers caused by ultra-high energy cosmic rays. EUSO-SPB1 uses a fluorescence detector that observes the atmosphere in a nadir observation mode from a near space altitude. During the 12-day flight, an onboard first level trigger detected more than \num{175000} candidate events. This paper presents an approach to recognize air showers in this dataset. The approach uses a feature extraction method to create a simpler representation of an event and then it uses established machine learning techniques to classify data into at least two classes - shower and noise. The machine learning models are trained on a set of air shower simulations put on top of the background observed during the flight and a set of events from the flight. We present the efficiency of the method on datasets of simulated events. The flight data events are also used in unsupervised learning methods to identify groups of events with similar features. The presented methods allow us to shorten the candidate events list and, thanks to the groups of similar events identified by the unsupervised methods, the classification of the triggered events is made simpler.
Learning Task-Specific Generalized Convolutions in the Permutohedral Lattice
Authors: Anne S. Wannenwetsch, Martin Kiefel, Peter V. Gehler, Stefan Roth
Date: 2019-09-09 00:00:00
Tasks: Decoder, Optical Flow Estimation, Semantic Segmentation

Dense prediction tasks typically employ encoder-decoder architectures, but the prevalent convolutions in the decoder are not image-adaptive and can lead to boundary artifacts. Different generalized convolution operations have been introduced to counteract this. We go beyond these by leveraging guidance data to redefine their inherent notion of proximity. Our proposed network layer builds on the permutohedral lattice, which performs sparse convolutions in a high-dimensional space allowing for powerful non-local operations despite small filters. Multiple features with different characteristics span this permutohedral space. In contrast to prior work, we learn these features in a task-specific manner by generalizing the basic permutohedral operations to learnt feature representations. As the resulting objective is complex, a carefully designed framework and learning procedure are introduced, yielding rich feature embeddings in practice. We demonstrate the general applicability of our approach in different joint upsampling tasks. When adding our network layer to state-of-the-art networks for optical flow and semantic segmentation, boundary artifacts are removed and the accuracy is improved.