pwc-archive/papers-with-abstracts huggingface dataset:
Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
Authors: Erik-Jan van Kesteren, Chang Sun, Daniel L. Oberski, Michel Dumontier, Lianne Ippel
Date: 2019-11-08 00:00:00
Tasks: Privacy Preserving

Combining data from varied sources has considerable potential for knowledge discovery: collaborating data parties can mine data in an expanded feature space, allowing them to explore a larger range of scientific questions. However, data sharing among different parties is highly restricted by legal conditions, ethical concerns, and / or data volume. Fueled by these concerns, the fields of cryptography and distributed learning have made great progress towards privacy-preserving and distributed data mining. However, practical implementations have been hampered by the limited scope or computational complexity of these methods. In this paper, we greatly extend the range of analyses available for vertically partitioned data, i.e., data collected by separate parties with different features on the same subjects. To this end, we present a novel approach for privacy-preserving generalized linear models, a fundamental and powerful framework underlying many prediction and classification procedures. We base our method on a distributed block coordinate descent algorithm to obtain parameter estimates, and we develop an extension to compute accurate standard errors without additional communication cost. We critically evaluate the information transfer for semi-honest collaborators and show that our protocol is secure against data reconstruction. Through both simulated and real-world examples we illustrate the functionality of our proposed algorithm. Without leaking information, our method performs as well on vertically partitioned data as existing methods on combined data -- all within mere minutes of computation time. We conclude that our method is a viable approach for vertically partitioned data analysis with a wide range of real-world applications.
Lipschitz Constrained Parameter Initialization for Deep Transformers
Authors: Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Jingyi Zhang
Date: 2019-11-08 00:00:00
Proceeding: ACL 2020 6
Tasks: Decoder, Translation

The Transformer translation model employs residual connection and layer normalization to ease the optimization difficulties caused by its multi-layer encoder/decoder structure. Previous research shows that even with residual connection and layer normalization, deep Transformers still have difficulty in training, and particularly Transformer models with more than 12 encoder/decoder layers fail to converge. In this paper, we first empirically demonstrate that a simple modification made in the official implementation, which changes the computation order of residual connection and layer normalization, can significantly ease the optimization of deep Transformers. We then compare the subtle differences in computation order in considerable detail, and present a parameter initialization method that leverages the Lipschitz constraint on the initialization of Transformer parameters that effectively ensures training convergence. In contrast to findings in previous research we further demonstrate that with Lipschitz parameter initialization, deep Transformers with the original computation order can converge, and obtain significant BLEU improvements with up to 24 layers. In contrast to previous research which focuses on deep encoders, our approach additionally enables Transformers to also benefit from deep decoders.
Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates
Authors: Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, Alfons Juan
Date: 2019-11-08 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, speech-recognition, Speech Recognition, Speech-to-Text, Speech-to-Text Translation, Translation

Current research into spoken language translation (SLT),or speech-to-text translation, is often hampered by the lack of specific data resources for this task, as currently available SLT datasets are restricted to a limited set of language pairs. In this paper we present Europarl-ST, a novel multilingual SLT corpus containing paired audio-text samples for SLT from and into 6 European languages, for a total of 30 different translation directions. This corpus has been compiled using the debates held in the European Parliament in the period between 2008 and 2012. This paper describes the corpus creation process and presents a series of automatic speech recognition, machine translation and spoken language translation experiments that highlight the potential of this new resource. The corpus is released under a Creative Commons license and is freely accessible and downloadable.
Building Segmentation through a Gated Graph Convolutional Neural Network with Deep Structured Feature Embedding
Authors: Yilei Shi, Qingyu Li, Xiao Xiang Zhu
Date: 2019-11-08 00:00:00
Tasks: General Classification, Segmentation, Semantic Segmentation

Automatic building extraction from optical imagery remains a challenge due to, for example, the complexity of building shapes. Semantic segmentation is an efficient approach for this task. The latest development in deep convolutional neural networks (DCNNs) has made accurate pixel-level classification tasks possible. Yet one central issue remains: the precise delineation of boundaries. Deep architectures generally fail to produce fine-grained segmentation with accurate boundaries due to their progressive down-sampling. Hence, we introduce a generic framework to overcome the issue, integrating the graph convolutional network (GCN) and deep structured feature embedding (DSFE) into an end-to-end workflow. Furthermore, instead of using a classic graph convolutional neural network, we propose a gated graph convolutional network, which enables the refinement of weak and coarse semantic predictions to generate sharp borders and fine-grained pixel-level classification. Taking the semantic segmentation of building footprints as a practical example, we compared different feature embedding architectures and graph neural networks. Our proposed framework with the new GCN architecture outperforms state-of-the-art approaches. Although our main task in this work is building footprint extraction, the proposed method can be generally applied to other binary or multi-label segmentation tasks.
A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation
Authors: Yun Chen, Liangyou Li, Xin Jiang, Xiao Chen, Qun Liu
Date: 2019-11-08 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Machine Translation, NMT, Sentence, Translation

Despite the success of neural machine translation (NMT), simultaneous neural machine translation (SNMT), the task of translating in real time before a full sentence has been observed, remains challenging due to the syntactic structure difference and simultaneity requirements. In this paper, we propose a general framework for adapting neural machine translation to translate simultaneously. Our framework contains two parts: prefix translation that utilizes a consecutive NMT model to translate source prefixes and a stopping criterion that determines when to stop the prefix translation. Experiments on three translation corpora and two language pairs show the efficacy of the proposed framework on balancing the quality and latency in adapting NMT to perform simultaneous translation.
Quality Aware Generative Adversarial Networks
Authors: Parimala Kancharla, Sumohana S. Channappayya
Date: 2019-11-08 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Full reference image quality assessment, Full-Reference Image Quality Assessment, Image Generation, Image Quality Assessment, No-Reference Image Quality Assessment, SSIM

Generative Adversarial Networks (GANs) have become a very popular tool for implicitly learning high-dimensional probability distributions. Several improvements have been made to the original GAN formulation to address some of its shortcomings like mode collapse, convergence issues, entanglement, poor visual quality etc. While a significant effort has been directed towards improving the visual quality of images generated by GANs, it is rather surprising that objective image quality metrics have neither been employed as cost functions nor as regularizers in GAN objective functions. In this work, we show how a distance metric that is a variant of the Structural SIMilarity (SSIM) index (a popular full-reference image quality assessment algorithm), and a novel quality aware discriminator gradient penalty function that is inspired by the Natural Image Quality Evaluator (NIQE, a popular no-reference image quality assessment algorithm) can each be used as excellent regularizers for GAN objective functions. Specifically, we demonstrate state-of-the-art performance using the Wasserstein GAN gradient penalty (WGAN-GP) framework over CIFAR-10, STL10 and CelebA datasets.
AI Aided Noise Processing of Spintronic Based IoT Sensor for Magnetocardiography Application
Authors: Attayeb Mohsen, Muftah Al-Mahdawi, Mostafa M. Fouda, Mikihiko Oogane, Yasuo Ando, Zubair Md Fadlullah
Date: 2019-11-08 00:00:00
Tasks: Electrocardiography (ECG)

As we are about to embark upon the highly hyped "Society 5.0", powered by the Internet of Things (IoT), traditional ways to monitor human heart signals for tracking cardio-vascular conditions are challenging, particularly in remote healthcare settings. On the merits of low power consumption, portability, and non-intrusiveness, there are no suitable IoT solutions that can provide information comparable to the conventional Electrocardiography (ECG). In this paper, we propose an IoT device utilizing a spintronic ultra-sensitive sensor that measures the magnetic fields produced by cardio-vascular electrical activity, i.e. Magentocardiography (MCG). After that, we treat the low-frequency noise generated by the sensors, which is also a challenge for most other sensors dealing with low-frequency bio-magnetic signals. Instead of relying on generic signal processing techniques such as averaging or filtering, we employ deep-learning training on bio-magnetic signals. Using an existing dataset of ECG records, MCG labels are synthetically constructed. A unique deep learning structure composed of combined Convolutional Neural Network (CNN) with Gated Recurrent Unit (GRU) is trained using the labeled data moving through a striding window, which is able to smartly capture and eliminate the noise features. Simulation results are reported to evaluate the effectiveness of the proposed method that demonstrates encouraging performance.
A Cooperative Coordination Solver for Travelling Thief Problems
Authors: Majid Namazi, Conrad Sanderson, M. A. Hakim Newton, Abdul Sattar
Date: 2019-11-08 00:00:00

The travelling thief problem (TTP) is a representative of multi-component optimisation problems with interacting components. TTP combines the knapsack problem (KP) and the travelling salesman problem (TSP). A thief performs a cyclic tour through a set of cities, and pursuant to a collection plan, collects a subset of items into a rented knapsack with finite capacity. The aim is to maximise profit while minimising renting cost. Existing TTP solvers typically solve the KP and TSP components in an interleaved manner: the solution of one component is kept fixed while the solution of the other component is modified. This suggests low coordination between solving the two components, possibly leading to low quality TTP solutions. The 2-OPT heuristic is often used for solving the TSP component, which reverses a segment in the tour. Within TTP, 2-OPT does not take into account the collection plan, which can result in a lower objective value. This in turn can result in the tour modification to be rejected by a solver. We propose an expanded form of 2-OPT to change the collection plan in coordination with tour modification. Items regarded as less profitable and collected in cities located earlier in the reversed segment are substituted by items that tend to be more profitable and not collected in cities located later in the reversed segment. The collection plan is further changed through a modified form of the hill-climbing bit-flip search, where changes in the collection state are only permitted for boundary items, which are defined as lowest profitable collected items or highest profitable uncollected items. This restriction reduces the time spent on the KP component, allowing more tours to be evaluated by the TSP component within a time budget. The proposed approaches form the basis of a new cooperative coordination solver, which is shown to outperform several state-of-the-art TTP solvers.
iSarcasm: A Dataset of Intended Sarcasm
Authors: Silviu Oprea, Walid Magdy
Date: 2019-11-08 00:00:00
Proceeding: ACL 2020 6
Tasks: Sarcasm Detection

We consider the distinction between intended and perceived sarcasm in the context of textual sarcasm detection. The former occurs when an utterance is sarcastic from the perspective of its author, while the latter occurs when the utterance is interpreted as sarcastic by the audience. We show the limitations of previous labelling methods in capturing intended sarcasm and introduce the iSarcasm dataset of tweets labeled for sarcasm directly by their authors. Examining the state-of-the-art sarcasm detection models on our dataset showed low performance compared to previously studied datasets, which indicates that these datasets might be biased or obvious and sarcasm could be a phenomenon under-studied computationally thus far. By providing the iSarcasm dataset, we aim to encourage future NLP research to develop methods for detecting sarcasm in text as intended by the authors of the text, not as labeled under assumptions that we demonstrate to be sub-optimal.
Not Enough Data? Deep Learning to the Rescue!
Authors: Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, Naama Zwerdling
Date: 2019-11-08 00:00:00
Tasks: Data Augmentation, Deep Learning, General Classification, LAMBADA, Language Modeling, Language Modelling, text-classification, Text Classification, Text Generation

Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.
Ground Metric Learning on Graphs
Authors: Matthieu Heitz, Nicolas Bonneel, David Coeurjolly, Marco Cuturi, Gabriel Peyré
Date: 2019-11-08 00:00:00
Tasks: Metric Learning

Optimal transport (OT) distances between probability distributions are parameterized by the ground metric they use between observations. Their relevance for real-life applications strongly hinges on whether that ground metric parameter is suitably chosen. Selecting it adaptively and algorithmically from prior knowledge, the so-called ground metric learning GML) problem, has therefore appeared in various settings. We consider it in this paper when the learned metric is constrained to be a geodesic distance on a graph that supports the measures of interest. This imposes a rich structure for candidate metrics, but also enables far more efficient learning procedures when compared to a direct optimization over the space of all metric matrices. We use this setting to tackle an inverse problem stemming from the observation of a density evolving with time: we seek a graph ground metric such that the OT interpolation between the starting and ending densities that result from that ground metric agrees with the observed evolution. This OT dynamic framework is relevant to model natural phenomena exhibiting displacements of mass, such as for instance the evolution of the color palette induced by the modification of lighting and materials.
Accurate Vision-based Manipulation through Contact Reasoning
Authors: Alina Kloss, Maria Bauza, Jiajun Wu, Joshua B. Tenenbaum, Alberto Rodriguez, Jeannette Bohg
Date: 2019-11-08 00:00:00
Tasks: State Estimation

Planning contact interactions is one of the core challenges of many robotic tasks. Optimizing contact locations while taking dynamics into account is computationally costly and, in environments that are only partially observable, executing contact-based tasks often suffers from low accuracy. We present an approach that addresses these two challenges for the problem of vision-based manipulation. First, we propose to disentangle contact from motion optimization. Thereby, we improve planning efficiency by focusing computation on promising contact locations. Second, we use a hybrid approach for perception and state estimation that combines neural networks with a physically meaningful state representation. In simulation and real-world experiments on the task of planar pushing, we show that our method is more efficient and achieves a higher manipulation accuracy than previous vision-based approaches.
Pretrained Language Models for Document-Level Neural Machine Translation
Authors: Liangyou Li, Xin Jiang, Qun Liu
Date: 2019-11-08 00:00:00
Tasks: Machine Translation, NMT, Sentence, Translation

Previous work on document-level NMT usually focuses on limited contexts because of degraded performance on larger contexts. In this paper, we investigate on using large contexts with three main contributions: (1) Different from previous work which pertrained models on large-scale sentence-level parallel corpora, we use pretrained language models, specifically BERT, which are trained on monolingual documents; (2) We propose context manipulation methods to control the influence of large contexts, which lead to comparable results on systems using small and large contexts; (3) We introduce a multi-task training for regularization to avoid models overfitting our training corpora, which further improves our systems together with a deeper encoder. Experiments are conducted on the widely used IWSLT data sets with three language pairs, i.e., Chinese--English, French--English and Spanish--English. Results show that our systems are significantly better than three previously reported document-level systems.
Domain Robustness in Neural Machine Translation
Authors: Mathias Müller, Annette Rios, Rico Sennrich
Date: 2019-11-08 00:00:00
Proceeding: AMTA 2020 10
Tasks: Machine Translation, NMT, Translation

Translating text that diverges from the training domain is a key challenge for machine translation. Domain robustness---the generalization of models to unseen test domains---is low for both statistical (SMT) and neural machine translation (NMT). In this paper, we study the performance of SMT and NMT models on out-of-domain test sets. We find that in unknown domains, SMT and NMT suffer from very different problems: SMT systems are mostly adequate but not fluent, while NMT systems are mostly fluent, but not adequate. For NMT, we identify such hallucinations (translations that are fluent but unrelated to the source) as a key reason for low domain robustness. To mitigate this problem, we empirically compare methods that are reported to improve adequacy or in-domain robustness in terms of their effectiveness at improving domain robustness. In experiments on German to English OPUS data, and German to Romansh (a low-resource setting) we find that several methods improve domain robustness. While those methods do lead to higher BLEU scores overall, they only slightly increase the adequacy of translations compared to SMT.
Unified Sample-Optimal Property Estimation in Near-Linear Time
Authors: Yi Hao, Alon Orlitsky
Date: 2019-11-08 00:00:00
Proceeding: NeurIPS 2019 12

We consider the fundamental learning problem of estimating properties of distributions over large domains. Using a novel piecewise-polynomial approximation technique, we derive the first unified methodology for constructing sample- and time-efficient estimators for all sufficiently smooth, symmetric and non-symmetric, additive properties. This technique yields near-linear-time computable estimators whose approximation values are asymptotically optimal and highly-concentrated, resulting in the first: 1) estimators achieving the $\mathcal{O}(k/(\varepsilon^2\log k))$ min-max $\varepsilon$-error sample complexity for all $k$-symbol Lipschitz properties; 2) unified near-optimal differentially private estimators for a variety of properties; 3) unified estimator achieving optimal bias and near-optimal variance for five important properties; 4) near-optimal sample-complexity estimators for several important symmetric properties over both domain sizes and confidence levels. In addition, we establish a McDiarmid's inequality under Poisson sampling, which is of independent interest.
Extracting temporal features into a spatial domain using autoencoders for sperm video analysis
Authors: Vajira Thambawita, Pål Halvorsen, Hugo Hammer, Michael Riegler, Trine B. Haugen
Date: 2019-11-08 00:00:00
Tasks: Deep Learning, Transfer Learning

In this paper, we present a two-step deep learning method that is used to predict sperm motility and morphology-based on video recordings of human spermatozoa. First, we use an autoencoder to extract temporal features from a given semen video and plot these into image-space, which we call feature-images. Second, these feature-images are used to perform transfer learning to predict the motility and morphology values of human sperm. The presented method shows it's capability to extract temporal information into spatial domain feature-images which can be used with traditional convolutional neural networks. Furthermore, the accuracy of the predicted motility of a given semen sample shows that a deep learning-based model can capture the temporal information of microscopic recordings of human semen.
Non-parametric Probabilistic Load Flow using Gaussian Process Learning
Authors: Parikshit Pareek, Chuan Wang, Hung D. Nguyen
Date: 2019-11-08 00:00:00

In this work, we propose a non-parametric probabilistic load flow (NP-PLF) technique based on the Gaussian Process (GP) learning to understand the power system behavior under uncertainty for better operational decisions. The technique can provide "semi-explicit" power flow solutions by implementing the learning and testing steps which map control variables to inputs. The proposed NP-PLF leverages upon GP upper confidence bound (GP-UCB) sampling algorithm. The salient features of this NP-PLF method are: i) applicable for power flow problem having power injection uncertainty with an unknown class of distribution; ii) providing probabilistic learning bound (PLB) which further provides control over the error and convergence; iii) capable of handling intermittent distributed generation as well as load uncertainties, and iv) applicable to both balanced and unbalanced power flow with different type and size of power systems. The simulation results performed on the IEEE 30-bus and IEEE 118-bus system show that the proposed method can learn the voltage function over the power injection subspace using a small number of training samples. Further, the testing with different input uncertainty distributions indicates that complete statistical information can be obtained for the probabilistic load flow problem with average percentage relative error of order $10^{-3}$\% on 50000 test points.