pwc-archive/papers-with-abstracts huggingface dataset:
Deep Audio Prior
Authors: Yapeng Tian, Chenliang Xu, DIngzeyu Li
Date: 2019-12-21 00:00:00
Tasks: blind source separation, Texture Synthesis

Deep convolutional neural networks are known to specialize in distilling compact and robust prior from a large amount of data. We are interested in applying deep networks in the absence of training dataset. In this paper, we introduce deep audio prior (DAP) which leverages the structure of a network and the temporal information in a single audio file. Specifically, we demonstrate that a randomly-initialized neural network can be used with carefully designed audio prior to tackle challenging audio problems such as universal blind source separation, interactive audio editing, audio texture synthesis, and audio co-separation. To understand the robustness of the deep audio prior, we construct a benchmark dataset \emph{Universal-150} for universal sound source separation with a diverse set of sources. We show superior audio results than previous work on both qualitative and quantitative evaluations. We also perform thorough ablation study to validate our design choices.
Predicting Heart Failure Readmission from Clinical Notes Using Deep Learning
Authors: Xiong Liu, Yu Chen, Jay Bae, Hu Li, Joseph Johnston, Todd Sanger
Date: 2019-12-21 00:00:00
Tasks: Deep Learning, Prediction, Readmission Prediction

Heart failure hospitalization is a severe burden on healthcare. How to predict and therefore prevent readmission has been a significant challenge in outcomes research. To address this, we propose a deep learning approach to predict readmission from clinical notes. Unlike conventional methods that use structured data for prediction, we leverage the unstructured clinical notes to train deep learning models based on convolutional neural networks (CNN). We then use the trained models to classify and predict potentially high-risk admissions/patients. For evaluation, we trained CNNs using the discharge summary notes in the MIMIC III database. We also trained regular machine learning models based on random forest using the same datasets. The result shows that deep learning models outperform the regular models in prediction tasks. CNN method achieves a F1 score of 0.756 in general readmission prediction and 0.733 in 30-day readmission prediction, while random forest only achieves a F1 score of 0.674 and 0.656 respectively. We also propose a chi-square test based method to interpret key features associated with deep learning predicted readmissions. It reveals clinical insights about readmission embedded in the clinical notes. Collectively, our method can make the human evaluation process more efficient and potentially facilitate the reduction of readmission rates.
Recurrent Hierarchical Topic-Guided RNN for Language Generation
Authors: Dandan Guo, Bo Chen, Ruiying Lu, Mingyuan Zhou
Date: 2019-12-21 00:00:00
Proceeding: ICML 2020 1
Tasks: Language Modeling, Language Modelling, Sentence, Text Generation

To simultaneously capture syntax and global semantics from a text corpus, we propose a new larger-context recurrent neural network (RNN) based language model, which extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation. Moving beyond a conventional RNN-based language model that ignores long-range word dependencies and sentence order, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence topic dependencies. For inference, we develop a hybrid of stochastic-gradient Markov chain Monte Carlo and recurrent autoencoding variational Bayes. Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms larger-context RNN-based language models, but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.
Tifinagh-IRCAM Handwritten character recognition using Deep learning
Authors: El Wardani Dadi
Date: 2019-12-21 00:00:00
Tasks: Deep Learning

In this paper, we exploit the benefits of the deep learning approach to design an efficient system of Amazigh handwritten recognition. Indeed, this approach has proved a greater efficiency in the various domains, especially recognition tasks. However, to take full advantage of this approach it's necessary to construct an adequate dataset of training and testing that represent faithfully the concerned problem. To this end, we have prepared our dataset of 102 writers each one contains 33 characters of IRCAM-Tifinagh. Inspired by the MNIST database, the set of characters is size-normalized and centered in a fixed-size image. The resulting is a grey level image of size 28x28, where the black color is the non-color of the character. The number of images produced after this preprocessing step is 3,366.
Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control
Authors: Julian Nubert, Johannes Köhler, Vincent Berenz, Frank Allgöwer, Sebastian Trimpe
Date: 2019-12-22 00:00:00
Tasks: Model Predictive Control

Fast feedback control and safety guarantees are essential in modern robotics. We present an approach that achieves both by combining novel robust model predictive control (MPC) with function approximation via (deep) neural networks (NNs). The result is a new approach for complex tasks with nonlinear, uncertain, and constrained dynamics as are common in robotics. Specifically, we leverage recent results in MPC research to propose a new robust setpoint tracking MPC algorithm, which achieves reliable and safe tracking of a dynamic setpoint while guaranteeing stability and constraint satisfaction. The presented robust MPC scheme constitutes a one-layer approach that unifies the often separated planning and control layers, by directly computing the control command based on a reference and possibly obstacle positions. As a separate contribution, we show how the computation time of the MPC can be drastically reduced by approximating the MPC law with a NN controller. The NN is trained and validated from offline samples of the MPC, yielding statistical guarantees, and used in lieu thereof at run time. Our experiments on a state-of-the-art robot manipulator are the first to show that both the proposed robust and approximate MPC schemes scale to real-world robotic systems.
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack
Authors: Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, Bo Li
Date: 2019-12-22 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Adversarial Attack, Adversarial Text, Decoder, Question Answering, Sentence, Sentiment Analysis, Text Generation

Adversarial attacks against natural language processing systems, which perform seemingly innocuous modifications to inputs, can induce arbitrary mistakes to the target models. Though raised great concerns, such adversarial attacks can be leveraged to estimate the robustness of NLP models. Compared with the adversarial example generation in continuous data domain (e.g., image), generating adversarial text that preserves the original meaning is challenging since the text space is discrete and non-differentiable. To handle these challenges, we propose a target-controllable adversarial attack framework T3, which is applicable to a range of NLP tasks. In particular, we propose a tree-based autoencoder to embed the discrete text data into a continuous representation space, upon which we optimize the adversarial perturbation. A novel tree-based decoder is then applied to regularize the syntactic correctness of the generated text and manipulate it on either sentence (T3(Sent)) or word (T3(Word)) level. We consider two most representative NLP tasks: sentiment analysis and question answering (QA). Extensive experimental results and human studies show that T3 generated adversarial texts can successfully manipulate the NLP models to output the targeted incorrect answer without misleading the human. Moreover, we show that the generated adversarial texts have high transferability which enables the black-box attacks in practice. Our work sheds light on an effective and general way to examine the robustness of NLP models. Our code is publicly available at https://github.com/AI-secure/T3/.
Adversarial Cross-Domain Action Recognition with Co-Attention
Authors: Boxiao Pan, Zhangjie Cao, Ehsan Adeli, Juan Carlos Niebles
Date: 2019-12-22 00:00:00
Tasks: Action Recognition

Action recognition has been a widely studied topic with a heavy focus on supervised learning involving sufficient labeled videos. However, the problem of cross-domain action recognition, where training and testing videos are drawn from different underlying distributions, remains largely under-explored. Previous methods directly employ techniques for cross-domain image recognition, which tend to suffer from the severe temporal misalignment problem. This paper proposes a Temporal Co-attention Network (TCoN), which matches the distributions of temporally aligned action features between source and target domains using a novel cross-domain co-attention mechanism. Experimental results on three cross-domain action recognition datasets demonstrate that TCoN improves both previous single-domain and cross-domain methods significantly under the cross-domain setting.
Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings
Authors: Andreas Hanselowski, Iryna Gurevych
Date: 2019-12-17 00:00:00
Tasks: Word Embeddings

Word embeddings are rich word representations, which in combination with deep neural networks, lead to large performance gains for many NLP tasks. However, word embeddings are represented by dense, real-valued vectors and they are therefore not directly interpretable. Thus, computational operations based on them are also not well understood. In this paper, we present an approach for analyzing structures in the semantic vector space to get a better understanding of the underlying semantic encoding principles. We present a framework for decomposing word embeddings into smaller meaningful units which we call sub-vectors. The framework opens up a wide range of possibilities analyzing phenomena in vector space semantics, as well as solving concrete NLP problems: We introduce the category completion task and show that a sub-vector based approach is superior to supervised techniques; We present a sub-vector based method for solving the word analogy task, which substantially outperforms different variants of the traditional vector-offset method.
BERTQA -- Attention on Steroids
Authors: Ankit Chadha, Rewa Sood
Date: 2019-12-14 00:00:00

In this work, we extend the Bidirectional Encoder Representations from Transformers (BERT) with an emphasis on directed coattention to obtain an improved F1 performance on the SQUAD2.0 dataset. The Transformer architecture on which BERT is based places hierarchical global attention on the concatenation of the context and query. Our additions to the BERT architecture augment this attention with a more focused context to query (C2Q) and query to context (Q2C) attention via a set of modified Transformer encoder units. In addition, we explore adding convolution-based feature extraction within the coattention architecture to add localized information to self-attention. We found that coattention significantly improves the no answer F1 by 4 points in the base and 1 point in the large architecture. After adding skip connections the no answer F1 improved further without causing an additional loss in has answer F1. The addition of localized feature extraction added to attention produced an overall dev F1 of 77.03 in the base architecture. We applied our findings to the large BERT model which contains twice as many layers and further used our own augmented version of the SQUAD 2.0 dataset created by back translation, which we have named SQUAD 2.Q. Finally, we performed hyperparameter tuning and ensembled our best models for a final F1/EM of 82.317/79.442 (Attention on Steroids, PCE Test Leaderboard).
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks
Authors: Angelos Filos, Sebastian Farquhar, Aidan N. Gomez, Tim G. J. Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud de Kroon, Yarin Gal
Date: 2019-12-22 00:00:00
Tasks: Out-of-Distribution Detection

Evaluation of Bayesian deep learning (BDL) methods is challenging. We often seek to evaluate the methods' robustness and scalability, assessing whether new tools give `better' uncertainty estimates than old ones. These evaluations are paramount for practitioners when choosing BDL tools on-top of which they build their applications. Current popular evaluations of BDL methods, such as the UCI experiments, are lacking: Methods that excel with these experiments often fail when used in application such as medical or automotive, suggesting a pertinent need for new benchmarks in the field. We propose a new BDL benchmark with a diverse set of tasks, inspired by a real-world medical imaging application on \emph{diabetic retinopathy diagnosis}. Visual inputs (512x512 RGB images of retinas) are considered, where model uncertainty is used for medical pre-screening---i.e. to refer patients to an expert when model diagnosis is uncertain. Methods are then ranked according to metrics derived from expert-domain to reflect real-world use of model uncertainty in automated diagnosis. We develop multiple tasks that fall under this application, including out-of-distribution detection and robustness to distribution shift. We then perform a systematic comparison of well-tuned BDL techniques on the various tasks. From our comparison we conclude that some current techniques which solve benchmarks such as UCI `overfit' their uncertainty to the dataset---when evaluated on our benchmark these underperform in comparison to simpler baselines. The code for the benchmark, its baselines, and a simple API for evaluating new BDL tools are made available at https://github.com/oatml/bdl-benchmarks.
Energy-Aware Multi-Server Mobile Edge Computing: A Deep Reinforcement Learning Approach
Authors: Navid Naderializadeh, Morteza Hashemi
Date: 2019-12-22 00:00:00
Tasks: Deep Reinforcement Learning, Edge-computing, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We investigate the problem of computation offloading in a mobile edge computing architecture, where multiple energy-constrained users compete to offload their computational tasks to multiple servers through a shared wireless medium. We propose a multi-agent deep reinforcement learning algorithm, where each server is equipped with an agent, observing the status of its associated users and selecting the best user for offloading at each step. We consider computation time (i.e., task completion time) and system lifetime as two key performance indicators, and we numerically demonstrate that our approach outperforms baseline algorithms in terms of the trade-off between computation time and system lifetime.
Rapid Whole-Heart CMR with Single Volume Super-resolution
Authors: Jennifer A. Steeden, Michael Quail, Alexander Gotschy, Andreas Hauptmann, Simon Arridge, Rodney Jones, Vivek Muthurangu
Date: 2019-12-22 00:00:00
Tasks: Anatomy, Super-Resolution

Background: Three-dimensional, whole heart, balanced steady state free precession (WH-bSSFP) sequences provide delineation of intra-cardiac and vascular anatomy. However, they have long acquisition times. Here, we propose significant speed ups using a deep learning single volume super resolution reconstruction, to recover high resolution features from rapidly acquired low resolution WH-bSSFP images. Methods: A 3D residual U-Net was trained using synthetic data, created from a library of high-resolution WH-bSSFP images by simulating 0.5 slice resolution and 0.5 phase resolution. The trained network was validated with synthetic test data, as well as prospective low-resolution data. Results: Synthetic low-resolution data had significantly better image quality after super-resolution reconstruction. Qualitative image scores showed super-resolved images had better edge sharpness, fewer residual artefacts and less image distortion than low-resolution images, with similar scores to high-resolution data. Quantitative image scores showed super-resolved images had significantly better edge sharpness than low-resolution or high-resolution images, with significantly better signal-to-noise ratio than high-resolution data. Vessel diameters measurements showed over-estimation in the low-resolution measurements, compared to the high-resolution data. No significant differences and no bias was found in the super-resolution measurements. Conclusion: This paper demonstrates the potential of using a residual U-Net for super-resolution reconstruction of rapidly acquired low-resolution whole heart bSSFP data within a clinical setting. The resulting network can be applied very quickly, making these techniques particularly appealing within busy clinical workflow. Thus, we believe that this technique may help speed up whole heart CMR in clinical practice.
Emotion Recognition Using Wearables: A Systematic Literature Review Work in progress
Authors: Stanisław Saganowski, Anna Dutkowiak, Adam Dziadek, Maciej Dzieżyc, Joanna Komoszyńska, Weronika Michalska, Adam Polak, Michał Ujma, Przemysław Kazienko
Date: 2019-12-22 00:00:00
Tasks: Emotion Recognition, Systematic Literature Review

Wearables like smartwatches or wrist bands equipped with pervasive sensors enable us to monitor our physiological signals. In this study, we address the question whether they can help us to recognize our emotions in our everyday life for ubiquitous computing. Using the systematic literature review, we identified crucial research steps and discussed the main limitations and problems in the domain.
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing
Authors: Vishal Monga, Yuelong Li, Yonina C. Eldar
Date: 2019-12-22 00:00:00
Tasks: Rolling Shutter Correction

Deep neural networks provide unprecedented performance gains in many real world problems in signal and image processing. Despite these gains, future development and practical deployment of deep networks is hindered by their blackbox nature, i.e., lack of interpretability, and by the need for very large training sets. An emerging technique called algorithm unrolling or unfolding offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are used widely in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention and is rapidly growing both in theoretic investigations and practical applications. The growing popularity of unrolled deep networks is due in part to their potential in developing efficient, high-performance and yet interpretable network architectures from reasonable size training sets. In this article, we review algorithm unrolling for signal and image processing. We extensively cover popular techniques for algorithm unrolling in various domains of signal and image processing including imaging, vision and recognition, and speech processing. By reviewing previous works, we reveal the connections between iterative algorithms and neural networks and present recent theoretical results. Finally, we provide a discussion on current limitations of unrolling and suggest possible future research directions.
Teaching Responsible Data Science: Charting New Pedagogical Territory
Authors: Julia Stoyanovich, Armanda Lewis
Date: 2019-12-23 00:00:00
Tasks: Decision Making, Ethics

Although numerous ethics courses are available, with many focusing specifically on technology and computer ethics, pedagogical approaches employed in these courses rely exclusively on texts rather than on software development or data analysis. Technical students often consider these courses unimportant and a distraction from the "real" material. To develop instructional materials and methodologies that are thoughtful and engaging, we must strive for balance: between texts and coding, between critique and solution, and between cutting-edge research and practical applicability. Finding such balance is particularly difficult in the nascent field of responsible data science (RDS), where we are only starting to understand how to interface between the intrinsically different methodologies of engineering and social sciences. In this paper we recount a recent experience in developing and teaching an RDS course to graduate and advanced undergraduate students in data science. We then dive into an area that is critically important to RDS -- transparency and interpretability of machine-assisted decision-making, and tie this area to the needs of emerging RDS curricula. Recounting our own experience, and leveraging literature on pedagogical methods in data science and beyond, we propose the notion of an "object-to-interpret-with". We link this notion to "nutritional labels" -- a family of interpretability tools that are gaining popularity in RDS research and practice. With this work we aim to contribute to the nascent area of RDS education, and to inspire others in the community to come together to develop a deeper theoretical understanding of the pedagogical needs of RDS, and contribute concrete educational materials and methodologies that others can use. All course materials are publicly available at https://dataresponsibly.github.io/courses.
Multilevel Monte Carlo estimation of log marginal likelihood
Authors: Takashi Goda, Kei Ishikawa
Date: 2019-12-23 00:00:00

In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
A Compared Study Between Some Subspace Based Algorithms
Authors: Xing Liu, Xiao-Jun Wu, Zhen Liu, He-Feng Yin
Date: 2019-12-23 00:00:00
Tasks: Face Recognition

The technology of face recognition has made some progress in recent years. After studying the PCA, 2DPCA, R1-PCA, L1-PCA, KPCA and KECA algorithms, in this paper ECA (2DECA) is proposed by extracting features in PCA (2DPCA) based on Renyi entropy contribution. And then we conduct a study on the 2DL1-PCA and 2DR1-PCA algorithms. On the basis of the experiments, this paper compares the difference of the recognition accuracy and operational efficiency between the above algorithms.
Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time
Authors: Jeongho Kim, Insoon Yang
Date: 2019-12-23 00:00:00
Proceeding: L4DC 2020 6
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we introduce Hamilton-Jacobi-Bellman (HJB) equations for Q-functions in continuous time optimal control problems with Lipschitz continuous controls. The standard Q-function used in reinforcement learning is shown to be the unique viscosity solution of the HJB equation. A necessary and sufficient condition for optimality is provided using the viscosity solution framework. By using the HJB equation, we develop a Q-learning method for continuous-time dynamical systems. A DQN-like algorithm is also proposed for high-dimensional state and control spaces. The performance of the proposed Q-learning algorithm is demonstrated using 1-, 10- and 20-dimensional dynamical systems.