pwc-archive/papers-with-abstracts huggingface dataset:
Explanation vs Attention: A Two-Player Game to Obtain Attention for VQA
Authors: Badri N. Patro, Anupriy, Vinay P. Namboodiri
Date: 2019-11-19 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

In this paper, we aim to obtain improved attention for a visual question answering (VQA) task. It is challenging to provide supervision for attention. An observation we make is that visual explanations as obtained through class activation mappings (specifically Grad-CAM) that are meant to explain the performance of various networks could form a means of supervision. However, as the distributions of attention maps and that of Grad-CAMs differ, it would not be suitable to directly use these as a form of supervision. Rather, we propose the use of a discriminator that aims to distinguish samples of visual explanation and attention maps. The use of adversarial training of the attention regions as a two-player game between attention and explanation serves to bring the distributions of attention maps and visual explanations closer. Significantly, we observe that providing such a means of supervision also results in attention maps that are more closely related to human attention resulting in a substantial improvement over baseline stacked attention network (SAN) models. It also results in a good improvement in rank correlation metric on the VQA task. This method can also be combined with recent MCB based methods and results in consistent improvement. We also provide comparisons with other means for learning distributions such as based on Correlation Alignment (Coral), Maximum Mean Discrepancy (MMD) and Mean Square Error (MSE) losses and observe that the adversarial loss outperforms the other forms of learning the attention maps. Visualization of the results also confirms our hypothesis that attention maps improve using this form of supervision.
Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution
Authors: Chih-Chung Hsu, Chia-Hsiang Lin
Date: 2019-11-20 00:00:00
Tasks: Generative Adversarial Network, Image Super-Resolution, SSIM, Super-Resolution

Deep learning-based single image super-resolution enables very fast and high-visual-quality reconstruction. Recently, an enhanced super-resolution based on generative adversarial network (ESRGAN) has achieved excellent performance in terms of both qualitative and quantitative quality of the reconstructed high-resolution image. In this paper, we propose to add one more shortcut between two dense-blocks, as well as add shortcut between two convolution layers inside a dense-block. With this simple strategy of adding more shortcuts in the proposed network, it enables a faster learning process as the gradient information can be back-propagated more easily. Based on the improved ESRGAN, the dual reconstruction is proposed to learn different aspects of the super-resolved image for judiciously enhancing the quality of the reconstructed image. In practice, the super-resolution model is pre-trained solely based on pixel distance, followed by fine-tuning the parameters in the model based on adversarial loss and perceptual loss. Finally, we fuse two different models by weighted-summing their parameters to obtain the final super-resolution model. Experimental results demonstrated that the proposed method achieves excellent performance in the real-world image super-resolution challenge. We have also verified that the proposed dual reconstruction does further improve the quality of the reconstructed image in terms of both PSNR and SSIM.
Joint NN-Supported Multichannel Reduction of Acoustic Echo, Reverberation and Noise
Authors: Guillaume Carbajal, Romain Serizel, Emmanuel Vincent, Eric Humbert
Date: 2019-11-20 00:00:00

We consider the problem of simultaneous reduction of acoustic echo, reverberation and noise. In real scenarios, these distortion sources may occur simultaneously and reducing them implies combining the corresponding distortion-specific filters. As these filters interact with each other, they must be jointly optimized. We propose to model the target and residual signals after linear echo cancellation and dereverberation using a multichannel Gaussian modeling framework and to jointly represent their spectra by means of a neural network. We develop an iterative block-coordinate ascent algorithm to update all the filters. We evaluate our system on real recordings of acoustic echo, reverberation and noise acquired with a smart speaker in various situations. The proposed approach outperforms in terms of overall distortion a cascade of the individual approaches and a joint reduction approach which does not rely on a spectral model of the target and residual signals.
Learning Embeddings from Cancer Mutation Sets for Classification Tasks
Authors: Geoffroy Dubourg-Felonneau, Yasmeen Kussad, Dominic Kirkham, John W Cassidy, Nirmesh Patel, Harry W Clifford
Date: 2019-11-20 00:00:00
Tasks: Classification, Clustering, General Classification

Analysis of somatic mutation profiles from cancer patients is essential in the development of cancer research. However, the low frequency of most mutations and the varying rates of mutations across patients makes the data extremely challenging to statistically analyze as well as difficult to use in classification problems, for clustering, visualization or for learning useful information. Thus, the creation of low dimensional representations of somatic mutation profiles that hold useful information about the DNA of cancer cells will facilitate the use of such data in applications that will progress precision medicine. In this paper, we talk about the open problem of learning from somatic mutations, and present Flatsomatic: a solution that utilizes variational autoencoders (VAEs) to create latent representations of somatic profiles. The work done in this paper shows great potential for this method, with the VAE embeddings performing better than PCA for a clustering task, and performing equally well to the raw high dimensional data for a classification task. We believe the methods presented herein can be of great value in future research and in bringing data-driven models into precision oncology.
Sibling Neural Estimators: Improving Iterative Image Decoding with Gradient Communication
Authors: Ankur Mali, Alexander G. Ororbia, Clyde Lee Giles
Date: 2019-11-20 00:00:00
Tasks: Decoder, Image Compression, Image Reconstruction

For lossy image compression, we develop a neural-based system which learns a nonlinear estimator for decoding from quantized representations. The system links two recurrent networks that \help" each other reconstruct same target image patches using complementary portions of spatial context that communicate via gradient signals. This dual agent system builds upon prior work that proposed the iterative refinement algorithm for recurrent neural network (RNN)based decoding which improved image reconstruction compared to standard decoding techniques. Our approach, which works with any encoder, neural or non-neural, This system progressively reduces image patch reconstruction error over a fixed number of steps. Experiment with variants of RNN memory cells, with and without future information, find that our model consistently creates lower distortion images of higher perceptual quality compared to other approaches. Specifically, on the Kodak Lossless True Color Image Suite, we observe as much as a 1:64 decibel (dB) gain over JPEG, a 1:46 dB gain over JPEG 2000, a 1:34 dB gain over the GOOG neural baseline, 0:36 over E2E (a modern competitive neural compression model), and 0:37 over a single iterative neural decoder.
MMTM: Multimodal Transfer Module for CNN Fusion
Authors: Hamid Reza Vaezi Joze, Amirreza Shaban, Michael L. Iuzzolino, Kazuhito Koishida
Date: 2019-11-20 00:00:00
Proceeding: CVPR 2020 6
Tasks: Action Recognition, Action Recognition In Videos, Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition, Semantic Segmentation, Speech Enhancement

In late fusion, each modality is processed in a separate unimodal Convolutional Neural Network (CNN) stream and the scores of each modality are fused at the end. Due to its simplicity late fusion is still the predominant approach in many state-of-the-art multimodal applications. In this paper, we present a simple neural network module for leveraging the knowledge from multiple modalities in convolutional neural networks. The propose unit, named Multimodal Transfer Module (MMTM), can be added at different levels of the feature hierarchy, enabling slow modality fusion. Using squeeze and excitation operations, MMTM utilizes the knowledge of multiple modalities to recalibrate the channel-wise features in each CNN stream. Despite other intermediate fusion methods, the proposed module could be used for feature modality fusion in convolution layers with different spatial dimensions. Another advantage of the proposed method is that it could be added among unimodal branches with minimum changes in the their network architectures, allowing each branch to be initialized with existing pretrained weights. Experimental results show that our framework improves the recognition accuracy of well-known multimodal networks. We demonstrate state-of-the-art or competitive performance on four datasets that span the task domains of dynamic hand gesture recognition, speech enhancement, and action recognition with RGB and body joints.
The dynamics of the stomatognathic system from 4D multimodal data
Authors: Agnieszka A. Tomaka, Leszek Luchowski, Dariusz Pojda, Michał Tarnawski, Krzysztof Domino
Date: 2019-11-20 00:00:00
Tasks: Time Series Analysis

The purpose of this chapter is to discuss methods of acquisition, visualization and analysis of the dynamics of a complex biomedical system, illustrated by the human stomatognathic system. The stomatognathic system consists of the teeth and the skull bones with the maxilla and the mandible. Its dynamics can be described by the change of mutual position of the lower/mandibular part versus the upper/maxillary one due to the physiological motion of opening, chewing and swallowing. In order to analyse the dynamics of the stomatognathic system its morphology and motion has to be digitized, which is done using static and dynamic multimodal imagery like CBCT and 3D scans data and temporal measurements of motion. The integration of multimodal data incorporates different direct and indirect methods of registration - aligning of all the data in the same coordinate system. The integrated sets of data form 4D multimodal data which can be further visualized, modeled, and subjected to multivariate time series analysis. Example results are shown. Although there is no direct method of imaging the TMJ motion, the integration of multimodal data forms an adequate tool. As medical imaging becomes ever more diverse and ever more accessible, organizing the imagery and measurements into unified, comprehensive records can deliver to the doctor the most information in the most accessible form, creating a new quality in data simulation, analysis and interpretation.
Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks
Authors: Hector J. Hortua, Riccardo Volpi, Dimitri Marinelli, Luigi Malagò
Date: 2019-11-19 00:00:00

In this paper, we present the first study that compares different models of Bayesian Neural Networks (BNNs) to predict the posterior distribution of the cosmological parameters directly from the Cosmic Microwave Background temperature and polarization maps. We focus our analysis on four different methods to sample the weights of the network during training: Dropout, DropConnect, Reparameterization Trick (RT), and Flipout. We find out that Flipout outperforms all other methods regardless of the architecture used, and provides tighter constraints for the cosmological parameters. Moreover we compare with MCMC posterior analysis obtaining comparable error correlation among parameters, with BNNs being orders of magnitude faster in inference, although less accurate. Thanks to the speed of the inference process with BNNs, the posterior distribution, outcome of the neural network, can be used as the initial proposal for the Markov Chain. We show that this combined approach increases the acceptance rate in the Metropolis-Hasting algorithm and accelerates the convergence of the MCMC, while reaching the same final accuracy. In the second part of the paper, we present a guide to the training and calibration of a successful multi-channel BNN for the CMB temperature and polarization map. We show how tuning the regularization parameter for the standard deviation of the approximate posterior on the weights in Flipout and RT we can produce unbiased and reliable uncertainty estimates, i.e., the regularizer acts like a hyperparameter analogous to the dropout rate in Dropout. Finally, we show how polarization, when combined with the temperature in a unique multi-channel tensor fed to a single BNN, helps to break degeneracies among parameters and provides stringent constraints.
Adversarial Robustness of Flow-Based Generative Models
Authors: Phillip Pope, Yogesh Balaji, Soheil Feizi
Date: 2019-11-20 00:00:00
Tasks: Adversarial Robustness

Flow-based generative models leverage invertible generator functions to fit a distribution to the training data using maximum likelihood. Despite their use in several application domains, robustness of these models to adversarial attacks has hardly been explored. In this paper, we study adversarial robustness of flow-based generative models both theoretically (for some simple models) and empirically (for more complex ones). First, we consider a linear flow-based generative model and compute optimal sample-specific and universal adversarial perturbations that maximally decrease the likelihood scores. Using this result, we study the robustness of the well-known adversarial training procedure, where we characterize the fundamental trade-off between model robustness and accuracy. Next, we empirically study the robustness of two prominent deep, non-linear, flow-based generative models, namely GLOW and RealNVP. We design two types of adversarial attacks; one that minimizes the likelihood scores of in-distribution samples, while the other that maximizes the likelihood scores of out-of-distribution ones. We find that GLOW and RealNVP are extremely sensitive to both types of attacks. Finally, using a hybrid adversarial training procedure, we significantly boost the robustness of these generative models.
Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation
Authors: Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei zhang, Yuzhong Qu
Date: 2019-11-20 00:00:00
Tasks: Entity Alignment, Knowledge Graphs

Graph neural networks (GNNs) have emerged as a powerful paradigm for embedding-based entity alignment due to their capability of identifying isomorphic subgraphs. However, in real knowledge graphs (KGs), the counterpart entities usually have non-isomorphic neighborhood structures, which easily causes GNNs to yield different representations for them. To tackle this problem, we propose a new KG alignment network, namely AliNet, aiming at mitigating the non-isomorphism of neighborhood structures in an end-to-end manner. As the direct neighbors of counterpart entities are usually dissimilar due to the schema heterogeneity, AliNet introduces distant neighbors to expand the overlap between their neighborhood structures. It employs an attention mechanism to highlight helpful distant neighbors and reduce noises. Then, it controls the aggregation of both direct and distant neighborhood information using a gating mechanism. We further propose a relation loss to refine entity representations. We perform thorough experiments with detailed ablation studies and analyses on five entity alignment datasets, demonstrating the effectiveness of AliNet.
Seq2Seq RNN based Gait Anomaly Detection from Smartphone Acquired Multimodal Motion Data
Authors: Riccardo Bonetto, Mattia Soldan, Alberto Lanaro, Simone Milani, Michele Rossi
Date: 2019-11-19 00:00:00
Tasks: Anomaly Detection

Smartphones and wearable devices are fast growing technologies that, in conjunction with advances in wireless sensor hardware, are enabling ubiquitous sensing applications. Wearables are suitable for indoor and outdoor scenarios, can be placed on many parts of the human body and can integrate a large number of sensors capable of gathering physiological and behavioral biometric information. Here, we are concerned with gait analysis systems that extract meaningful information from a user's movements to identify anomalies and changes in their walking style. The solution that is put forward is subject-specific, as the designed feature extraction and classification tools are trained on the subject under observation. A smartphone mounted on an ad-hoc made chest support is utilized to gather inertial data and video signals from its built-in sensors and rear-facing camera. The collected video and inertial data are preprocessed, combined and then classified by means of a Recurrent Neural Network (RNN) based Sequence-to-Sequence (Seq2Seq) model, which is used as a feature extractor, and a following Convolutional Neural Network (CNN) classifier. This architecture provides excellent results, being able to correctly assess anomalies in 100% of the cases, for the considered tests, surpassing the performance of support vector machine classifiers.
DermGAN: Synthetic Generation of Clinical Skin Images with Pathology
Authors: Amirata Ghorbani, Vivek Natarajan, David Coz, Yu-An Liu
Date: 2019-11-20 00:00:00
Tasks: Data Augmentation

Despite the recent success in applying supervised deep learning to medical imaging tasks, the problem of obtaining large and diverse expert-annotated datasets required for the development of high performant models remains particularly challenging. In this work, we explore the possibility of using Generative Adverserial Networks (GAN) to synthesize clinical images with skin condition. We propose DermGAN, an adaptation of the popular Pix2Pix architecture, to create synthetic images for a pre-specified skin condition while being able to vary its size, location and the underlying skin color. We demonstrate that the generated images are of high fidelity using objective GAN evaluation metrics. In a Human Turing test, we note that the synthetic images are not only visually similar to real images, but also embody the respective skin condition in dermatologists' eyes. Finally, when using the synthetic images as a data augmentation technique for training a skin condition classifier, we observe that the model performs comparably to the baseline model overall while improving on rare but malignant conditions.
Learning Stylized Character Expressions from Humans
Authors: Deepali Aneja, Alex Colburn, Gary Faigin, Linda Shapiro, Barbara Mones
Date: 2019-11-19 00:00:00
Tasks: Retrieval

We present DeepExpr, a novel expression transfer system from humans to multiple stylized characters via deep learning. We developed : 1) a data-driven perceptual model of facial expressions, 2) a novel stylized character data set with cardinal expression annotations : FERG (Facial Expression Research Group) - DB (added two new characters), and 3) . We evaluated our method on a set of retrieval tasks on our collected stylized character dataset of expressions. We have also shown that the ranking order predicted by the proposed features is highly correlated with the ranking order provided by a facial expression expert and Mechanical Turk (MT) experiments.
Equivariant online predictions of non-stationary time series
Authors: Kōsaku Takanashi, Kenichiro McAlinn
Date: 2019-11-20 00:00:00
Tasks: Epidemiology, Time Series, Time Series Analysis

We discuss the finite sample theoretical properties of online predictions in non-stationary time series under model misspecification. To analyze the theoretical predictive properties of statistical methods under this setting, we first define the Kullback-Leibler risk, in order to place the problem within a decision theoretic framework. Under this framework, we show that a specific class of dynamic models -- random walk dynamic linear models -- produce exact minimax predictive densities. We first show this result under Gaussian assumptions, then relax this assumption using semi-martingale processes. This result provides a theoretical baseline, under both non-stationary and stationary time series data, for which other models can be compared against. We extend the result to the synthesis of multiple predictive densities. Three topical applications in epidemiology, climatology, and economics, confirm and highlight our theoretical results.
Mini Lesions Detection on Diabetic Retinopathy Images via Large Scale CNN Features
Authors: Qilei Chen, Xinzi Sun, Ning Zhang, Yu Cao, Benyuan Liu
Date: 2019-11-19 00:00:00
Tasks: Lesion Detection, object-detection, Object Detection, Region Proposal

Diabetic retinopathy (DR) is a diabetes complication that affects eyes. DR is a primary cause of blindness in working-age people and it is estimated that 3 to 4 million people with diabetes are blinded by DR every year worldwide. Early diagnosis have been considered an effective way to mitigate such problem. The ultimate goal of our research is to develop novel machine learning techniques to analyze the DR images generated by the fundus camera for automatically DR diagnosis. In this paper, we focus on identifying small lesions on DR fundus images. The results from our analysis, which include the lesion category and their exact locations in the image, can be used to facilitate the determination of DR severity (indicated by DR stages). Different from traditional object detection for natural images, lesion detection for fundus images have unique challenges. Specifically, the size of a lesion instance is usually very small, compared with the original resolution of the fundus images, making them diffcult to be detected. We analyze the lesion-vs-image scale carefully and propose a large-size feature pyramid network (LFPN) to preserve more image details for mini lesion instance detection. Our method includes an effective region proposal strategy to increase the sensitivity. The experimental results show that our proposed method is superior to the original feature pyramid network (FPN) method and Faster RCNN.
Vision: A Deep Learning Approach to provide walking assistance to the visually impaired
Authors: Nikhil Thakurdesai, Anupam Tripathi, Dheeraj Butani, Smita Sankhe
Date: 2019-11-20 00:00:00
Tasks: Depth Estimation, object-detection, Object Detection

Blind people face a lot of problems in their daily routines. They have to struggle a lot just to do their day-to-day chores. In this paper, we have proposed a system with the objective to help the visually impaired by providing audio aid guiding them to avoid obstacles, which will assist them to move in their surroundings. Object Detection using YOLO will help them detect the nearby objects and Depth Estimation using monocular vision will tell the approximate distance of the detected objects from the user. Despite a higher accuracy, stereo vision has many hardware constraints, which makes monocular vision the preferred choice for this application.