pwc-archive/papers-with-abstracts huggingface dataset:
InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation
Authors: Yaohui Wang, Francois Bremond, Antitza Dantcheva
Date: 2021-01-08 00:00:00
Tasks: Generative Adversarial Network, Video Generation

In this work, we introduce an unconditional video generative model, InMoDeGAN, targeted to (a) generate high quality videos, as well as to (b) allow for interpretation of the latent space. For the latter, we place emphasis on interpreting and manipulating motion. Towards this, we decompose motion into semantic sub-spaces, which allow for control of generated samples. We design the architecture of InMoDeGAN-generator in accordance to proposed Linear Motion Decomposition, which carries the assumption that motion can be represented by a dictionary, with related vectors forming an orthogonal basis in the latent space. Each vector in the basis represents a semantic sub-space. In addition, a Temporal Pyramid Discriminator analyzes videos at different temporal resolutions. Extensive quantitative and qualitative analysis shows that our model systematically and significantly outperforms state-of-the-art methods on the VoxCeleb2-mini and BAIR-robot datasets w.r.t. video quality related to (a). Towards (b) we present experimental results, confirming that decomposed sub-spaces are interpretable and moreover, generated motion is controllable.
One-Class Classification: A Survey
Authors: Pramuditha Perera, Poojan Oza, Vishal M. Patel
Date: 2021-01-08 00:00:00
Tasks: Classification, General Classification, Multi-class Classification, One-Class Classification, Survey

One-Class Classification (OCC) is a special case of multi-class classification, where data observed during training is from a single positive class. The goal of OCC is to learn a representation and/or a classifier that enables recognition of positively labeled queries during inference. This topic has received considerable amount of interest in the computer vision, machine learning and biometrics communities in recent years. In this article, we provide a survey of classical statistical and recent deep learning-based OCC methods for visual recognition. We discuss the merits and drawbacks of existing OCC approaches and identify promising avenues for research in this field. In addition, we present a discussion of commonly used datasets and evaluation metrics for OCC.
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
Authors: Ruohan Gao, Kristen Grauman
Date: 2021-01-08 00:00:00
Proceeding: CVPR 2021 1
Tasks: Speech Separation

We introduce a new approach for audio-visual speech separation. Given a video, the goal is to extract the speech associated with a face in spite of simultaneous background sounds and/or other human speakers. Whereas existing methods focus on learning the alignment between the speaker's lip movements and the sounds they generate, we propose to leverage the speaker's face appearance as an additional prior to isolate the corresponding vocal qualities they are likely to produce. Our approach jointly learns audio-visual speech separation and cross-modal speaker embeddings from unlabeled video. It yields state-of-the-art results on five benchmark datasets for audio-visual speech separation and enhancement, and generalizes well to challenging real-world videos of diverse scenarios. Our video results and code: http://vision.cs.utexas.edu/projects/VisualVoice/.
Quantum Tensor Network in Machine Learning: An Application to Tiny Object Classification
Authors: Fanjie Kong, Xiao-Yang Liu, Ricardo Henao
Date: 2021-01-08 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, image-classification, Image Classification, Object, Tensor Networks

Tiny object classification problem exists in many machine learning applications like medical imaging or remote sensing, where the object of interest usually occupies a small region of the whole image. It is challenging to design an efficient machine learning model with respect to tiny object of interest. Current neural network structures are unable to deal with tiny object efficiently because they are mainly developed for images featured by large scale objects. However, in quantum physics, there is a great theoretical foundation guiding us to analyze the target function for image classification regarding to specific objects size ratio. In our work, we apply Tensor Networks to solve this arising tough machine learning problem. First, we summarize the previous work that connects quantum spin model to image classification and bring the theory into the scenario of tiny object classification. Second, we propose using 2D multi-scale entanglement renormalization ansatz (MERA) to classify tiny objects in image. In the end, our experimental results indicate that tensor network models are effective for tiny object classification problem and potentially will beat state-of-the-art. Our codes will be available online https://github.com/timqqt/MERA_Image_Classification.
The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels
Authors: Austin Stone, Oscar Ramirez, Kurt Konolige, Rico Jonschkowski
Date: 2021-01-07 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

Robots have to face challenging perceptual settings, including changes in viewpoint, lighting, and background. Current simulated reinforcement learning (RL) benchmarks such as DM Control provide visual input without such complexity, which limits the transfer of well-performing methods to the real world. In this paper, we extend DM Control with three kinds of visual distractions (variations in background, color, and camera pose) to produce a new challenging benchmark for vision-based control, and we analyze state of the art RL algorithms in these settings. Our experiments show that current RL methods for vision-based control perform poorly under distractions, and that their performance decreases with increasing distraction complexity, showing that new methods are needed to cope with the visual complexities of the real world. We also find that combinations of multiple distraction types are more difficult than a mere combination of their individual effects.
Learning Guided Electron Microscopy with Active Acquisition
Authors: Lu Mi, Hao Wang, Yaron Meirovitch, Richard Schalek, Srinivas C. Turaga, Jeff W. Lichtman, Aravinthan D. T. Samuel, Nir Shavit
Date: 2021-01-07 00:00:00
Tasks: Diversity

Single-beam scanning electron microscopes (SEM) are widely used to acquire massive data sets for biomedical study, material analysis, and fabrication inspection. Datasets are typically acquired with uniform acquisition: applying the electron beam with the same power and duration to all image pixels, even if there is great variety in the pixels' importance for eventual use. Many SEMs are now able to move the beam to any pixel in the field of view without delay, enabling them, in principle, to invest their time budget more effectively with non-uniform imaging. In this paper, we show how to use deep learning to accelerate and optimize single-beam SEM acquisition of images. Our algorithm rapidly collects an information-lossy image (e.g. low resolution) and then applies a novel learning method to identify a small subset of pixels to be collected at higher resolution based on a trade-off between the saliency and spatial diversity. We demonstrate the efficacy of this novel technique for active acquisition by speeding up the task of collecting connectomic datasets for neurobiology by up to an order of magnitude.
Single Image Super-Resolution
Authors: Baran Ataman, Mert Seker, David Mckee
Date: 2021-01-08 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

This study presents a chronological overview of the single image super-resolution problem. We first define the problem thoroughly and mention some of the serious challenges. Then the problem formulation and the performance metrics are defined. We give an overview of the previous methods relying on reconstruction based solutions and then continue with the deep learning approaches. We pick 3 landmark architectures and present their results quantitatively. We see that the latest proposed network gives favorable output compared to the previous methods.
Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images
Authors: Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, Jianzhuang Liu
Date: 2021-01-08 00:00:00
Proceeding: CVPR 2021 1
Tasks: Denoising, Image Denoising, Self-Supervised Learning

In the last few years, image denoising has benefited a lot from the fast development of neural networks. However, the requirement of large amounts of noisy-clean image pairs for supervision limits the wide use of these models. Although there have been a few attempts in training an image denoising model with only single noisy images, existing self-supervised denoising approaches suffer from inefficient network training, loss of useful information, or dependence on noise modeling. In this paper, we present a very simple yet effective method named Neighbor2Neighbor to train an effective image denoising model with only noisy images. Firstly, a random neighbor sub-sampler is proposed for the generation of training image pairs. In detail, input and target used to train a network are images sub-sampled from the same noisy image, satisfying the requirement that paired pixels of paired images are neighbors and have very similar appearance with each other. Secondly, a denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance. The proposed Neighbor2Neighbor framework is able to enjoy the progress of state-of-the-art supervised denoising networks in network architecture design. Moreover, it avoids heavy dependence on the assumption of the noise distribution. We explain our approach from a theoretical perspective and further validate it through extensive experiments, including synthetic experiments with different noise distributions in sRGB space and real-world experiments on a denoising benchmark dataset in raw-RGB space.
Unsupervised Domain Adaptation of Black-Box Source Models
Authors: Haojian Zhang, Yabin Zhang, Kui Jia, Lei Zhang
Date: 2021-01-08 00:00:00
Tasks: Domain Adaptation, Learning with noisy labels, Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to learn models for a target domain of unlabeled data by transferring knowledge from a labeled source domain. In the traditional UDA setting, labeled source data are assumed to be available for adaptation. Due to increasing concerns for data privacy, source-free UDA is highly appreciated as a new UDA setting, where only a trained source model is assumed to be available, while labeled source data remain private. However, trained source models may also be unavailable in practice since source models may have commercial values and exposing source models brings risks to the source domain, e.g., problems of model misuse and white-box attacks. In this work, we study a subtly different setting, named Black-Box Unsupervised Domain Adaptation (B$^2$UDA), where only the application programming interface of source model is accessible to the target domain; in other words, the source model itself is kept as a black-box one. To tackle B$^2$UDA, we propose a simple yet effective method, termed Iterative Learning with Noisy Labels (IterLNL). With black-box models as tools of noisy labeling, IterLNL conducts noisy labeling and learning with noisy labels (LNL), iteratively. To facilitate the implementation of LNL in B$^2$UDA, we estimate the noise rate from model predictions of unlabeled target data and propose category-wise sampling to tackle the unbalanced label noise among categories. Experiments on benchmark datasets show the efficacy of IterLNL. Given neither source data nor source models, IterLNL performs comparably with traditional UDA methods that make full use of labeled source data.
ADiag: Graph Neural Network Based Diagnosis of Alzheimer's Disease
Authors: Vishnu Ram Sampathkumar
Date: 2021-01-08 00:00:00
Tasks: Diagnostic, Graph Neural Network

Alzheimer's Disease (AD) is the most widespread neurodegenerative disease, affecting over 50 million people across the world. While its progression cannot be stopped, early and accurate diagnostic testing can drastically improve quality of life in patients. Currently, only qualitative means of testing are employed in the form of scoring performance on a battery of cognitive tests. The inherent disadvantage of this method is that the burden of an accurate diagnosis falls on the clinician's competence. Quantitative methods like MRI scan assessment are inaccurate at best,due to the elusive nature of visually observable changes in the brain. In lieu of these disadvantages to extant methods of AD diagnosis, we have developed ADiag, a novel quantitative method to diagnose AD through GraphSAGE Network and Dense Differentiable Pooling (DDP) analysis of large graphs based on thickness difference between different structural regions of the cortex. Preliminary tests of ADiag have revealed a robust accuracy of 83%, vastly outperforming other qualitative and quantitative diagnostic techniques.
Deep Convolutional Neural Network based Classification of Alzheimer's Disease using MRI data
Authors: Ali Nawaz, Syed Muhammad Anwar, Rehan Liaqat, Javid Iqbal, Ulas Bagci, Muhammad Majid
Date: 2021-01-08 00:00:00
Tasks: Diagnostic, General Classification, Multi-class Classification

Alzheimer's disease (AD) is a progressive and incurable neurodegenerative disease which destroys brain cells and causes loss to patient's memory. An early detection can prevent the patient from further damage of the brain cells and hence avoid permanent memory loss. In past few years, various automatic tools and techniques have been proposed for diagnosis of AD. Several methods focus on fast, accurate and early detection of the disease to minimize the loss to patients mental health. Although machine learning and deep learning techniques have significantly improved medical imaging systems for AD by providing diagnostic performance close to human level. But the main problem faced during multi-class classification is the presence of highly correlated features in the brain structure. In this paper, we have proposed a smart and accurate way of diagnosing AD based on a two-dimensional deep convolutional neural network (2D-DCNN) using imbalanced three-dimensional MRI dataset. Experimental results on Alzheimer Disease Neuroimaging Initiative magnetic resonance imaging (MRI) dataset confirms that the proposed 2D-DCNN model is superior in terms of accuracy, efficiency, and robustness. The model classifies MRI into three categories: AD, mild cognitive impairment, and normal control: and has achieved 99.89% classification accuracy with imbalanced classes. The proposed model exhibits noticeable improvement in accuracy as compared to the state-fo-the-art methods.
Predicting Semen Motility using three-dimensional Convolutional Neural Networks
Authors: Priyansi, Biswaroop Bhattacharjee, Junaid Rahim
Date: 2021-01-08 00:00:00
Tasks: Deep Learning

Manual and computer aided methods to perform semen analysis are time-consuming, requires extensive training and prone to human error. The use of classical machine learning and deep learning based methods using videos to perform semen analysis have yielded good results. The state-of-the-art method uses regular convolutional neural networks to perform quality assessments on a video of the provided sample. In this paper we propose an improved deep learning based approach using three-dimensional convolutional neural networks to predict sperm motility from microscopic videos of the semen sample. We make use of the VISEM dataset that consists of video and tabular data of semen samples collected from 85 participants. We were able to achieve good results from significantly less data points. Our models indicate that deep learning based automatic semen analysis may become a valuable and effective tool in fertility and IVF labs.
Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks
Authors: Marissa Dotter, Sherry Xie, Keith Manville, Josh Harguess, Colin Busho, Mikel Rodriguez
Date: 2021-01-08 00:00:00
Tasks: Adversarial Attack, Attribute, Self-Driving Cars

Machine Learning (ML) models are known to be vulnerable to adversarial inputs and researchers have demonstrated that even production systems, such as self-driving cars and ML-as-a-service offerings, are susceptible. These systems represent a target for bad actors. Their disruption can cause real physical and economic harm. When attacks on production ML systems occur, the ability to attribute the attack to the responsible threat group is a critical step in formulating a response and holding the attackers accountable. We pose the following question: can adversarially perturbed inputs be attributed to the particular methods used to generate the attack? In other words, is there a way to find a signal in these attacks that exposes the attack algorithm, model architecture, or hyperparameters used in the attack? We introduce the concept of adversarial attack attribution and create a simple supervised learning experimental framework to examine the feasibility of discovering attributable signals in adversarial attacks. We find that it is possible to differentiate attacks generated with different attack algorithms, models, and hyperparameters on both the CIFAR-10 and MNIST datasets.
A review for Tone-mapping Operators on Wide Dynamic Range Image
Authors: Ziyi Liu
Date: 2021-01-08 00:00:00
Tasks: Tone Mapping

The dynamic range of our normal life can exceeds 120 dB, however, the smart-phone cameras and the conventional digital cameras can only capture a dynamic range of 90 dB, which sometimes leads to loss of details for the recorded image. Now, some professional hardware applications and image fusion algorithms have been devised to take wide dynamic range (WDR), but unfortunately existing devices cannot display WDR image. Tone mapping (TM) thus becomes an essential step for exhibiting WDR image on our ordinary screens, which convert the WDR image into low dynamic range (LDR) image. More and more researchers are focusing on this topic, and give their efforts to design an excellent tone mapping operator (TMO), showing detailed images as the same as the perception that human eyes could receive. Therefore, it is important for us to know the history, development, and trend of TM before proposing a practicable TMO. In this paper, we present a comprehensive study of the most well-known TMOs, which divides TMOs into traditional and machine learning-based category.
Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks
Authors: Sebastian Palacio, Philipp Engler, Jörn Hees, Andreas Dengel
Date: 2021-01-07 00:00:00
Tasks: General Classification, Image Classification, Multi-Task Learning

Classification problems solved with deep neural networks (DNNs) typically rely on a closed world paradigm, and optimize over a single objective (e.g., minimization of the cross-entropy loss). This setup dismisses all kinds of supporting signals that can be used to reinforce the existence or absence of a particular pattern. The increasing need for models that are interpretable by design makes the inclusion of said contextual signals a crucial necessity. To this end, we introduce the notion of Self-Supervised Autogenous Learning (SSAL) models. A SSAL objective is realized through one or more additional targets that are derived from the original supervised classification task, following architectural principles found in multi-task learning. SSAL branches impose low-level priors into the optimization process (e.g., grouping). The ability of using SSAL branches during inference, allow models to converge faster, focusing on a richer set of class-relevant features. We show that SSAL models consistently outperform the state-of-the-art while also providing structured predictions that are more interpretable.
Knowledge AI: New Medical AI Solution for Medical image Diagnosis
Authors: YingNi Wang, Shuge Lei, Jian Dai, Kehong Yuan
Date: 2021-01-08 00:00:00
Tasks: Clinical Knowledge

The implementation of medical AI has always been a problem. The effect of traditional perceptual AI algorithm in medical image processing needs to be improved. Here we propose a method of knowledge AI, which is a combination of perceptual AI and clinical knowledge and experience. Based on this method, the geometric information mining of medical images can represent the experience and information and evaluate the quality of medical images.
GRAPPA-GANs for Parallel MRI Reconstruction
Authors: Nader Tavaf, Amirsina Torfi, Kamil Ugurbil, Pierre-Francois Van de Moortele
Date: 2021-01-05 00:00:00
Tasks: Generative Adversarial Network, MRI Reconstruction, SSIM

k-space undersampling is a standard technique to accelerate MR image acquisitions. Reconstruction techniques including GeneRalized Autocalibrating Partial Parallel Acquisition(GRAPPA) and its variants are utilized extensively in clinical and research settings. A reconstruction model combining GRAPPA with a conditional generative adversarial network (GAN) was developed and tested on multi-coil human brain images from the fastMRI dataset. For various acceleration rates, GAN and GRAPPA reconstructions were compared in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For an acceleration rate of R=4, PSNR improved from 33.88 using regularized GRAPPA to 37.65 using GAN. GAN consistently outperformed GRAPPA for various acceleration rates.
Differentially Private Federated Learning for Cancer Prediction
Authors: Constance Beguier, Jean Ogier du Terrail, Iqraa Meah, Mathieu Andreux, Eric W. Tramel
Date: 2021-01-08 00:00:00
Tasks: Federated Learning, Prediction

Since 2014, the NIH funded iDASH (integrating Data for Analysis, Anonymization, SHaring) National Center for Biomedical Computing has hosted yearly competitions on the topic of private computing for genomic data. For one track of the 2020 iteration of this competition, participants were challenged to produce an approach to federated learning (FL) training of genomic cancer prediction models using differential privacy (DP), with submissions ranked according to held-out test accuracy for a given set of DP budgets. More precisely, in this track, we are tasked with training a supervised model for the prediction of breast cancer occurrence from genomic data split between two virtual centers while ensuring data privacy with respect to model transfer via DP. In this article, we present our 3rd place submission to this competition. During the competition, we encountered two main challenges discussed in this article: i) ensuring correctness of the privacy budget evaluation and ii) achieving an acceptable trade-off between prediction performance and privacy budget.