pwc-archive/papers-with-abstracts huggingface dataset:
Generative Models for Low-Dimensional Video Representation and Compressive Sensing
Authors: Rakib Hyder, M. Salman Asif
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Compressive Sensing, Denoising, Video Compressive Sensing

Generative priors have become highly effective in solving inverse problems including denoising, inpainting, and reconstruction from few and noisy measurements. With a generative model we can represent an image with a much lower dimensional latent codes. In the context of compressive sensing, if the unknown image belongs to the range of a pretrained generative network, then we can recover the image by estimating the underlying compact latent code from the available measurements. However, recent studies revealed that even untrained deep neural networks can work as a prior for recovering natural images. These approaches update the network weights keeping latent codes fixed to reconstruct the target image from the given measurements. In this paper, we optimize over network weights and latent codes to use untrained generative network as prior for video compressive sensing problem. We show that by optimizing over latent code, we can additionally get concise representation of the frames which retain the structural similarity of the video frames. We also apply low-rank constraint on the latent codes to represent the video sequences in even lower dimensional latent space. We empirically show that our proposed methods provide better or comparable accuracy and low computational complexity compared to the existing methods.
Retrieving Signals with Deep Complex Extractors
Authors: Chiheb Trabelsi, Olexa Bilaniuk, Ousmane Dia, Ying Zhang, Mirco Ravanelli, Jonathan Binas, Negar Rostamzadeh, Christopher  J Pal
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Audio Source Separation, Retrieval

Recent advances have made it possible to create deep complex-valued neural networks. Despite this progress, many challenging learning tasks have yet to leverage the power of complex representations. Building on recent advances, we propose a new deep complex-valued method for signal retrieval and extraction in the frequency domain. As a case study, we perform audio source separation in the Fourier domain. Our new method takes advantage of the convolution theorem which states that the Fourier transform of two convolved signals is the elementwise product of their Fourier transforms. Our novel method is based on a complex-valued version of Feature-Wise Linear Modulation (FiLM) and serves as the keystone of our proposed signal extraction method. We also introduce a new and explicit amplitude and phase-aware loss, which is scale and time invariant, taking into account the complex-valued components of the spectrogram. Using the Wall Street Journal Dataset, we compared our phase-aware loss to several others that operate both in the time and frequency domains and demonstrate the effectiveness of our proposed signal extraction method and proposed loss.
Exploring Properties of the Deep Image Prior
Authors: Andreas Kattamis, Tameem Adel, Adrian Weller
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12

The Deep Image Prior (DIP, Ulyanov et al., 2017) is a fascinating recent approach for recovering images which appear natural, yet is not fully understood. This work aims at shedding some further light on this approach by investigating the properties of the early outputs of the DIP. First, we show that these early iterations demonstrate invariance to adversarial perturbations by classifying progressive DIP outputs and using a novel saliency map approach. Next we explore using DIP as a defence against adversaries, showing good potential. Finally, we examine the adversarial invariancy of the early DIP outputs, and hypothesize that these outputs may remove non-robust image features. By comparing classification confidence values we show some evidence confirming this hypothesis.
GAN priors for Bayesian inference
Authors: Dhruv V. Patel, Assad A. Oberai
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Bayesian Inference, Generative Adversarial Network

Bayesian inference is used extensively to infer and to quantify the uncertainty in a field of interest from a measurement of a related field when the two are linked by a mathematical model. Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically. In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges. We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in a physics-based inverse problem and an inverse problem arising in computer vision. In this latter example, we also demonstrate how the knowledge of the spatial variation of uncertainty may be used to select an optimal strategy of placing the sensors (i.e. taking measurements), where information about the image is revealed one sub-region at a time.
Y-net: A Physics-constrained and Semi-supervised Learning Approach to the Phase Problem in Computational Electron Imaging
Authors: Nouamane Laanait, Junqi Yin, Albina Borisevich
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Retrieval

The phase problem in diffraction physics is one of the oldest inverse problems in all of science. The central difficulty that any approach to solving this inverse problem must overcome is that half of the information, namely the phase of the diffracted beam, is always missing. In the context of electron microscopy, the phase problem is generally non-linear and solutions provided by phase-retrieval techniques are known to be poor approximations to the physics of electrons interacting with matter. Here, we show that a semi-supervised learning approach can effectively solve the phase problem in electron microscopy/scattering. In particular, we introduce a new Deep Neural Network (DNN), Y-net, which simultaneously learns a reconstruction algorithm via supervised training in addition to learning a physics-based regularization via unsupervised training. We demonstrate that this constrained, semi-supervised approach is an order of magnitude more data-efficient and accurate than the same model trained in a purely supervised fashion. In addition, the architecture of the Y-net model provides for a straightforward evaluation of the consistency of the model's prediction during inference and is generally applicable to the phase problem in other settings.
PatchDIP Exploiting Patch Redundancy in Deep Image Prior for Denoising
Authors: Muhammad Asim, Fahad Shamshad, Ali Ahmed
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Denoising

The structure of a deep convolutional neural network initialized with random weights is able to sufficiently capture the patterns in a natural image. This finding motivates using deep neural network as an effective prior for natural images. In this work, we show that this strong prior, enforced by the structure of a ConvNet, can be augmented with the information that recurs in different patches of a natural image to boost the performance. We demonstrate that the self-similarity in the image patches can be exploited alongside deep image prior by optimizing the network weights to fit patches extracted from a single noisy image. Our results indicate that employing deep image prior on noisy patches provides an additional disincentive for the network to fit noise, and is encouraged to exploit redundancies among the patches yielding better denoising performance.
Subsampled Fourier Ptychography via Pretrained Invertible and Untrained Network Priors
Authors: Fahad Shamshad, Asif Hanif, Ali Ahmed
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12

Recently pretrained generative models have shown promising results for subsampled Fourier Ptychography (FP) in terms of quality of reconstruction for extremely low sampling rate and high noise. However, one of the significant drawbacks of these pretrained generative priors is their limited representation capabilities. Moreover, training these generative models requires access to a large number of fully-observed clean samples of a particular class of images like faces or digits that is prohibitive to obtain in the context of FP. In this paper, we propose to leverage the power of pretrained invertible and untrained generative models to mitigate the representation error issue and requirement of a large number of example images (for training generative models) respectively. Through extensive experiments, we demonstrate the effectiveness of proposed approaches in the context of FP for low sampling rates and high noise levels.
Learning Network Parameters in the ReLU Model
Authors: Arya Mazumdar, Ankit Singh Rawat
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: model

Rectified linear units, or ReLUs, have become a preferred activation function for artificial neural networks. In this paper we consider the problem of learning a generative model in the presence of nonlinearity (modeled by the ReLU functions). Given a set of signal vectors $\mathbf{y}^i \in \mathbb{R}^d, i =1, 2, \dots , n$, we  aim to learn the network parameters, i.e., the $d\times k$ matrix $A$, under the model $\mathbf{y}^i = \mathrm{ReLU}(A\mathbf{c}^i +\mathbf{b})$, where $\mathbf{b}\in \mathbb{R}^d$ is a random bias vector, and {$\mathbf{c}^i \in \mathbb{R}^k$ are arbitrary unknown latent vectors}. We show that it is possible to recover the column space of $A$ within an error of $O(d)$ (in Frobenius norm) under certain conditions on the  distribution of $\mathbf{b}$.
Unrolled, model-based networks for lensless imaging
Authors: Kristina Monakhova, Joshua Yurtsever, Grace Kuo, Nick Antipa, Kyrollos Yanny, Laura Waller
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: model

We develop end-to-end learned reconstructions for lensless mask-based cameras, including an experimental system for capturing aligned lensless and lensed images for training.  Various reconstruction methods are explored, on a scale from classic iterative approaches (based on the physical imaging model) to deep learned methods with many learned parameters.  In the middle ground, we present several variations of unrolled alternating direction method of multipliers (ADMM) with varying numbers of learned parameters. The network structure combines knowledge of the physical imaging model with learned parameters updated from the data, which compensate for artifacts caused by physical approximations. Our unrolled approach is 20X faster than classic methods and produces better reconstruction quality than both the classic and deep methods on our experimental system.
Precise asymptotics for phase retrieval and compressed sensing with random generative priors
Authors: Benjamin Aubin, Bruno Loureiro, Antoine Baker, Florent Krzakala, Lenka Zdeborova
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: compressed sensing, Retrieval

We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix.
We analyse sharp asymptotics of the information-theoretically optimal performance and that of the best known polynomial algorithms under a generative prior consisting of a single layer neural network with a random weight matrix. 
We compare the performance to sparse separable priors and conclude that generative priors might be advantageous in terms of algorithmic performance. 
In particular, while sparsity does not allow to perform compressive phase retrieval efficiently close to its information-theoretic limit, it is found that under the random generative prior compressed phase retrieval becomes tractable.
Compressed Sensing and Overparametrized Networks: Overfitting Peaks in a Model of Misparametrized Sparse Regression in the Interpolation Limit
Authors: Partha P Mitra
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: compressed sensing, regression

Current practice in machine learning is to employ deep nets in an overparametrized limit, with the nominal number of parameters typically exceeding the number of measurements. This resembles the situation in compressed sensing, or in sparse regression with $l_1$ penalty terms, and provides a theoretical avenue for understanding phenomena that arise in the context of deep nets. One such phenonemon is the success of deep nets in providing good generalization in an interpolating regime with zero training error. Traditional statistical practice calls for regularization or smoothing to prevent "overfitting" (poor generalization performance). However, recent work shows that there exist data interpolation procedures which are statistically consistent and provide good generalization performance\cite{belkin2018overfitting} ("perfect fitting"). In this context, it has been suggested that "classical" and "modern" regimes for machine learning are separated by a peak in the generalization error ("risk") curve, a phenomenon dubbed "double descent"\cite{belkin2019reconciling}. While such overfitting peaks do exist and arise from ill-conditioned design matrices, here we challenge the interpretation of the overfitting peak as demarcating the regime where good generalization occurs under overparametrization. 

We propose a model of Misparamatrized Sparse Regression (MiSpaR) and analytically compute the GE curves for $l_2$ and $l_1$ penalties. We show that the overfitting peak arising in the interpolation limit is dissociated from the regime of good generalization. The analytical expressions are obtained in the so called "thermodynamic" limit. We find an additional interesting phenomenon: increasing overparametrization in the fitting model increases sparsity, which should intuitively improve performance of $l_1$ penalized regression. However, at the same time, the relative number of measurements decrease compared to the number of fitting parameters, and eventually overparametrization does lead to poor generalization. Nevertheless, $l_1$ penalized regression can show good generalization performance under conditions of data interpolation even with a large amount of overparametrization. These results provide a theoretical avenue into studying inverse problems in the interpolating regime using overparametrized fitting functions such as deep nets.
A GAN based solver of black-box inverse problems
Authors: Michael Gillhofer, Hubert Ramsauer, Johannes Brandstetter, Bernhard Schäfl, Sepp Hochreiter
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12

We propose a GAN based approach to solve inverse problems which have non-differential or non-continuous forward relations. In the standard sense, an inverse problem is interpreted as the process of calculating factors that produce observations. We reformulate the inverse problem such that the discriminator is a binary classifier and the generator is used to produce samples in a local region of the input domain of the forward relation. Our GAN based approach solves inverse problems by using adversarial training but without relying on the gradients of the original problem formulation. We prove the efficacy of our approach by applying it to an artificially generated topology optimization problem. We demonstrate that despite not having access to derivatives of f our method leads to similar results than more traditional topology optimization methods.
Sample Complexity Lower Bounds for Compressive Sensing with Generative Models
Authors: Zhaoqiang Liu, Jonathan Scarlett
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Compressive Sensing

The goal of standard compressive sensing is to estimate an unknown vector from linear measurements under the assumption of sparsity in some basis. Recently, it has been shown that significantly fewer measurements may be required if the sparsity assumption is replaced by the assumption that the unknown vector lies near the range of a suitably-chosen generative model.  In particular, in (Bora {\em et al.}, 2017) it was shown that roughly $O(k\log L)$ random Gaussian measurements suffice for accurate recovery when the $k$-input generative model is bounded and $L$-Lipschitz, and that $O(kd \log w)$ measurements suffice for $k$-input ReLU networks with depth $d$ and width $w$.  In this paper, we establish corresponding algorithm-independent lower bounds on the sample complexity using tools from minimax statistical analysis.  In accordance with the above upper bounds, our results are summarized as follows: (i) We construct an $L$-Lipschitz generative model capable of generating group-sparse signals, and show that the resulting necessary number of measurements is $\Omega(k \log L)$; (ii) Using similar ideas, we construct two-layer ReLU networks of high width requiring $\Omega(k \log w)$ measurements, as well as lower-width deep ReLU networks requiring $\Omega(k d)$ measurements.  As a result, we establish that the scaling laws derived in (Bora {\em et al.}, 2017) are optimal or near-optimal in the absence of further assumptions.
Energy Dissipation with Plug-and-Play Priors
Authors: Hendrik Sommerhoff, Andreas Kolb, Michael Moeller
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: Denoising, Image Denoising

Neural networks have reached outstanding performance for solving various ill-posed inverse problems in imaging. However, drawbacks of end-to-end learning approaches in comparison to classical variational methods are the requirement of expensive retraining for even slightly different problem statements and the lack of provable error bounds during inference. Recent works tackled the first problem by using networks trained for Gaussian image denoising as generic plug-and-play regularizers in energy minimization algorithms. Even though this obtains state-of-the-art results on many tasks, heavy restrictions on the network architecture have to be made if provable convergence of the underlying fixed point iteration is a requirement. More recent work has proposed to train networks to output descent directions with respect to a given energy function with a provable guarantee of convergence to a minimizer of that energy. However, each problem and energy requires the training of a separate network.
In this paper we consider the combination of both approaches by projecting the outputs of a plug-and-play denoising network onto the cone of descent directions to a given energy. This way, a single pre-trained network can be used for a wide variety of reconstruction tasks. Our results show improvements compared to classical energy minimization methods while still having provable convergence guarantees.
A Hybrid Architecture for On-Device Compressive Machine Learning
Authors: Yang Li, Thomas Strohmer
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Deep_Invers 2019 12
Tasks: BIG-bench Machine Learning, Compressive Sensing, image-classification, Image Classification

Developing machine learning techniques that can protect the privacy of users’ data is of utmost importance, as tracking and selling our digital data (often without our permission and knowledge) has become a booming business model. By processing the data on the device that has collected the data, we can dramatically increase the level of privacy. On-device machine offers several additional benefits, such as low latency, efficient use of network bandwidth, and more autonomy. However, many devices deployed on the “edge” have very limited memory, weak processors, and scarce energy supply. This poses the challenge of envisioning new machine learning architectures that can function properly under such dire conditions. This issue is particularly urgent with the emergence of the Internet of Things.

We propose a hybrid hardware-software framework that facilitates increased privacy protection due to on-device processing and moreover has the potential to significantly reduce the computational complexity and memory requirements of on-device machine learning. In the first step, inspired by compressive sensing, data is collected in compressed form simultaneously with the sensing process. Thus this compression happens already at the hardware level during data acquisition. But unlike in compressive sensing, this compression is achieved via a projection operator that is specifically tailored to the desired machine learning task. The second step consists of a specially designed and trained deep network. Numerical simulations in image classification illustrate the viability of our method.
On recognition of Cyrillic Text
Authors: Kostiantyn Liepieshov, Oles Dobosevych
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Document_Intelligen 2019 12
Tasks: Handwritten Text Recognition

We introduce the largest (among publicly available) dataset for Cyrillic Handwritten Text Recognition and the first dataset for Cyrillic Text in the Wild Recognition, as well as suggest a method for recognizing Cyrillic Handwritten Text and Text in the Wild. Based on this approach, we develop a system that can reduce the document processing time for one of the largest mathematical competitions in Ukraine by 12 days and the amount of used paper by 0.5 ton.
Information Extraction from Text Regions with Complex Tabular Structure
Authors: Kaixuan Zhang, Zejiang Shen, Jie zhou, Melissa Dell
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Document_Intelligen 2019 12

Recent innovations have improved layout analysis of document images, significantly improving our ability to identify text and non-text regions. However, extracting information from within text regions remains quite challenging because the text region may have a complex structure. In this paper, we present a new dataset with complex text structure, and propose new methods to robustly retrieve information from the complex text region.
Doc2Dial: a Framework for Dialogue Composition Grounded in Business Documents
Authors: Song Feng, Kshitij Fadni, Q. Vera Liao, Luis A. Lastras
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Document_Intelligen 2019 12

 We introduce Doc2Dial, an end-to-end framework for generating conversational data grounded in business documents via crowdsourcing. Such data can be used to train automated dialogue agents performing customer care tasks for the enterprises or organizations. In particular, the framework takes the documents as input and generates the tasks for obtaining the annotations for simulating dialog flows. The dialog flows are used to guide the collection of utterances produced by crowd workers. The outcomes include dialogue data grounded in the given documents, as well as various types of annotations that help ensure the quality of the data and the flexibility to (re)composite dialogues.
Towards Neural Similarity Evaluator
Authors: Hassan Kané, Yusuf Kocyigit, Pelkins Ajanoh, Ali Abdalla, Mohamed Coulibali
Date: 2019-09-14 00:00:00
Proceeding: NeurIPS Workshop Document_Intelligen 2019 12

We review three limitations of BLEU and ROUGE – the most popular metrics
used to assess reference summaries against hypothesis summaries, come up with
criteria for what a good metric should behave like and propose concrete ways to
assess the performance of a metric in detail and show the potential of Transformers-based Language Models to assess reference summaries against hypothesis summaries.