nils-herrmann/clean_scirepeval huggingface dataset:
Video temporal super-resolution using nonlocal registration and self-similarity

In this paper we present a novel temporal super-resolution method for increasing the frame-rate of single videos. The proposed algorithm is based on motion-compensated 3-D patches, i.e., a sequence of 2-D blocks following a given motion trajectory. The trajectories are computed through a coarse-to-fine motion estimation strategy embedding a regularized block-wise distance metric that takes into account the coherence of neighbouring motion vectors. Our algorithm comprises two stages. In the first stage, a nonlocal search procedure is used to find a set of 3-D patches (targets) similar to a given patch (reference), subsequently all targets are registered at sub-pixel precision with respect to the reference in an upsampled 3-D FFT domain, and finally all registered patches are aggregated at their appropriate locations in the high-resolution video. The second stage is used to further improve the estimation quality by correcting each 3-D patch of the video obtained from the first stage with a linear operator learned from the self-similarity of patches at a lower temporal scale. Our experimental evaluation on color videos shows that the proposed approach achieves high quality super-resolution results from both an objective and subjective point of view.
Quantized oversampled filter banks with erasures

Oversampled filter banks can be used to enhance resilience to erasures in communication systems in much the same way that finite-dimensional frames have previously been applied. This paper extends previous finite dimensional treatments to frames and signals in l/sub 2/(Z) with frame expansions that can be implemented efficiently with filter banks. It is shown that tight frames attain best performance. In particular, if encoding with a uniform frame, the quantization error is minimized if and only if the frame is tight. In case of one erasure and if encoding with a strongly uniform frame, tight frames are still optimal. In case of more erasures, an expression for the mean square error is given and some general considerations are presented.
The Distributed Karhunen–Loève Transform

The Karhunen-Loeve transform (KLT) is a key element of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approaches to the KLT, where several distributed terminals observe disjoint subsets of a random vector. We introduce several versions of the distributed KLT. First, a local KLT is introduced, which is the optimal solution for a given terminal, assuming all else is fixed. This local KLT is different and in general improves upon the marginal KLT which simply ignores other terminals. Both optimal approximation and compression using this local KLT are derived. Two important special cases are studied in detail, namely, the partial observation KLT which has access to a subset of variables, but aims at reconstructing them all, and the conditional KLT which has access to side information at the decoder. We focus on the jointly Gaussian case, with known correlation structure, and on approximation and compression problems. Then, the distributed KLT is addressed by considering local KLTs in turn at the various terminals, leading to an iterative algorithm which is locally convergent, sometimes reaching a global optimum, depending on the overall correlation structure. For compression, it is shown that the classical distributed source coding techniques admit a natural transform coding interpretation, the transform being the distributed KLT. Examples throughout illustrate the performance of the proposed distributed KLT. This distributed transform has potential applications in sensor networks, distributed image databases, hyper-spectral imagery, and data fusion
Guaranteed Performance in the FRI Setting

Finite Rate of Innovation (FRI) sampling theory has shown that it is possible to sample and perfectly reconstruct classes of non-bandlimited signals such as streams of Diracs. In the case of noisy measurements, FRI methods achieve the optimal performance given by the Cramér-Rao bound up to a certain PSNR and breaks down for smaller PSNRs. To the best of our knowledge, the precise anticipation of the breakdown event in FRI settings is still an open problem. In this letter, we address this issue by investigating the subspace swap event which has been broadly recognised as the reason for performance breakdown in SVD-based parameter estimation algorithms. We work out at which noise level the absence of subspace swap is guaranteed and this gives us an accurate prediction of the breakdown PSNR which we also relate to the sampling rate and the distance between adjacent Diracs. Simulation results validate the reliability of our analysis.
Spike sorting at sub-Nyquist rates

Spike sorting relies on the ability to establish the temporal occurrence of action potentials and their relation to specific neurons. Neural information is intrinsically compressible and as such suitable for sparse sampling. Potentially, this should allow for the use of multi-channel recordings, which is particularly advantageous to improve spike sorting. In this paper we propose a novel algorithm capable of sampling neural data at sub-Nyquist rates, yielding the same performance for spike sorting as traditional schemes.
Distributed compression of the plenoptic function

In this paper, we consider the problem of distributed compression in camera sensor networks. Due to the spatial proximity of the different cameras, acquired images can be highly dependent. The correlation in the visual information retrieved is related to the structure of the plenoptic function and can be estimated using geometrical information such as the position of the cameras and some bounds on the location of the objects. We propose a distributed compression scheme that takes advantage of this geometrical information in order to reduce the overall transmission rate from the sensors to a common central receiver. This new approach allows for a flexible repartition of the transmission bit-rates amongst the encoders and is optimal in many cases. Moreover, we show that our coding scheme can be made resilient to a fixed number of occlusions and that perfect reconstruction and interpolation are possible at the receiver.
Identifying a multiple plane plenoptic function from a swiped image

Blur in images, caused by camera motion with an open shutter, is usually thought of as a problem. The algorithm described in this paper shows instead that it is possible to use the blur caused by the integration of light rays at different locations along a moving camera trajectory to extract information about the light rays that are present within the scene. Retrieving the light rays present within a scene from different viewpoints is equivalent to retrieving the plenoptic function of the scene. In this paper, we focus on a specific case in which the blurred image of a scene, containing fronto-parallel planes with uniform unknown textures, is analysed to recreate the plenoptic function. The image is captured by a digital single lens camera with shutter open, moving in a straight line between two points, resulting in a swiped image. We estimate the EPI from this blurred image, and the EPI can be used to generate unblurred images for a given camera location.
The graph FRI framework-spline wavelet theory and sampling on circulant graphs

The objective of this work is to consider sparse representations of certain classes of signals on circulant graphs, by introducing families of graph wavelets which possess vanishing (exponential) moment properties. In light of this, we propose a novel framework of sampling and perfect reconstruction of sparse and wavelet-sparse signals on circulant graphs, which we denote as the Graph FRI framework, as an extension to the traditional discrete case. Given the dimensionality-reduced GFT of a sparse signal on a graph G, we can perfectly reconstruct the latter, while inferring a distinct down-sampling pattern and the structure of the associated coarsened graph through decomposition of the GFT-basis as the product between a coefficient matrix C and the multiresolution filtering operation with a low-pass graph e-spline filter. Hereby, we demonstrate that for a sufficiently banded adjacency matrix A of G, the obtained coarse graph preserves the original generating set S of G in a scheme of spectral sampling with respect to the original eigenbasis of A.
Geometry-Driven Distributed Compression of the Plenoptic Function: Performance Bounds and Constructive Algorithms

In this paper, we study the sampling and the distributed compression of the data acquired by a camera sensor network. The effective design of these sampling and compression schemes requires, however, the understanding of the structure of the acquired data. To this end, we show that the a priori knowledge of the configuration of the camera sensor network can lead to an effective estimation of such structure and to the design of effective distributed compression algorithms. For idealized scenarios, we derive the fundamental performance bounds of a camera sensor network and clarify the connection between sampling and distributed compression. We then present a distributed compression algorithm that takes advantage of the structure of the data and that outperforms independent compression algorithms on real multiview images.
Subspace-based methods for image registration and super-resolution

Super-resolution algorithms combine multiple low resolution images into a single high resolution image. They have received a lot of attention recently in various application domains such as HDTV, satellite imaging, and video surveillance. These techniques take advantage of the aliasing present in the input images to reconstruct high frequency information of the resulting image. One of the major challenges in such algorithms is a good alignment of the input images: subpixel precision is required to enable accurate reconstruction. In this paper, we give an overview of some subspace techniques that address this problem. We first formulate super-resolution in a multichannel sampling framework with unknown offsets. Then, we present three registration methods: one approach using ideas from variable projections, one using a Fourier description of the aliased signals, and one using a spline description of the sampling kernel. The performance of the algorithms is evaluated in numerical simulations.
A closed-form expression for the bandwidth of the plenoptic function under finite field of view constraints

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we examine the bandwidth of the plenoptic function when both the field of view and the scene width are finite. This analysis is carried out on two planar Lambertian scenes, a fronto-parallel plane and a slanted plane, and in both cases the texture is bandlimited. We derive an exact closed-form expression for the plenoptic spectrum of a slanted plane with sinusoidal texture. We show that in both cases the finite constraints lead to band-unlimited spectra. By determining the essential bandwidth, we derive a sampling curve that gives an adequate camera spacing for a given distance between the scene and the camera line.
Rate-distortion optimal tree algorithms for piecewise polynomials

In this paper, we are interested in a coding scheme, which achieves oracle like asymptotic rate-distortion (R-D) behavior with polynomial complexity for 1-D as well as 2-D signals. In particular, for the 1-D case we present a coding scheme which utilizes binary tree segmentation with optimal bit allocation among different segments. Investigation of the algorithm reveals the inherent weakness in the initial coding scheme, leading to a suboptimal performance. The optimal binary tree scheme in the 1-D case can be easily extended to the 2-D case as an optimal quadtree scheme with the similar computational complexity. The proposed optimal quadtree scheme also achieves the oracle like R-D performance for some simple classes of images whereas for the 2-D case there is no known algorithm achieving the right R-D behavior at reasonable computational cost.
Sensing and Communication With and Without Bits

The successful design of sensor network architectures depends crucially on the structure of the sampling, observation, and communication processes. One of the most fundamental questions concerns the sufficiency of discrete approximations in time, space, and amplitude. In the case of space and time, the question can be rephrased as whether there is a spatio-temporal sampling theorem for typical data sets in sensor networks. This question has a positive answer in many cases of interest. The issue of discretization of amplitudes is more subtle and can be expressed as the question of whether there is a (source/channel) separation theorem for typical sensor networks. We show that this question has a negative answer in general and that the price of separation can be large. To illustrate these issues, we review the underlying theory and discuss specific examples
Image based rendering with depth cameras: How many are needed?

Image based rendering is a technique for producing arbitrary viewpoints of a scene using multiple images instead of exact object models. The recent emergence of low-price, fast, and reliable cameras for measuring depth makes possible the augmentation of traditional color images with depth images. This combination promises to improve the rendering quality of an arbitrary viewpoint and thus have a great impact on IBR. A key issue is to understand, for any particular scene of interest, how many depth images and how many color images are necessary in order to obtain good rendering results. In this paper, using a framework akin to the plenoptic function, we perform a spectral analysis of multi-view depth images in order to determine the relationship between the number of depth and color images required. Our analysis is then validated using both synthetic and real images.
Localization of diffusive sources using spatiotemporal measurements

Consider a diffusion field induced by a finite number of localized and instantaneous sources. In this paper, we study the problem of estimating these sources from the spatiotemporal samples taken by a network of spatially distributed sensors. We start by considering the case of a single source but with unknown activation time. We show that the diffusion field at any spatial location is a scaled and shifted version of a common prototype function, and that this function is the unique solution to a particular differential equation. This observation leads to an efficient algorithm that can estimate the unknown parameters of the source by solving a system of linear equations. Following this algorithm we then present an acquisition strategy based around the Strang-Fix condition to obtain the appropriate measurements from the sensors. We then extend this result to the case of multiple sources with different activation time. For the algorithm proposed in this work, the minimum number of sensors required is d + 1, where d is the spatial dimension of the field. This requirement is independent of the number of active sources.
Rate-distortion optimized tree-structured compression algorithms for piecewise polynomial images

This paper presents novel coding algorithms based on tree-structured segmentation, which achieve the correct asymptotic rate-distortion (R-D) behavior for a simple class of signals, known as piecewise polynomials, by using an R-D based prune and join scheme. For the one-dimensional case, our scheme is based on binary-tree segmentation of the signal. This scheme approximates the signal segments using polynomial models and utilizes an R-D optimal bit allocation strategy among the different signal segments. The scheme further encodes similar neighbors jointly to achieve the correct exponentially decaying R-D behavior (D(R)/spl sim/c/sub 0/2/sup -c1R/), thus improving over classic wavelet schemes. We also prove that the computational complexity of the scheme is of O(NlogN). We then show the extension of this scheme to the two-dimensional case using a quadtree. This quadtree-coding scheme also achieves an exponentially decaying R-D behavior, for the polygonal image model composed of a white polygon-shaped object against a uniform black background, with low computational cost of O(NlogN). Again, the key is an R-D optimized prune and join strategy. Finally, we conclude with numerical results, which show that the proposed quadtree-coding scheme outperforms JPEG2000 by about 1 dB for real images, like cameraman, at low rates of around 0.15 bpp.
Spike detection using FRI methods and protein calcium sensors: Performance analysis and comparisons

Fast and accurate detection of action potentials from neurophysiological data is key to the study of information processing in the nervous system. Previous work has shown that finite rate of innovation (FRI) theory can be used to successfully reconstruct spike trains from noisy calcium imaging data. This is due to the fact that calcium imaging data can be modeled as streams of decaying exponentials which are a subclass of FRI signals. Recent progress in the development of genetically encoded calcium indicators (GECIs) has produced protein calcium sensors that exceed the sensitivity of the synthetic dyes traditionally used in calcium imaging experiments. In this paper, we compare the suitability for spike detection of the kinetics of a new family of GECIs (the GCaMP6 family) with the synthetic dye Oregon Green BAPTA-1. We demonstrate the high performance of the FRI algorithm on surrogate data for each calcium indicator and we calculate the Cramér-Rao lower bound on the uncertainty of the position of a detected spike in calcium imaging data for each calcium indicator.
A Sampling Framework for Solving Physics-Driven Inverse Source Problems

Partial differential equations are central to describing many physical phenomena. In many applications, these phenomena are observed through a sensor network, with the aim of inferring their underlying properties. Leveraging from certain results in sampling and approximation theory, we present a new framework for solving a class of inverse source problems for physical fields governed by linear partial differential equations. Specifically, we demonstrate that the unknown field sources can be recovered from a sequence of, so called, generalized measurements by using multidimensional frequency estimation techniques. Next we show that—for physics-driven fields—this sequence of generalized measurements can be estimated by computing a linear weighted sum of the sensor measurements; whereby the exact weights (of the sums) correspond to those that reproduce multidimensional exponentials, when used to linearly combine translates of a particular prototype function related to the Green's function of our underlying field. Explicit formulas are then derived for the sequence of weights, which map sensor samples to the exact sequence of generalized measurements when the Green's function satisfies the generalized Strang-Fix condition. Otherwise, the same mapping yields a close approximation of the generalized measurements. Based on this new framework, we develop practical, noise robust, sensor network strategies for solving the inverse source problem, and then present numerical simulation results to verify their performance.
Reverse engineering of signal acquisition chains using the theory of sampling signals with finite rate of innovation

This paper presents a novel theoretical framework for the reverse engineering of signal acquisition chains. We investigate how signals are transformed through a chain of signal acquisition and reconstruction stages. The signals, at different stages in the chain, are modelled using the theory of sampling signals with Finite Rate of Innovation (FRI). The model allows us to determine the chain structure and corresponding acquisition history from unknown query signals and to retrieve important parameters relating to the acquisition chain.
Local Feature Extraction for Image Super-Resolution

The problem of image super-resolution from a set of low resolution multiview images has recently received much attention and can be decomposed, at least conceptually, into two consecutive steps as: registration and restoration. The ability to accurately register the input images is key to the success and the quality of image super-resolution algorithms. Using recent results from the sampling theory for signals with finite rate of innovation (FRI), we propose in this paper a new technique for subpixel extraction from low resolution images of local features like step edges and corners for image registration. By exploiting the knowledge of the sampling kernel, we are able to locate exactly the step edges on synthetic images. We also present results of full frame super-resolution of real low resolution images using our registration technique. We obtain super-resolved images with a much improved visual quality compared to using a standard local feature detection approach like a subpixel Harris corner detector.
Sampling signals with finite rate of innovation in the presence of noise

Recently, it has been shown that it is possible to sample non-bandlimited signals that possess a limited number of degrees of freedom and uniquely reconstruct them from a finite number of uniform samples. These signals include, amongst others, streams of Diracs. In this paper, we investigate the problem of estimating the innovation parameters of a stream of Diracs from its noisy samples taken with polynomial or exponential reproducing kernels. For the one-Dirac case, we provide exact expressions for the Cramér- Rao bounds of this estimation problem. Furthermore, we propose methods to reconstruct the location of a single Dirac that reach the optimal performance given by the unbiased Cramér-Rao bounds down to noise levels of 5dB.
Adaptive plenoptic sampling

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we present a method of non-uniformly sampling a scene, with a smoothly varying surface, given a finite number of samples. This method approximates such a scene with a set of slanted planes subject to the constraint of finite number of samples. We use the recent spectral analysis of a single slanted plane to determine a piecewise constant spatial sampling rate for the scene. Finally, we show that this sampling rate results in a non-uniform sampling scheme that reconstructs the plenoptic function beyond that of uniform sampling.
Transform coder identification

The widespread popularity of transform coding has made it central to a wide range of methods in forensics, quality assessment and digital restoration. However, most approaches require prior knowledge of the transform coding parameters. In this paper, we consider the challenging problem of identifying the transform matrix as well as the quantization step sizes of a transform coder, given a set of P non-overlapping N-dimensional vectors observed as its output. We formulate the problem in terms of finding the lattice with the largest determinant that contains all observed vectors and we propose an algorithm that is able to find the optimal solution. Our experimental analysis shows that the probability of success of the algorithm quickly approaches 1 for small values of (P - N). The complexity of the proposed algorithm grows linearly with the dimensionality N.