nils-herrmann/clean_scirepeval huggingface dataset:
An efficient hole filling for depth image based rendering

An efficient hole filling algorithm is presented providing texture compilation for invisible pixels during depth image based rendering (DIBR). Some pixels cannot be assigned to proper texture values due to occluding foreground object; thus, compilation is required to obtain visually pleasing virtual views from a set of reference views. The proposed algorithm exploits reliable texture transition from visible pixels to the holes among horizontal and vertical directions. Transition is performed by successive weighted summation (SWS) based on color and depth similarities between the neighboring pixels, producing adaptively weighted and connected aggregation. Depending on local depth variation, assignment of proper texture to the hole pixels is achieved by combining horizontal and vertical information. According to the experimental results, proposed approach yields visually pleasing compilation of occluded regions with drastically decreased number of operations per pixel.
Segmentation in multi-view video via color, depth and motion cues

In the light of dense depth map estimation, motion estimation and object segmentation, the research on multi-view video (MW) content has becoming increasingly popular due to its wide application areas in the near future. In this work, object segmentation problem is studied by additional cues due to depth and motion fields. Segmentation is achieved by modeling images as graphical models and performing popular normalized cuts method with some modifications. In the graphical models, each node is represented by a group of pixels, instead of individual pixels, which are obtained as a result of over-segmentation of the images. These over-segmented regions are also utilized in the dense depth map estimation step; in which 3D planar models are assigned for each of these sub-regions. Moreover, optical flow is estimated based on afline motion assumption for these regions. The links of the graphical models are weighted according to the depth, motion and color similarities of the pixel groups due to these regions. Once the links are obtained, segmentation is achieved by recursively bi-partitioning the graph via removing the weak links. Experiments indicate that the proposed framework achieves precise segmentation results for MVV sequences.
Watermarking for Image Based Rendering via homography-based virtual camera location estimation

The recent advances in image based rendering (IBR) have pioneered freely determining the viewing position and angle in a scene from multi-view video. Noting that a person could also record a personal video for this arbitrarily selected view and misuse this content, apparently, copyright and copy protection problems also exist and should be solved for IBR applications. In our recent work , we have proposed a watermarking method, which embeds the watermark pattern into every frame of multi- view video and extracts this watermark from a rendered image, generated by the nearest-interpolation based light-field rendering (LFR). This paper presents a novel solution for the challenging problem of watermark detection in bilinear interpolation, namely the most attractive and promising interpolation method for LFR-based applications. Moreover, the location of the virtual camera could be completely arbitrary in this new formulation. The results show that the watermark could be extracted successfully for LFR via bilinear interpolation for any imagery camera location and rotation, as long as the visual quality of the rendered image is preserved.
Object segmentation in multi-view video via color, depth and motion cues

In the light of dense depth map estimation, motion estimation and object segmentation, the research on multi-view video (MVV) content has becoming increasingly popular due to its wide application areas in the near future. In this work, object segmentation problem is studied by additional cues due to depth and motion fields. Segmentation is achieved by modeling images as graphical models and performing popular Normalized Cuts method with some modifications. In the graphical models, each node is represented by a group of pixels, instead of individual pixels, which are obtained as a result of over-segmentation of the images. These over-segmented regions are also utilized in the dense depth map estimation step; in which 3D planar models are assigned for each of these sub-regions. Moreover, optical flow is estimated based on affine motion assumption for these regions. The links of the graphical models are weighted according to the depth, motion and color similarities of the pixel groups due to these regions. Once the links are obtained, segmentation is achieved by recursively bi-partitioning the graph via removing the weak links. Experiments indicate that the proposed framework achieves precise segmentation results for MVV sequences.
Parameter Estimation in TV Image Restoration Using Variational Distribution Approximation

In this paper, we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. Within the hierarchical Bayesian formulation, the reconstructed image and the unknown hyperparameters for the image prior and the noise are simultaneously estimated. The proposed algorithms provide approximations to the posterior distributions of the latent variables using variational methods. We show that some of the current approaches to TV-based image restoration are special cases of our framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyperparameters and clearly outperform existing methods when additional information is included.
Total Variation Image Restoration and Parameter Estimation using Variational Posterior Distribution Approximation

In this paper we propose novel algorithms for total variation (TV) based image restoration and parameter estimation utilizing variational distribution approximations. By following the hierarchical Bayesian framework, we simultaneously estimate the reconstructed image and the unknown hyper parameters for both the image prior and the image degradation noise. Our algorithms provide an approximation to the posterior distributions of the unknowns so that both the uncertainty of the estimates can be measured and different values from these distributions can be used for the estimates. We also show that some of the current approaches to TV-based image restoration are special cases of our variational framework. Experimental results show that the proposed approaches provide competitive performance without any assumptions about unknown hyper parameters and clearly outperform existing methods when additional information is included.
Interventional MRI with sparse sampling using union-of-subspaces

A significant problem in interventional magnetic resonance imaging is limited imaging speed. This paper addresses this problem using a new signal model known as union-of-subspaces. This model enables an effective use of sparse sampling and prior information to significantly improve the imaging speed. The proposed method has been validated using simulations on real interventional imaging data, and is shown to provide high speed and quality imaging. The approach is very flexible and can be applied to other imaging applications as well.
Bayesian Compressive Sensing Using Laplace Priors

In this paper, we model the components of the compressive sensing (CS) problem, i.e., the signal acquisition process, the unknown signal coefficients and the model parameters for the signal and noise using the Bayesian framework. We utilize a hierarchical form of the Laplace prior to model the sparsity of the unknown signal. We describe the relationship among a number of sparsity priors proposed in the literature, and show the advantages of the proposed model including its high degree of sparsity. Moreover, we show that some of the existing models are special cases of the proposed model. Using our model, we develop a constructive (greedy) algorithm designed for fast reconstruction useful in practical settings. Unlike most existing CS reconstruction methods, the proposed algorithm is fully automated, i.e., the unknown signal coefficients and all necessary parameters are estimated solely from the observation, and, therefore, no user-intervention is needed. Additionally, the proposed algorithm provides estimates of the uncertainty of the reconstructions. We provide experimental results with synthetic 1-D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.
Lossless Hyperspectral-Image Compression Using Context-Based Conditional Average

In this paper, a new algorithm for lossless compression of hyperspectral images is proposed. The spectral redundancy in hyperspectral images is exploited using a context-match method driven by the correlation between adjacent bands. This method is suitable for hyperspectral images in the band-sequential format. Moreover, this method compares favorably with the recent proposed lossless compression algorithms in terms of compression, with a lower complexity.
Reference-driven MR image reconstruction with sparsity and support constraints

The problem of reconstructing an MR image from limited (and sparsely sampled) k-space data in the presence of a reference image occurs in various applications, including interventional imaging and dynamic contrast-enhanced imaging. This paper addresses the problem using a dictionary composed of three types of basis functions: reference-weighted harmonic functions, wavelets, and pixel/voxel indicator functions. These bases are efficient for representing different image features such as global and local contrast changes from the reference to the target image as well as localized novel image features. The proposed image model and the associated reconstruction algorithm are described. Simulation results are also included to illustrate the improved performance of the proposed method over conventional compressed sensing type reconstruction methods.
Video retrieval using sparse Bayesian reconstruction

Every day, a huge amount of video data is generated for different purposes and applications. Fast and accurate algorithms for efficient video search and retrieval are therefore essential. The interesting properties of sparse representation and the new sampling theory named Compressive Sensing (CS) constitute the core of the new approach to video representation and retrieval we are presenting in this paper. Once the representation (where sparsity is expected) has been chosen and the observations have been taken, the proposed approach utilizes Bayesian modeling and inference to tackle the retrieval problem. In order to speed up the inference process the use of Principal Components Analysis (PCA) to provide an alternative representation of the frames is analyzed. Experimental results validate the proposed approach whose robustness against noise is also examined.
Fast bayesian compressive sensing using Laplace priors

In this paper we model the components of the compressive sensing (CS) problem using the Bayesian framework by utilizing a hierarchical form of the Laplace prior to model sparsity of the unknown signal. This signal prior includes some of the existing models as special cases and achieves a high degree of sparsity. We develop a constructive (greedy) algorithm resulting from this formulation where necessary parameters are estimated solely from the observation and therefore no user-intervention is needed. We provide experimental results with synthetic 1D signals and images, and compare with the state-of-the-art CS reconstruction algorithms demonstrating the superior performance of the proposed approach.
Accelerating cardiovascular imaging by exploiting regional low-rank structure via group sparsity

Sparse sampling of (k, t)-space has proved useful for cardiac MRI. This paper builds on previous work on using partial separability (PS) and spatial-spectral sparsity for high-quality image reconstruction from highly undersampled (k, t)-space data. This new method uses a more flexible control over the PS-induced low-rank constraint via group-sparse regularization. A novel algorithm is also described to solve the corresponding (1,2)-norm regularized inverse problem. Reconstruction results from simulated cardiovascular imaging data are presented to demonstrate the performance of the proposed method.
Fast total variation image restoration with parameter estimation using bayesian inference

In this paper we propose two fast Total Variation (TV) based algorithms for image restoration by utilizing variational posterior distribution approximation. The unknown image and the hyperparameters for the image and observation models are formulated and estimated simultaneously within a hierachical Bayesian framework, rendering the algorithms fully-automated without any free parameters. Experimental results demonstrate that the proposed algorithms provide restoration results competitive to existing methods in terms of image quality while achieving superior computational efficiency.
Total variation super resolution using a variational approach

In this paper we propose a novel algorithm for super resolution based on total variation prior and variational distribution approximations. We formulate the problem using a hierarchical Bayesian model where the reconstructed high resolution image and the model parameters are estimated simultaneously from the low resolution observations. The algorithm resulting from this formulation utilizes variational inference and provides approximations to the posterior distributions of the latent variables. Due to the simultaneous parameter estimation, the algorithm is fully automated so parameter tuning is not required. Experimental results show that the proposed approach outperforms some of the state-of-the-art super resolution algorithms.
Using the Kullback-Leibler divergence to combine image priors in Super-Resolution image reconstruction

This paper is devoted to the combination of image priors in Super Resolution (SR) image reconstruction. Taking into account that each combination of a given observation model and a prior model produces a different posterior distribution of the underlying High Resolution (HR) image, the use of variational posterior distribution approximation on each posterior will produce as many posterior approximations as priors we want to combine. A unique approximation is obtained here by finding the distribution on the HR image given the observations that minimizes a linear convex combination of the Kullback-Leibler divergences associated with each posterior distribution. We find this distribution in closed form and also relate the proposed approach to other prior combination methods in the literature. The estimated HR images are compared with images provided by other SR reconstruction methods.
Sparse Bayesian image restoration

In this paper we propose a novel Bayesian algorithm for image restoration and parameter estimation. We utilize an image prior where Gaussian distributions are placed per pixel in the high-pass filter outputs of the image. By following the hierarchical Bayesian framework, we simultaneously estimate the unknown image and hyperparameters for both the image prior and the image degradation noise. We show that the proposed formulation is a special case of the popular lp-norm based formulations with p = 0, and therefore enforces sparsity to an high extent in the filtered image coefficients. Moreover, the proposed formulation results in a convex optimization problem, and therefore does not suffer from the robustness issues common with non-convex image priors. Experimental results demonstrate that the proposed algorithm provides superior performance compared to state-of-the-art restoration algorithms although no user-supervision is required.
Bayesian Group-Sparse Modeling and Variational Inference

In this paper, we present a general class of multivariate priors for group-sparse modeling within the Bayesian framework. We show that special cases of this class correspond to multivariate versions of several classical priors used for sparse modeling. Hence, this general prior formulation is helpful in analyzing the properties of different modeling approaches and their connections. We derive the estimation procedures with these priors using variational inference for fully Bayesian estimation. In addition, we discuss the differences between the proposed inference and deterministic inference approaches with these priors. Finally, we show the flexibility of this modeling by considering several extensions such as multiple measurements, within-group correlations, and overlapping groups.
Compressive sensing of light fields

We propose a novel camera design for light field image acquisition using compressive sensing. By utilizing a randomly coded non-refractive mask in front of the aperture, incoherent measurements of the light passing through different regions are encoded in the captured images. A novel reconstruction algorithm is proposed to recover the original light field image from these acquisitions. Using the principles of compressive sensing, we demonstrate that light field images with high angular dimension can be captured with only a few acquisitions. Moreover, the proposed design provides images with high spatial resolution and signal-to-noise-ratio (SNR), and therefore does not suffer from limitations common to existing light-field camera designs. Experimental results demonstrate the efficiency of the proposed system.
Reference-guided sparsifying transform design for compressive sensing MRI

Compressive sensing (CS) MRI aims to accurately reconstruct images from undersampled k-space data. Most CS methods employ analytical sparsifying transforms such as total-variation and wavelets to model the unknown image and constrain the solution space during reconstruction. Recently, nonparametric dictionary-based methods for CS-MRI reconstruction have shown significant improvements over the classical methods. These existing techniques focus on learning the representation basis for the unknown image for a synthesis-based reconstruction. In this paper, we present a new framework for analysis-based reconstruction, where the sparsifying transform is learnt from a reference image to capture the anatomical structure of unknown image, and is used to guide the reconstruction process. We demonstrate with experimental data the high performance of the proposed approach over traditional methods.
Image prior combination in super-resolution image registration & reconstruction

In this paper the application of image prior combinations to the Bayesian Super Resolution (SR) image registration and reconstruction problem is studied. A sparse image prior based on the horizontal and vertical first order differences is combined with a non-sparse SAR prior. Since, for a given observation model, each prior produces a different posterior distribution of the underlying High Resolution (HR) image, the use of variational posterior distribution approximation on each posterior will produce as many posterior approximations as priors we want to combine. A unique approximation is obtained here by finding the distribution on the HR image given the observations that minimize a linear convex combination of the Kullback-Leibler (KL) divergences associated with each posterior distribution. We find this distribution in closed form. The estimated HR images are compared with images provided by other SR reconstruction methods.
Low-rank matrix completion by variational sparse Bayesian learning

There has been a significant interest in the recovery of low-rank matrices from an incomplete of measurements, due to both theoretical and practical developments demonstrating the wide applicability of the problem. A number of methods have been developed for this recovery problem, however, a principled method for choosing the unknown target rank is generally missing. In this paper, we present a recovery algorithm based on sparse Bayesian learning (SBL) and automatic relevance determination principles. Starting from a matrix factorization formulation and enforcing the low-rank constraint in the estimates as a sparsity constraint, we develop an approach that is very effective in determining the correct rank while providing high recovery performance. We provide empirical results and comparisons with current state-of-the-art methods that illustrate the potential of this approach.
Bayesian Simultaneous Sparse Approximation With Smooth Signals

In the simultaneous sparse approximation problem, several latent vectors corresponding to independent random realizations from a common sparsity profile are recovered from an undercomplete set of measurements. In this paper, we address an extension of this problem, where in addition to the common sparsity profile, the vectors of interest are assumed to have a high correlation among each other. Specifically, we consider the case when the non-zero rows in the combined latent signal vectors are considered to be temporally smooth signals. We present a Bayesian formulation of the problem and develop a greedy inference algorithm based on sparse Bayesian learning for independent observations. A difficulty is that unlike for existing greedy methods, there is no closed form expression for the maximizer of the objective function in the greedy algorithm when row correlations are introduced. We derive two methods to maximize the objective function, one based on the EM algorithm and another on a fixed-point iteration. Empirical results show that the proposed method provides better reconstruction results compared to existing methods, especially when the signal-to-noise ratio is low and the latent signal vectors are highly correlated. We also demonstrate the application of the proposed method to source localization in magnetoencephalography, where it obtains a temporally smooth solution with accurate localization of the brain activity.
Variational Bayesian Methods For Multimedia Problems

In this paper we present an introduction to Variational Bayesian (VB) methods in the context of probabilistic graphical models, and discuss their application in multimedia related problems. VB is a family of deterministic probability distribution approximation procedures that offer distinct advantages over alternative approaches based on stochastic sampling and those providing only point estimates. VB inference is flexible to be applied in different practical problems, yet is broad enough to subsume as its special cases several alternative inference approaches including Maximum A Posteriori (MAP) and the Expectation-Maximization (EM) algorithm. In this paper we also show the connections between VB and other posterior approximation methods such as the marginalization-based Loopy Belief Propagation (LBP) and the Expectation Propagation (EP) algorithms. Specifically, both VB and EP are variational methods that minimize functionals based on the Kullback-Leibler (KL) divergence. LBP, traditionally developed using graphical models, can also be viewed as a VB inference procedure. We present several multimedia related applications illustrating the use and effectiveness of the VB algorithms discussed herein. We hope that by reading this tutorial the readers will obtain a general understanding of Bayesian methods and establish connections among popular algorithms used in practice.