nils-herrmann/clean_scirepeval huggingface dataset:
Structure and texture filling-in of missing image blocks in wireless transmission and compression

An approach for filling-in blocks of missing data in wireless image transmission is presented in this paper. When compression algorithms such as JPEG are used as part of the wireless transmission process, images are first tiled into blocks of 8/spl times/8 pixels. When such images are transmitted over fading channels, the effects of noise can kill entire blocks of the image. Instead of using common retransmission query protocols, we aim to reconstruct the lost data using correlation between the lost block and its neighbors. If the lost block contained structure, it is reconstructed using an image inpainting algorithm, while texture synthesis is used for the textured blocks. The switch between the two schemes is done in a fully automatic fashion based on the surrounding available blocks. The performance of this method is tested for various images and combinations of lost blocks. The viability of this method for image compression, in association with lossy JPEG, is also discussed.
A contrario hierarchical image segmentation

Hierarchies are a powerful tool for image segmentation, they produce a multiscale representation which allows to design robust algorithms and can be stored in tree-like structures which provide an efficient implementation. These hierarchies are usually constructed explicitly or implicitly by means of region merging algorithms. These algorithms obtain the segmentation from the hierarchy by either using a greedy merging order or by cutting the hierarchy at a fixed scale. Our main contribution is to enlarge the search space of these algorithms to the set of all possible partitions spanned by a certain hierarchy, and to cast the segmentation as a selection problem within this space. The importance of this is two-fold. First, we are enlarging the search space of classic greedy algorithms and thus potentially improving the segmentation results. Second, this space is considerably smaller than the space of all possible partitions, thus we are reducing the complexity. In addition, we embed the selection process on a statistical a contrario framework which allows us to reduce the number of free parameters of our algorithm to only one.
Color Stabilization Along Time and Across Shots of the Same Scene, for One or Several Cameras of Unknown Specifications

We propose a method for color stabilization of shots of the same scene, taken under the same illumination, where one image is chosen as reference and one or several other images are modified so that their colors match those of the reference. We make use of two crucial but often overlooked observations: first, that the core of the color correction chain in a digital camera is simply a multiplication by a 3 × 3 matrix; second, that to color-match a source image to a reference image we do not need to compute their two color correction matrices, it is enough to compute the operation that transforms one matrix into the other. This operation is a 3 × 3 matrix as well, which we call H. Once we have H, we just multiply by it each pixel value of the source and obtain an image which matches in color the reference. To compute H we only require a set of pixel correspondences, we do not need any information about the cameras used, neither models nor specifications or parameter values. We propose an implementation of our framework which is very simple and fast, and show how it can be successfully employed in a number of situations, comparing favorably with the state of the art. There is a wide range of applications of our technique, both for amateur and professional photography and video: color matching for multicamera TV broadcasts, color matching for 3D cinema, color stabilization for amateur video, etc.
Structure and texture filling-in of missing image blocks in wireless transmission and compression applications

An approach for filling-in blocks of missing data in wireless image transmission is presented. When compression algorithms such as JPEG are used as part of the wireless transmission process, images are first tiled into blocks of 8 x 8 pixels. When such images are transmitted over fading channels, the effects of noise can destroy entire blocks of the image. Instead of using common retransmission query protocols, we aim to reconstruct the lost data using correlation between the lost block and its neighbors. If the lost block contained structure, it is reconstructed using an image inpainting algorithm, while texture synthesis is used for the textured blocks. The switch between the two schemes is done in a fully automatic fashion based on the surrounding available blocks. The performance of this method is tested for various images and combinations of lost blocks. The viability of this method for image compression, in association with lossy JPEG, is also discussed.
Fusion-Based Variational Image Dehazing

We propose a novel image-dehazing technique based on the minimization of two energy functionals and a fusion scheme to combine the output of both optimizations. The proposed fusion-based variational image-dehazing (FVID) method is a spatially varying image enhancement process that first minimizes a previously proposed variational formulation that maximizes contrast and saturation on the hazy input. The iterates produced by this minimization are kept, and a second energy that shrinks faster intensity values of well-contrasted regions is minimized, allowing to generate a set of difference-of-saturation (DiffSat) maps by observing the shrinking rate. The iterates produced in the first minimization are then fused with these DiffSat maps to produce a haze-free version of the degraded input. The FVID method does not rely on a physical model from which to estimate a depth map, nor it needs a training stage on a database of human-labeled examples. Experimental results on a wide set of hazy images demonstrate that FVID better preserves the image structure on nearby regions that are less affected by fog, and it is successfully compared with other current methods in the task of removing haze degradation from faraway regions.
Variational problems and PDEs on implicit surfaces

A novel framework for solving variational problems and partial differential equations for scalar and vector-valued data defined on surfaces is introduced. The key idea is to implicitly represent the surface as the level set of a higher dimensional function, and solve the surface equations in a fixed Cartesian coordinate system using this new embedding function. The equations are then both intrinsic to the surface and defined in the embedding space. This approach thereby eliminates the need for performing complicated and inaccurate computations on triangulated surfaces, as is commonly done in the literature. We describe the framework and present examples in computer graphics and image processing applications, including texture synthesis, flow field visualization, as well as image and vector field intrinsic regularization for data defined on 3D surfaces.
Real-time, accurate depth of field using anisotropic diffusion and programmable graphics cards

Computer graphics cameras lack the finite depth of field (DOF) present in real world ones. This results in all objects being rendered sharp regardless of their depth, reducing the realism of the scene. On top of that, real-world DOF provides a depth cue, that helps the human visual system decode the elements of a scene. Several methods have been proposed to render images with finite DOF, but these have always implied an important trade-off between speed and accuracy. We introduce a novel anisotropic diffusion partial differential equation (PDE) that is applied to the 2D image of the scene rendered with a pin-hole camera. In this PDE, the amount of blurring on the 2D image depends on the depth information of the 3D scene, present in the Z-buffer. This equation is well posed, has existence and uniqueness results, and it is a good approximation of the optical phenomenon, without the visual artifacts and depth inconsistencies present in other approaches. Because both inputs to our algorithm are present at the graphics card at every moment, we can run the processing entirely in the GPU. This fact, coupled with the particular numerical scheme chosen for our PDE, allows for real-time rendering using a programmable graphics card.
A variational model for filling-in gray level and color images

A variational approach for filling-in regions of missing data in gray-level and color images is introduced in this paper. The approach is based on joint interpolation of the image gray-levels and gradient/isophores directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed solving the variational problem via its gradient descent flow, which lends to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist's principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given.
Morphing Active Contours

A method for deforming curves in a given image to a desired position in a second image is introduced. The algorithm is based on deforming the first image toward the second one via a partial differential equation (PDE), while tracking the deformation of the curves of interest in the first image with an additional, coupled PDE; both the images and the curves on the frame/slices of interest are used for tracking. The technique can be applied to object tracking and sequential segmentation. The topology of the deforming curve can change without any special topology handling procedures added to the scheme. This permits, for example, the automatic tracking of scenes where, due to occlusions, the topology of the objects of interest changes from frame to frame. In addition, this work introduces the concept of projecting velocities to obtain systems of coupled PDEs for image analysis applications. We show examples for object tracking and segmentation of electronic microscopy.
Simultaneous structure and texture image inpainting

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.
A Comprehensive Framework for Image Inpainting

Inpainting is the art of modifying an image in a form that is not detectable by an ordinary observer. There are numerous and very different approaches to tackle the inpainting problem, though as explained in this paper, the most successful algorithms are based upon one or two of the following three basic techniques: copy-and-paste texture synthesis, geometric partial differential equations (PDEs), and coherence among neighboring pixels. We combine these three building blocks in a variational model, and provide a working algorithm for image inpainting trying to approximate the minimum of the proposed energy functional. Our experiments show that the combination of all three terms of the proposed energy works better than taking each term separately, and the results obtained are within the state-of-the-art.
Contrast invariant inpainting with a 3rd order, optimal PDE

PDE-based image inpainting has become a very active area of research after the pioneering works of Masnou and Morel (1998), Bertalmio et al. (2000) and Ballester et al. (2001). In this paper we take a different approach, inspired by the excellent work of Caselles et al. (1998). We view the inpainting problem as a particular case of image interpolation in which we intend to propagate level lines. Expressing this in terms of local neighborhoods and using a Taylor expansion we derive a third order PDE that performs inpainting. This PDE is optimal in the sense that it is the most accurate third order PDE which can ensure continuation of level lines. It is also contrast invariant. This is a novel PDE, which both in its accuracy and contrast invariance outperforms the approaches cited above.
A Multi-Modal Approach to Perceptual Tone Mapping

We present an improvement of TSTM, a recently proposed tone mapping operator for High Dynamic Range (HDR) images, based on a multi-modal analysis. One of the key features of TSTM is a suitable implementation of the Naka-Rushton equation that mimics the visual adaptation performed by the human visual system coherently with Weber-Fechner’s law of contrast perception. In the present paper we use the Gaussian Mixture Model (GMM) in order to detect the modes of the log-scale luminance histogram of a given HDR image and then we use the information provided by GMM to properly devise a Naka-Rushton equation for each mode. Finally, we properly select the parameters in order to merge those equations into a continuous function. Tests and comparisons to show how this new method is capable of improving the performances of TSTM are provided and commented, as well as comparisons with state of the art methods.
Navier-stokes, fluid dynamics, and image and video inpainting

Image inpainting involves filling in part of an image or video using information from the surrounding area. Applications include the restoration of damaged photographs and movies and the removal of selected objects. We introduce a class of automated methods for digital inpainting. The approach uses ideas from classical fluid dynamics to propagate isophote lines continuously from the exterior into the region to be inpainted. The main idea is to think of the image intensity as a 'stream function for a two-dimensional incompressible flow. The Laplacian of the image intensity plays the role of the vorticity of the fluid; it is transported into the region to be inpainted by a vector field defined by the stream function. The resulting algorithm is designed to continue isophotes while matching gradient vectors at the boundary of the inpainting region. The method is directly based on the Navier-Stokes equations for fluid dynamics, which has the immediate advantage of well-developed theoretical and numerical results. This is a new approach for introducing ideas from computational fluid dynamics into problems in computer vision and image analysis.
Automatic, Fast and Perceptually Accurate Gamut Mapping Based on Vision Science Models

Gamut mapping transforms colors of the original (image or video) content to the color palette of the display device with the simultaneous goals of (a) reproducing content accurately while preserving the artistic intent of the original content’s creator and (b) exploiting the full color rendering potential of the target display device. The rapid advancement in display technologies has created a pressing need to develop automatic and fast gamut mapping algorithms that can deal with imagery intended for both conventional and emerging displays. In this paper, we propose a novel framework based on retinal and color perception models from vision science that offers a functionality to perform both gamut reduction and gamut extension, while preserving hue and taking into account the analysis of the colors of the input image. We evaluate the performance of the proposed framework visually and by using a perceptually-based error metric, according to which the gamut-mapped results of our framework outperform those of the state-of-the-art methods.
Perceptually-based Gamut Extension Algorithm for Emerging Wide Color Gamut Display and Projection Technologies

While wide color gamut (WCG) capabilities are a key element of emerging display and projection technologies, at present most image content is recorded using standards such as DCI-P3 for cinema or BT.709 for TV that have a reduced color gamut. Therefore, there is a need for gamut extension methods that process regular content and allow to appreciate the full color potential of new displays, improving user experience. We present a gamut extension algorithm that is based on visual perception models and explicitly takes into account low chromatic colors such as skin tones. It produces results that look natural, are free of artifacts of any kind, and outperform the state of the art. The method is fast, allowing for operation interaction if needed.
Fusion of Bracketing Pictures

When taking pictures in a night scene with artificial lighting, a very common situation, for most cameras the light coming from the scene is not enough and we are posed with the following problem: we can use the flash which yields a bright, sharp image but with colors quite different from the ones present in the real scene, or we can either increase the exposure setting so as to let the camera absorb more light, or just keep the normal (short) exposure setting and take a dark picture. If we discard the flash as an option and use instead the exposure bracketing feature available in most photo cameras, we obtain a series of pictures taken in rapid succession with different exposure times, with the implicit idea that the user picks from this set the better looking image. But it's a quite common situation that none of these images is good enough: in general, good color information is retained from longer exposure settings while sharp details are obtained from the shorter ones. In this work we propose a technique for automatically recombining a bracketed pair of images into a single picture that reflects the optimal properties of each one. The proposed technique uses stereo matching to extract the optimal color information from the overexposed image and Non-local means denoising to suppress noise while retaining sharp details in the luminance of the underexposed image.
A Decomposition Framework for Image Denoising Algorithms

In this paper, we consider an image decomposition model that provides a novel framework for image denoising. The model computes the components of the image to be processed in a moving frame that encodes its local geometry (directions of gradients and level lines). Then, the strategy we develop is to denoise the components of the image in the moving frame in order to preserve its local geometry, which would have been more affected if processing the image directly. Experiments on a whole image database tested with several denoising methods show that this framework can provide better results than denoising the image directly, both in terms of Peak signal-to-noise ratio and Structural similarity index metrics.
Geometry-Based Demosaicking

Demosaicking is a particular case of interpolation problems where, from a scalar image in which each pixel has either the red, the green or the blue component, we want to interpolate the full-color image. State-of-the-art demosaicking algorithms perform interpolation along edges, but these edges are estimated locally. We propose a level-set-based geometric method to estimate image edges, inspired by the image in-painting literature. This method has a time complexity of O(S) , where S is the number of pixels in the image, and compares favorably with the state-of-the-art algorithms both visually and in most relevant image quality measures.
Morphing active contours: a geometric approach to topology-independent image segmentation and tracking

A method for deforming curves in a given image to a desired position in a second image is introduced in this paper. The algorithm is based on deforming the first image toward the second one via a partial differential equation, while tracking the deformation of the curves of interest in the first image with an additional, coupled, partial differential equation. The technique can be applied to object tracking and slice-by-slice segmentation of 3D data. The topology of the deforming curve can change, without any special topology handling procedures added to the scheme. This permits for example the automatic tracking of scenes where, due to occlusions, the topology of the objects of interest changes from frame to frame.
An active regions approach for the segmentation of 3D biological tissue

Some of the most successful algorithms for the automated segmentation of images use an active regions approach, where a curve is evolved so as to maximize the disparity of its interior and exterior. But these techniques require the manual selection of several parameters, which make impractical the work with long image sequences or with a very dissimilar set of sequences. Unfortunately this is precisely the case with 3D biological image sequences. In this work we improve on previous active regions algorithms in two aspects: by introducing a way to compute and update the optimum weights for the different channels involved (color, texture, etc.) and by estimating if the moving curve has lost any object so as to launch a re-initialization step. Our method is shown to outperform previous approaches. Several examples of biological image sequences, quite long and different among themselves, are presented.
Variational Approach for the Fusion of Exposure Bracketed Pairs

When taking pictures of a dark scene with artificial lighting, ambient light is not sufficient for most cameras to obtain both accurate color and detail information. The exposure bracketing feature usually available in many camera models enables the user to obtain a series of pictures taken in rapid succession with different exposure times; the implicit idea is that the user picks the best image from this set. But in many cases, none of these images is good enough; in general, good brightness and color information are retained from longer-exposure settings, whereas sharp details are obtained from shorter ones. In this paper, we propose a variational method for automatically combining an exposure-bracketed pair of images within a single picture that reflects the desired properties of each one. We introduce an energy functional consisting of two terms, one measuring the difference in edge information with the short-exposure image and the other measuring the local color difference with a warped version of the long-exposure image. This method is able to handle camera and subject motion as well as noise, and the results compare favorably with the state of the art.