nils-herrmann/clean_scirepeval huggingface dataset:
A novel mode decision for depth map coding in 3D-AVS

In this paper, a new mode decision scheme is proposed for depth map coding in 3D-AVS. The novelty of the paper mainly contains the following two points. Firstly, an improved distortion estimation model of synthesized views is proposed. Secondly, for the mode decision of depth map coding, the distortion is represented to be the weighted sum of depth distortion and estimated distortion of the synthesized view. We proposed a new scheme to derive the weighting factors adaptively based on the disparity. Then the distortion is utilized to calculate the rate distortion cost for mode decision. Experimental results demonstrate that the proposed scheme achieves remarkable performance improvement in 3D-AVS. The average BD-rate gain is about 12%.
Spatial-temporal residue network based in-loop filter for video coding

Deep learning has demonstrated tremendous break through in the area of image/video processing. In this paper, a spatial-temporal residue network (STResNet) based in-loop filter is proposed to suppress visual artifacts such as blocking, ringing in video coding. Specifically, the spatial and temporal information is jointly exploited by taking both current block and co-located block in reference frame into consideration during the processing of in-loop filter. The architecture of STResNet only consists of four convolution layers which shows hospitality to memory and coding complexity. Moreover, to fully adapt the input content and improve the performance of the proposed in-loop filter, coding tree unit (CTU) level control flag is applied in the sense of rate-distortion optimization. Extensive experimental results show that our scheme provides up to 5.1% bit-rate reduction compared to the state-of-the-art video coding standard.
Adaptive Motion Vector Resolution Scheme for Enhanced Video Coding

In the state-of-the-art H.265/HEVC video coding standard, the motion vector is always fixed to be 1/4-pixel resolution for the entire video sequence regardless of the different video contents, which is not efficient for prediction coding. In this paper, we propose a frame level adaptive motion vector resolution selection scheme based on a rate-distortion model in terms of motion vector resolution. In the proposed rate-distortion model, the relationship between the distortion and the motion vector resolution is approximated with a linear model. And a rate model of motion vector is built, which reflects the relationship between the coding bits of motion vector and its value. With the proposed rate-distortion model, an optimal motion vector resolution minimizing the total rate-distortion cost will be selected for each frame. Experimental results show that the proposed scheme can achieve 1.5%, 1.3% and 2.5% BD-rate gain on average for Random Access, Lowdelay-B and Lowdelay-P configurations without complexity increment.
Depth Estimation of Video Sequences With Perceptual Losses

3-D vision plays an important role in intelligent perception of robot, while it requires extra 3-D sensors. Depth estimation from monocular videos provides an alternative mechanism to recover the 3-D information. In this paper, we propose an unsupervised learning framework that uses the perceptual loss for depth estimation. Depth and pose networks are first trained to estimate the depth and the camera motion of the video sequence, respectively. With the estimated depth and pose of the original frame, the adjacent frame can be reconstructed. The pixel-wise differences between the constructed frame and the original frame are used as per-pixel loss. Meanwhile, reconstructed views and original views can be used to extract advanced features from a pre-trained network to define and optimize perceptual loss functions to assess the quality of reconstructions. We combine the respective advantages of these two methods and present an approach of generating a depth map by training the feed-forward network with per-pixel loss function and perceptual loss function. The experimental results show that our method can significantly improve the estimation accuracy of depth map.
An optimized probability estimation model for binary arithmetic coding

In this paper, we analyze the binary arithmetic coding of High Efficiency Video Coding (HEVC) and the second generation of audio and video coding standard (AVS2). Then an optimized probability estimation scheme is proposed for arithmetic coder. The proposed scheme is incorporated into the HEVC reference software (HM 16.0) and AVS2 reference software (RD 10.1). Experimental results demonstrate that the proposed scheme can efficiently improve the coding efficiency of entropy coding. The rate-distortion (R-D) performance gain can be up to 0.21% for AVS2 and 0.30% for HEVC respectively.
Image guided label map propagation in video sequences

In this paper, we propose a novel method to transmit the label maps by propagating from a key frame to non-key frames. The label map of a non-key frame is initialized by warping the label map of its corresponding key frame according to the motion estimation between them. Subsequently, the initialized label map is optimized with the guidance of its texture image. The optimization process minimizes an energy function which takes two constraints into consideration: (i) the data term measuring the similarity between an estimated label map and its initialized one, (ii) the regularization term enforcing the local smoothness in the label map and the consistency of region boundaries between the estimated label map and its corresponding texture image. Graph cuts based computation process is finally performed to generate the optimized label map. Experimental results show that our method achieves higher accuracy and better visual quality comparing with the state-of-the-art method.
Video compressive sensing via structured Laplacian modelling

Seeking a fair domain in which the signal can exhibit high sparsity is of essential significance in compressive sensing (CS). Most methods in the literature, however, use a fixed transform domain or prior information, which cannot adapt to various video contents. In this paper, we propose a video CS recovery algorithm based on the structured Laplacian model, which can effectually deal with the non-stationarity of natural videos. To build the model, structured patch groups are constructed according to the nonlocal similarity in a temporal scope. By incorporating the model into the CS paradigm, we can formulate an ℓ1-norm optimization problem, for which a solution based on the iterative shrinkage/thresholding algorithms (ISTA) is designed. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in both objective and subjective recovery quality.
Parallel intra coding for HEVC on CPU plus GPU platform

In High Efficiency Video Coding (HEVC), the intra coding performance is significantly improved due to the recursive splitting structure and up to 35 intra prediction modes. However, the computational complexity of intra coding increases largely as well. In this paper, a fast intra coding scheme is proposed based on CPU and GPU cooperation. Firstly, the intra prediction of variable blocks is performed in parallel on multi-cores GPU. Secondly, the intra prediction mode with minimum Sum of Absolute Difference (SAD) cost is selected and transmitted to the host CPU. Instead of exhaustively searching all the intra modes in Rough Mode Decision (RMD) process, the mode returned by the GPU is directly selected. Lastly, the texture gradient of each coding unit (CU) is assessed during parallel intra prediction, then used by the CPU for fast CU size decision. Experiment results show that the proposed parallel intra coding method achieves up to 62% complexity reduction with acceptable coding performance loss.
Framework of AVS2-video coding

The first generation of Audio-video coding standard (AVS1) will become an international standard of IEEE (IEEE.1857). The second generation of Audio-video coding standard (AVS2) is a successor to IEEE.1857 which targets to higher coding efficiency, especially for high resolution videos, with reasonable complexity increase. Currently, AVS2 working draft is developed, in which symbolic techniques of IEEE.1857, such as logarithmic arithmetic coding, are still used. It also introduces new coding tools, e.g. flexible partition structure which is based on macro-block structure and high efficient prediction methods utilizing more texture information and temporal redundancies, etc. The performance of AVS2 is improved by 37% compared to the IEEE.1857 Jiaqiang Profile and 16.4% compared to H.264 in average. This paper provides an overview of the framework of AVS2-video coding standard.
An accurate algorithm for color to gray and back

This paper proposes an accurate reversible algorithm to convert full color images to gray images with keeping chroma and spatial resolution. In our previous study, we proposed a reversible color-to-gray algorithm using the two-level Haar wavelet transform with a packet assignment technique. The authors improve the algorithm by devising a color embedding technique. The proposed algorithm distributes color information effectively in the one-level wavelet subbands for preserving chroma and spatial resolution. Experimental results show that the proposed algorithm can improve the recovering accuracy of color images from the textured gray images of the original color images. We also present a practical application to a color-shielded image transmission which can permit a color recovery only to a specific user.
Color image recovery system from printed gray image

This paper develops a color image recovery system from a color-embedded gray image. A reversible color-to-gray algorithm is an effective technique from several viewpoints such as running cost, data quantity, and security in practical applications. Most of the conventional algorithms work only on electronic gray images in the closed computer world, and do not work on real-world materials through the printing and capturing processes. We challenge this important problem and construct a color image recovery system from a printed gray image captured by a monochrome camera. The image geometry, gray level, and resolution change through the printing and capturing processes. We present a solution method to recovery color image by using a calibration pattern attached to the gray image. Experimental results show that the proposed system can dynamically recover a color image from a printed gray image that is set under a monochrome camera.
Surface Identification Using the Dichromatic Reflection Model

The author describes a method based on the dichromatic reflection model for identifying object surfaces. The surface spectral reflectance function of an inhomogeneous object is described as the sum of a constant interface (specular) reflectance and a body (diffuse) reflectance under all illumination geometries. The interface component is used to estimate the spectral power distribution of the illuminant without using a reference white standard, whereas the body component is used as the principal indication of the surface identity. The body reflectance function of each surface is recovered. A method to classify the observed reflectances is developed, and an algorithm to estimate a body reflectance function, unique to each surface, from the classified reflectances is proposed. The author shows the reliability of the surface classification method and the accuracy of estimated body reflectance function.
Spatio-velocity contrast sensitivity functions and video quality assessment

This paper describes achromatic/chromatic spatio-velocity contrast sensitivity functions (SV-CSFs) and their application to video quality assessment (VQA). SV-CSF is a modulation transfer function of the human visual system, and consists of the relationship among visual contrast sensitivities, spatial frequencies and velocities of objects. For simulating perceived videos, SV-CSFs are applied to filter original and distorted videos. The criteria in our VQA method are obtained by calculating CIELAB color differences between filtered videos. From the experimental results for the validation, it was shown that the proposed VQA method is a more efficient VQA method than conventional methods.
Image Analysis and Synthesis of Skin Color Textures by Wavelet Transform

The present paper proposes a skin color texture analysis method based on wavelet transform for detecting different texture patterns including local non-periodic features. We modeled skin color texture as the composition of four texture components of base color, internal skin texture, regular surface texture and local texture. The skin color image is decomposed to the four texture components by multi-resolution analysis using wavelet transform. A variety of skin images with different conditions of skin color and texture are created in a linear combination of the texture components. Experimental results show good separation of skin textures by wavelet analysis and realistic synthesized images
Estimation of the Torrance-Sparrow reflection model from a single multi-band image

A method is proposed for estimating various reflection parameters from images of an object taken by a multi-band CCD camera. The Torrance-Sparrow model is used for modeling light reflection on an object surface composed of inhomogeneous dielectric materials. First, we examine the validity of the Torrance-Sparrow model based on the spectral reflectance data. Next, we propose algorithms for estimating model parameters from images by a six-color camera. The parameters estimated from a single image are: 1) surface-spectral reflectance, 2) illuminant spectral distribution, 3) illumination direction, 4) surface roughness, and 5) ratio of body to interface intensity. The feasibility of the proposed method is demonstrated in an experiment using a painted object. The estimation accuracy of the whole model is confirmed based on computer graphics images. The method can be applied to objects with general shapes.
Region segmentation by multispectral imaging

This paper describes a method for region segmentation of a circuit board based on surface-spectral reflectance measurements by using a multispectral imaging system. The elements on a board are composed of different materials such as metal, paint, and inhomogeneous dielectric. First, we introduce the camera system for machine inspection of circuit boards. Second, an algorithm is proposed for obtaining reliable estimates of the spectral reflectances for a raw circuit board. Third, a reflectance analysis method, is presented for segmenting the reflectance image into five element regions of: (1) a substrate, (2) footprints, (3) resist-coated metals, (4) non-resist metals, and (5) silk-screen prints. The reflectance images observed at two illumination directions are used for identifying rough metallic surfaces. Finally, the feasibility of the method is shown in an experiment.
An HDR Spectral Imaging System for Time-Varying Omnidirectional Scene

Omni directional spectral imaging technology is crucial in many fields including computer vision and image analysis. It should be noted that the dynamic range of luminance values in natural scenes is quite large, and the scenes are time-varying since objects and light sources change dynamically. This paper proposes a system for acquiring high dynamic range (HDR) spectral images for time-varying Omni directional natural scenes. The system is constructed using two programmable high-speed video cameras with wide angle lenses and a programmable rotating table. Two different types of color filters are mounted on the two color video cameras for six band image acquisition. We present several algorithms for HDR image synthesis, lens distortion correction, image registration, and Omni directional image synthesis. Spectral power distributions of illuminants (color signals) are recovered from the captured six-band images based on the Wiener estimation algorithm. The performances of the proposed system are discussed in detail in terms of the system configurations, acquisition time, artifacts, and spectral estimation accuracy. Experimental results in actual scenes show that the proposed system is feasible and powerful for acquiring HDR spectral images of time-varying Omni directional scenes.
Analysis of experimental curves using singular value decomposition

This paper describes a singular value decomposition (SVD) technique for the analysis of experimental response curves. The SVD of a set of curves shows that curves can be decomposed to become a linear combination of some intrinsic-component patterns. The interpolation of curves can then be reduced to a simple interpolation of weighting coefficients. The SVD method is applied to an estimation problem of γ-ray response spectra, and a library of the response curves is constructed.
Estimating Reflection Parameters from a Single Color Image

This article proposes a method for estimating various parameters of a reflection model from a single color image of an object. We used an RGB charge-coupled device (CCD) camera for imaging and parameter estimation. We assumed the object surface is composed of an inhomogeneous dielectric material and used the Phong model to describe the surface's dichromatic reflection. This approach also generalizes to objects with smooth convex surfaces.
Behavior prediction from trajectories in a house by estimating transition model using stay points

In this paper we propose a novel method for predicting resident's behaviors in a house from one's movement trajectories. The method consists of 1) segmentation of trajectory data into staying or moving and classification of the segments and 2) prediction by time-series association rules from transition events of each segment. The method predicts the start time of target behaviors for daily life support, such as eating, taking a bath etc. The time lag between the prediction and the target behavior can be set up manually, thus the method is adaptable to a variety of supporting systems. The experimental results using real residents' trajectory data of almost two years demonstrate that prediction of behaviors by the proposed method is feasible.
A spectral-imaging system and algorithms for recovering spectral functions

Knowledge of such spectral functions as the surface-spectral reflectances of objects and the illuminant spectral power distribution is needed for realizing color constancy, accurate color reproduction, and rendering realistic images. This paper describes a spectral-imaging system using a liquid-crystal tunable filter and algorithms for recovering the spectral functions of illuminant and surface reflectance from the image data. First, we introduce a filtering mechanism and the camera system. Next, the camera outputs and the spectral functions are represented by finite-dimensional linear models. The algorithms for estimating the spectral functions are then presented using the image data. Finally the usefulness of the proposed imaging system and algorithms is shown in an experiment.
Reflectance-based material classification for printed circuit boards

This paper describes a method for classifying object materials on a raw circuit board based on surface-spectral reflectance. First we introduce a multi-spectral imaging system for observing tiny objects and capturing their spectral data. The imaging system is composed of a liquid-crystal tunable filter, a monochrome CCD camera, macro-lens and a personal computer. We describe how we can estimate the spectral reflectance functions of object surfaces by using the multi-spectral imaging system. We show that dielectric materials like plastics can be distinguished from metals based on the reflectance difference in changing illumination geometries. Then an algorithm is presented for classifying the objects into several circuit elements based on the estimated spectral-reflectances. Region segmentation results of the circuit board are demonstrated in an experiment using a real board. The performance of the proposed imaging system and algorithms is examined in comparison with the RGB-based methods using a normal color camera.
Spectral Invariant Representation for Spectral Reflectance Image

Although spectral images contain large amount of information, compared with color images, the image acquisition is affected by several factors such as shading and specular highlight. Many researchers have introduced color invariant and spectral invariant representations for these factors using the standard dichromatic reflection model of inhomogeneous dielectric materials. However, these representations are inadequate for other materials like metal. This paper proposes a more general spectral invariant representation for obtaining reliable spectral reflectance images. Our invariant representation is derived from the standard dichromatic reflection model for dielectric materials and the extended dichromatic reflection model for metals. We proof the invariant formulas for spectral images of most natural objects preserve spectral information and are invariant to highlights, shading, surface geometry, and illumination intensity. The method is applied to the problem of material classification and image segmentation of a raw circuit board. Experiments are done with real spectral images to examine the performance of the proposed method.
Adaptive Filtering for Color Image Sharpening and Denoising

Both tasks of sharpening and denoising color images are eternity problems for improving image quality. This paper proposes a filtering algorithm which brings a dramatic improvement in sharpening effects and reducing flat-area noises for natural color images. First, we generate a luminance slope map, which classifies a color image into the edge areas and the flat areas with uniform color. Second, the Gaussian derivative filter is applied to sharpening each segmented edge area. In order to keep color balance, the edge-sharpening filter is applied only to the luminance component of the color image. In the flat areas, a filter based on the SUSAN algorithm is applied to reducing the background noises with preserving sharpened edges. The performance of the proposed method is verified for natural image with Gaussian noise and impulse noise.
Imaging and rendering of oil paintings using a multi-band camera

The paper proposes a method for the imaging and rendering of art paintings using a multi-band camera system. The surface shape of an art painting is considered as a rough plane rather than a three-dimensional curved surface. First. we present an algorithm for estimating the surface normal at each pixel point, based on a photometric stereo without using a rangefinder. Next, an algorithm is presented for estimating the spectral reflectance function from a set of pixel values acquired at different illumination directions. Then, the surface reflectance and normal data are used for estimating the light reflection properties. The Torrance-Sparrow model is used for model fitting and parameter estimation. Finally, an experiment using an oil painting is executed for demonstrating the feasibility of the proposed method.
Object recognition by multi-spectral imaging with a liquid crystal filter

We describe a spectral-imaging system with a liquid-crystal tunable filter and algorithms for recovering the spectral functions of the illuminant and surface reflectance from the image data. The filter works as a polarization-interference filter with birefringent elements and linear polarizers. We define the camera outputs by a vector equation and describe the spectral functions by a linear model. Then algorithms are presented for estimating the spectral functions from the image data. An effective image-processing algorithm is proposed for highlight extraction and region segmentation. Experimental estimation of the illuminant and spectral reflectances shows the efficiency in measurement and the accuracy in estimation of the spectral functions.