nils-herrmann/clean_scirepeval huggingface dataset:
Spatial just noticeable distortion profile for image in DCT domain

In this paper, a DCT based JND model for monochrome pictures is proposed. This model incorporates the spatial contrast sensitivity function (CSF), the luminance adaptation effect and the contrast masking effect based on block classification. Gamma correction is also considered to compensate the original luminance adaptation effect which gives more accurate results. Moreover, a psychophysical experiment was designed to parameterize our model. Experimental results show that the proposed model is consistent with the human visual system. Compared with the other JND profiles, the proposed model can tolerate more distortion and has much better perceptual quality. The proposed JND model can be easily applied in many related areas, such as compression, watermarking, error protection, perceptual distortion metric, and so on.
Enhancing compression rate by just-noticeable distortion model for H.264/AVC

In this paper, we proposed an algorithm to enhance compression rate for the H.264/AVC. The properties of human visual system are utilized to reduce the bit rate without introducing noticeable visual distortion. The just-noticeable distortion (JND) threshold for every block in a frame is computed, and non-zero coefficients are removed if their magnitudes are less than the JND thresholds. Experimental results show that bit rate can be reduced by as much as 50% without degradation in visual quality.
Joint texture-shape optimization for MPEG-4 multiple video objects

In this paper, we present a uniform framework for optimal multiple video object bit allocation in MPEG-4. We combine the rate-distortion (R-D) models for the texture and shape information of arbitrarily shaped video objects to develop the joint texture-shape R-D models. The dynamic programming technique is applied to optimize the bit allocation for the multiple video objects. The simulation results demonstrate that the proposed joint texture-shape optimization algorithm outperforms the MPEG-4 verification model on the decoded picture quality.
Objective Quality Assessment of Image Retargeting by Incorporating Fidelity Measures and Inconsistency Detection.

The tremendous growth in mobile devices has resulted in huge generation and usage of digital images. Image quality assessment is thus an important issue for mobile media applications. In this paper, we focus on the quality evaluation of images generated by content-aware image retargeting, in which the reference and the distorted images are of different sizes. Through retargeting, many types of deformation inconsistency lead to shape distortion, deformation artifacts, and content information loss, worsening its perceptual quality. The deformation inconsistency occurs on different levels of the retargeted images. Limited by the accuracy of the alignment between the original and retargeted images, previous methods only focus on pixel-level and patch-level fidelity analyses and fail to detect deformation inconsistency. In this paper, we improve the alignment algorithm and propose a three-level representation of the retargeting process. Based on the analysis of this three-level representation, both fidelity measures and inconsistency detection are combined to determine the final retargeting quality. The proposed algorithm is validated on the public data sets RetargetMe and CUHK. Experimental results demonstrate that inconsistency detection contributes to accurately assessing the image retargeting perceptual quality. This inspires us to investigate more about deformation inconsistency to formulate the objective quality of image retargeting.
Two-Layer Directional Transform for High Performance Video Coding

This paper presents a directional transform scheme for coding interprediction errors in block-based hybrid video coding. It proposes a two-layer transform structure, where the first layer uses discrete wavelet transform to compact the residue energy to the LL band and then the second layer uses 2-D nonseparable directional transforms to deal with the arbitrary edge directions in the four subbands. By doing this, the edges in a macroblock are efficiently compacted to a few coefficients and at the same time the overhead used to indicate the transform directions is affordable. Experimental results show that the proposed scheme provides peak signal-to-noise ratio gain up to 0.46 dB, compared with H.264/AVC High Profile.
Strongly connected component multi-object tracking

Multi-object tracking (MOT) continues to gain more attention due to its relevance in the field of computer vision. Tracking-by-detection is one of the most used techniques in multi-object tracking, and this work follows that trend. Many methods generate tracklets in the initial step of tracking and generate final trajectories by optimization algorithms. An important challenge of such approaches is how to generate reliable tracklets which is crucial for generating final trajectories. Bounding box overlap and optical flow are two of such methods. A new clustering method is proposed for tracklet generation in our multi-object tracking algorithm. We present a novel approach that uses strongly connected component clusters in a K-NN directed graph with no defined class number for tracklet generation. We propose that such clusters of detections can accurately represent the tracklets of targets in a video. Binary Integer Programming is used to merge initial tracklets into their final trajectories. Experimental results on four standard benchmark datasets demonstrate the superiority of the proposed method.
Interworking video services with layered coding

Interworking between different video services is a critical factor in the success of a video communication system. A layered coding scheme which is well suited for use in a multimedia environment is described. The concept of layered coding, the coding algorithm and its coding efficiency are discussed. The idea of flexible layering is introduced, which, when combined with flexible picture formats, becomes an extremely powerful technique for interworking of video communications. Efficient interworking between video services in point-to-point or point-to-multipoint calls requires the capabilities of an appropriate communication network, such as broadband ISDN (B-ISDN).<<ETX>>
Subtractive impairment, additive impairment and image visual quality

In this paper, we propose an engineering-based image quality metric which distinguishes subtractive impairment from additive impairment. Since the amount of subtractive impairment is up-bounded by the total details within the reference image but the same limitation can't be applied to additive impairment, intuitively visual quality degradation due to the two types of impairments should be measured differently. In the proposed metric, subtractive and additive impairments are separated and represented in the wavelet domain, and their influences to image visual quality is measured by different equations. We tested the proposed metric on five subjectively-rated databases and proved its effectiveness in objective image quality assessment.
Adaptive cosine transform coding of images in perceptual domain

An adaptive cosine transform coding scheme for color images which incorporates human visual properties into the coding scheme is described. It employs adaptive quantization to exploit the statistical nature of the coefficients and adaptive block distortion equalization to reduce the block edge structures inherent in block transform coding schemes. Results show that the subjective quality of the reconstructed images at a bit rate of 0.4 bit/pixel or a compression ratio of 60:1 is very good.
Advanced rate control technologies for 3D-HDTV

This paper presents advanced rate control technologies for 3D-HDTV. We remodel the quadratic rate-distortion model for 3D multiview sequences based on their picture types. The proposed method consists of two levels for more accurate bits rate control. In a frame level, it sets quantization parameters based on HVS (human visual system) and reduces error bits by remodeling rate-quantization. In a MB level, rate control is activated to provide stricter buffer regulations and higher bit rate encoding based on the target bits and the quantization parameters calculated at the frame level. The proposed algorithm shows improvements in rate control as well as in PSNR compared to the conventional methods. It also provides more accurate rate control and higher image quality than TM5 and TMN8 at various bit rates.
Depth estimation and view synthesis for narrow-baseline video

In this paper, we propose the depth estimation and view synthesis approaches for the scene rendering of narrow-baseline videos. Depth estimation is performed by global energy minimization using graph cut technique. A novel smoothness energy term is proposed to enforce powerful smoothness of the estimated depth. In view synthesis, smart hole filling is proposed to efficiently remove the burr artifacts along the object boundary. Experimental results show that compared with the MPEG reference software, our method can generate more accurate depth maps and achieve much better objective (PSNR) and subjective quality in view synthesis.
3-D direction aligned wavelet transform for scalable video coding

This paper presents a novel 3-D direction aligned wavelet transform for scalable video coding. A new generalized 3- D directional threading technique is seamlessly incorporated into 3-D weighted adaptive lifting (WAL)-based wavelet transform to exploit the spatio-temporal correlation inside the video cube along the 3-D directional trajectory, leading to a new class of algorithm called 3-D direction aligned wavelet transform (DAWT). Experimental results show that the proposed 3-D WAL-based DAWT achieves better coding performance than the conventional 3-D discrete wavelet transform (3-D DWT).
Study of subjective and objective quality assessment of retargeted images

This paper presents the result of a recent large-scale subjective study of image retargeting quality on a collection of images generated by several representative image retargeting methods. Owning to many approaches to image retargeting that are developed, there is a need for a diverse independent public database of the retargeted images and the corresponding subjective scores that is freely available. We build an image retargeting quality database, in which 171 retargeted images (obtained from 57 natural source images of different contents) were generated by several representative image retargeting methods. The perceptual quality of each image is evaluated by at least 30 human subjects and the mean opinion scores (MOS) were recorded. Furthermore, several publicly available quality metrics for the retargeted images are evaluated on the built database. The database is made available  to the research community in order to further research on the perceptual quality assessment of the retargeted images.
Efficient H.264/AVC Video Coding with Adaptive Transforms

Transform has been widely used to remove spatial redundancy of prediction residuals in the modern video coding standards. However, since the residual blocks exhibit diverse characteristics in a video sequence, conventional transform methods with fixed transform kernels may result in low efficiency. To tackle this problem, we propose a novel content adaptive transform framework for the H.264/AVC-based video coding. The proposed method utilizes pixel rearrangement to dynamically adjust the transform kernels to adapt to the video content. In addition, unlike the traditional adaptive transforms, the proposed method obtains the transform kernels from the reconstructed block, and hence it consumes only one logic indicator for each transform unit. Moreover, a spiral-scanning method is developed to reorder the transform coefficients for better entropy coding. Experimental results on the Key Technical Area (KTA) platform show that the proposed method can achieve an average bitrate reduction of about 7.95% and 7.0% under all-intra and low-delay configurations, respectively.
Dynamic Bit Allocation for Multiple Video Object Coding

In MPEG-4, a visual scene may be treated as a composition of video objects and coded at object level. Such a flexible video coding framework makes it possible to code different video objects with different priority according to human perceptual characteristics. In this paper, we introduce a novel dynamic bit allocation framework to improve the subjective quality in such an object-based video coding system. We incorporate the rate distortion models with the dynamic priorities of the video objects and jointly encode video objects to minimize the weighted distortion within the bit budget constraint. We guarantee the human-interested video objects a better reconstructed quality by using the weighted bit allocation strategy in favour of the video objects with higher priority. To obtain the priority automatically, we apply a visual attention model. Comparing with traditional bit allocation algorithms, the objective quality of the object with higher priority is significantly improved under this framework. These results demonstrate the usefulness of this dynamic bit allocation framework
Edge-preserving regularization of disparity and motion fields

This paper describes a procedure for edge-preserving regularization of displacement fields. Initial displacement fields are regularized using a cost function that considers a smoothness term. Edge values are used as the weights in the smoothness term so that the discontinuities at edges are well preserved. The regularization technique is tested for both block-based and pixel-based cases for one-dimensional disparity fields and two-dimensional motion fields. The performance of our proposed algorithm is evaluated in aspects of entropy and distortion.
Disparity estimation with edge-based matching and interpolation

Stereo matching is one of the most active research areas in computer vision. In this paper, we present a disparity estimation algorithm using edge matching and spatial interpolation algorithm on color information. Firstly, we select some one to one corresponding points as seed points, then propagate them to edge map. Color based Gaussian weighted spatial interpolation is utilized on edge disparity map to obtain the final disparity map, which partly resolves the problem of occlusion. The performance of the algorithm is illustrated by simulations carried out on some pairs of images.
Complexity-Controllable Video Coding with Spatial Scalability

In this paper, a complexity-controllable video coding system is proposed with spatial scalability. The time complexity models of the encoder and decoder are investigated and the major complexity control parameters, such as the number of macroblocks, Intra prediction modes, the number of partitions in Inter prediction, the ratio of non-zero blocks, etc., are carefully analyzed for their controllability in the encoder and decoder. Based on the proposed time complexity model and the complexity-scalable video coding/decoding schemes, an appropriate coding scheme can be selected according to the system capability with performance tradeoff, which is beneficial for the video system design in mobile devices.
Scale- and Affine-Invariant Fan Feature

Most existing feature detectors assume no surface discontinuity within the keypoints' support regions and, hence, have little chance to match the keypoints located on or near the surface boundaries. These keypoints, though not many, are salient and representative. In this paper, we show that they can be successfully matched by using the proposed scale- and affine-invariant Fan features. Specifically, the image neighborhood of a keypoint is depicted by multiple fan subregions, namely Fan features, to provide robustness to surface discontinuity and background change. These Fan features are made scale-invariant by using the automatic scale selection method based on the Fan Laplacian of Gaussian (FLOG). Affine invariance is further introduced to the Fan features based on the affine shape diagnosis of the mirror-predicted surface patch. The Fan features are then described by Fan-SIFT, which is an extension of the famous scale-invariant feature transform (SIFT) descriptor. Experimental results of quantitative comparisons show that the proposed Fan feature has good repeatability that is comparable to the state-of-the-art features for general structured scenes. Moreover, by using Fan features, we can successfully match image structures near surface discontinuities despite significant scale, viewpoint, and background changes. These structures are complementary to those found by the traditional methods and are especially useful for describing weakly textured scenes, which is demonstrated in our experiments on image matching and object rendering.
Guided Face Cartoon Synthesis

In this paper, we propose a new method, called guided synthesis, to synthesize a face cartoon from a face photo. The guided synthesis is defined as a local linear model, which generates a cartoon image by incorporating the content of guidance images taken from the training set. Our synthesis operation is achieved based on four weight functions. The first is a photo-photo weight that aims to measure the similarity between an input photo patch and a training photo patch. The second is defined as a photo-cartoon weight, which is used to compute the likelihood by computing the similarity between a cartoon patch and an input photo patch. The third weight is defined in the synthesized photos, which is to set a smoothness constraint between neighboring synthesized patches. The final weight is designed to evaluate the similarity of a synthesized patch to an input patch based on the spatial distance. Experimental evaluation on a number of face photos demonstrates the good performance of the proposed method on the face cartoon synthesis.
Salient object segmentation using a switch scheme

In this paper, we propose a novel switch scheme and a saliency map binarization method for salient object segmentation. With the proposed switch scheme, the saliency map can be segmented by different methods according to its quality, which is evaluated by a method proposed in this paper. We also develop a binarization method by integrating three properties of the salient object. This method exclusively derives information from the saliency map (i.e., without referring to the original image). Experimental results demonstrate that the proposed binarization method can generate better segmentation results and the switch scheme can further improve the segmentation results by fully exploiting the merit of both segmentation methods.
Segmentation and Tracking of Moving objects for Content-Based Video Coding, "IEE

To enable content-based functionalities in video coding, a decomposition of the scene into physical objects is required. Such objects are normally not characterised by homogeneous colour, intensity, or optical flow. Therefore, conventional techniques based on these low-level features cannot perform the desired segmentation. The authors address segmentation and tracking of moving objects and present a new video object plane (VOP) segmentation algorithm that extracts semantically meaningful objects. A morphological motion filter detects physical objects by identifying areas that are moving differently from the background. A new filter criterion is introduced that measures the deviation of the estimated local motion from the synthesised global motion. A two-dimensional binary model is derived for the object of interest and tracked throughout the sequence by a Hausdorff object tracker. To accommodate for rotations and changes in shape, the model is updated every frame by a two-stage method that accounts for rigid and non-rigid moving parts of the object. The binary model then guides the actual VOP extraction, whereby a novel boundary post-processor ensures high boundary accuracy. Experimental results demonstrate the performance of the proposed algorithm.
Two-dimensional transform domain decimation techniques

Decimation is normally carried out in two passes; lowpass filtering and subsampling where the latter is normally performed in the time domain. This paper describes a technique whereby both the operations can be combined in the transform domain. The two-dimensional decimation scheme is first implemented in the discrete Fourier transform domain and then extended to the discrete cosine transform domain. It is further applied to a non-sinusoidal i.e. Hadamard transform domain. The effect of using ideal filter transfer function in transform domain decimation on the quality of the decimated images is investigated. This approach results in greater computational efficiency as the constraint of filter length to meet certain specifications is removed permitting the use of smaller transform block sizes.
A new approach in the object-oriented coding at low bit rates

The object-oriented coding technique segments each image of a video sequence into a number of moving objects and compensates the motion of each object. Thus it can reconstruct real motion better than the block-based coding algorithm at low bit rates. It uses mapping parameters to estimate motion information of each object. To obtain the good quality of a reconstructed image at a low bit rate, we introduce a new algorithm to the object-oriented coding algorithm by using variable length coding technique for the foreground regions after detecting the changed regions. In addition, the border information obtained from our segmentation technique is used to compress the first frame of the video sequence. Computer simulation results with a test sequence show that the proposed method achieves better results than the H.261 standard both in the quantitative measurements at 64 kbits/sec.
3-D Object-Based Scalable Wavelet Video Coding With Boundary Effect Suppression

This paper extends the lifting-based motion threading (MTh) technique from the frame-based coding to the object-based coding, attracted by the unique advantages of the object-based coding that do not exist in other coding schemes. The boundary effects, which exist in spatial and temporal wavelet transforms due to the manner of object-based coding, are well suppressed by 3-D shape-adaptive discrete wavelet transform (3D SA-DWT) via the lifting implementation in a unified framework. Based on the proposed object-based MTh technique, a 3-D object-based scalable wavelet video coder is developed. Experimental results show that the proposed coder achieves good performance compared with the existing object-based coders and provides the scalability functionality at the same time
Perceptual adaptive Lagrangian multiplier for high efficiency video coding

In high efficiency video coding (HEVC), Lagrangian rate distortion optimization (RDO) technique is used to optimize the rate distortion (RD) performance. However, the corresponding Lagrangian multiplier does not consider the perceptual characteristic of the input video and thus is not effective for perceptual video coding. To address this problem, an efficient perceptual adaptive Lagrangian multiplier for HEVC is proposed. Based on the human visual system (HVS) observation that the region with less perceptual sensitivity can tolerate more distortion, the Lagrangian multiplier is adaptively adjusted for each coding tree unit (CTU) based on its perceptual sensitivity so that the perceptual quality of the reconstructed video can be improved. The above-mentioned perceptual sensitivity for each CTU is measured according to two perceptual features-spatial energy ratio and temporal motion activity. Experimental results have shown that the proposed method is able to significantly improve the perceptual RD performance, compared with the original HEVC.