nils-herrmann/clean_scirepeval huggingface dataset:
Towards unsupervised attention object extraction by integrating visual attention and object growing

Content-related functionalities of image/video applications call for efficient tools that can automatically extract meaningful objects from images. However, traditional methods generally fail to capture objects of user interest because they totally neglect human visual attention perception. Aiming to address this problem, this study proposes a generic model for unsupervised extraction of viewer's attention objects from color images. We formulate the attention objects as a Markov random field (MRF). Then, the MRF is expressed in the form of a Gibbs random field with an energy function. The energy minimization that integrates visual attention and object growing provides a practical way to obtain attention objects. The proposed model works in a manner analogous to humans and has great promise to be a basic tool for content-based image/video applications. Experimental results show the effectiveness of the proposed model.
AVS video standard implementation for SoC design

AVS1-P2 is the newest video standard of Audio Video coding Standard (AVS) workgroup of China, which provides close performance to H.264/AVC main profile with lower complexity. In this paper, a platform independent software package is developed for AVS1-P2 decoder to facilitate embedded video codec development. In order to minimize the on-chip memory and save the time consumed in on-chip/off-chip data transfer, an MB-based architecture is developed by modifying the data flow, decoding hierarchy, and the buffer definition and management for low-level decoding kernels. Such system architecture provides over 80% reduction in on-chip memory compared to the frame-based architecture when decoding 720 p (1280times720) sequences. By modularizing the decoding kernels and data transfer modules, the proposed MB-based decoder facilitates the AVS video decoder development on the target platform.
Object segmentation from sparse views of wide-baseline images

In this paper, we propose an automatic approach to segment object from multiple sparse views of wide-baseline images. Depth and occlusion information recovered from multiple views assist the object initialization and segmentation processes. The initial object patch is extract based on a saliency map incorporating the depth and locality cues. We then formulate the object segmentation task as an energy minimization problem, which is solved by graph cut optimization. Based on the basic energy function, local background modeling, adaptive data fusion and 3D graph construction are developed to make the segmentation toward better results. Experimental results on self-recorded images and the benchmarks demonstrate the efficiency and robustness of the proposed approach.
Predictive classified vector quantization

A vector quantization scheme based on the classified vector quantization (CVQ) concept, called predictive classified vector quantization (PCVQ), is presented. Unlike CVQ where the classification information has to be transmitted, PCVQ predicts it, thus saving valuable bit rate. Two classifiers, one operating in the Hadamard domain and the other in the spatial domain, were designed and tested. The classification information was predicted in the spatial domain. The PCVQ schemes achieved bit rate reductions over the CVQ ranging from 20 to 32% for two commonly used color test images while maintaining the same acceptable image quality. Bit rates of 0.70-0.93 bits per pixel (bpp) were obtained depending on the image and PCVQ scheme used.
A rate distortion optimized transform for motion compensation residual

In this paper, we propose a rate distortion optimization based content adaptive transform method for motion compensation residuals. The proposed method utilizes pixel rearrangement to dynamically adjust the transform kernels to adapt to the residual content. Comparing with the traditional adaptive transforms, the highlight of this work is that it obtains the transform kernels from the decoded block, and hence it consumes only one overhead bit for each transform unit. Moreover, rate distortion optimization scheme is used to choose the best candidate kernels. Experimental results show that the proposed method achieves an average 0.35 dB gain of PSNR in comparison with the key technical areas (KTA) encoder.
Adaptive pre-interpolation filter for motion-compensated prediction

1 The proposed interpolation filter comprises two concatenating filters, adaptive pre-interpolation filter (APIF) and the normative interpolation filter in H.264/AVC. The former is applied only to the integer pixels in the reference frames; the latter generates all the sub-position samples, supported by the output of APIF. The convolution of APIF and the standard filter minimizes the motion prediction error on a frame basis. APIF preserves the merits of the adaptive interpolation filter (AIF) and the adaptive loop filter (ALF) in the key technical area (KTA) software and overcomes their drawbacks. The experimental results show that APIF has comparable or even better performance compared with the joint use of AIF and ALF.
A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment

In the field of objective image quality assessment (IQA), Spearman’s <inline-formula> <tex-math notation="LaTeX">$\rho$ </tex-math></inline-formula> and Kendall’s <inline-formula> <tex-math notation="LaTeX">$\tau$ </tex-math></inline-formula>, which straightforwardly assign uniform weights to all quality levels and assume that each pair of images is sortable, are the two most popular rank correlation indicators. These indicators can successfully measure the average accuracy of an IQA metric for ranking multiple processed images. However, two important perceptual properties are ignored. First, the sorting accuracy (<italic>SA</italic>) of high-quality images is usually more important than that of poor-quality images in many real-world applications, where only top-ranked images are pushed to the users. Second, due to the subjective uncertainty in making judgments, two perceptually similar images are usually barely sortable, and their ranks do not contribute to the evaluation of an IQA metric. To more accurately compare different IQA algorithms, in this paper, we explore a perceptually weighted rank correlation indicator, which rewards the capability of correctly ranking high-quality images and suppresses the attention toward insensitive rank mistakes. Specifically, we focus on activating a “valid” pairwise comparison of images whose quality difference exceeds a given sensory threshold (<italic>ST</italic>). Meanwhile, each image pair is assigned a unique weight that is determined by both the quality level and rank deviation. By modifying the perception threshold, we can illustrate the sorting accuracy with a sophisticated <italic>SA-ST</italic> curve rather than a single rank correlation coefficient. The proposed indicator offers new insight into interpreting visual perception behavior. Furthermore, the applicability of our indicator is validated for recommending robust IQA metrics for both degraded and enhanced image data.
Learning semantic concepts from user feedback log for image retrieval

To improve the performance of image retrieval systems, the well-known semantic gap needs to be bridged. Relevance feedback provides a strategy for learning semantic concepts from visual features. This paper reports a novel framework to learn semantic concepts from the accumulated user feedback log. The semantic concepts consist of two categories: explicit semantics and implicit semantics. The former can be directly estimated by analyzing the user-provided feedback log. The latter is learned according to the obtained explicit semantics. Finally, both explicit and implicit semantics are applied to an image retrieval system. Experiments on 10,000 images show the superiority of the proposed method
The perceptually transparent coding for image

In this paper, an improved transparent image coding algorithm is proposed, where the quantization factor can be tuned for each block according to the just-noticeable distortion (JND). The experimental results show that the images compressed by the proposed method are hardly distinguished from the original images. Therefore, the proposed method can achieve perceptually transparent quality for images. Moreover, the bitrate of the proposed algorithm is less than that of the state-of-the-arts lossless and near-lossless codecs.
Weighted Adaptive Lifting-Basedwavelet Transform

In this paper, we propose a new weighted adaptive lifting (WAL)-based wavelet transform that is designed to solve the problems existing in the previous adaptive directional lifting (ADL) approach. The proposed approach uses the weighted function to make sure that the prediction and update stages are consistent, the directional interpolation to improve the orientation property of interpolated image, and adaptive interpolation filter to adjust to statistical property of each image. Experimental results show that the proposed WAL-based wavelet transform for image coding outperforms the conventional lifting-based wavelet transform up to 3.02 dB in PSNR and significant improvement in subjective quality is also observed. Compared with the ADL approach, up to 1.18 dB improvement in PSNR is reported.
FaceSeg: Automatic Face Segmentation for Real-Time Video

Segmenting human faces automatically is very important for face recognition and verification, security system, and computer vision. In this paper, we present an accurate segmentation system for cutting human faces out from video sequences in real-time. First, a learning based face detector is developed to rapidly find human faces. To speed up the detection process, a face rejection cascade is constructed to remove most of negative samples while retaining all the face samples. Then, we develop a coarse-to-fine segmentation approach to extract the faces based on a min-cut optimization. Finally, a new matting algorithm is proposed to estimate the alpha-matte based on an adaptive trimap generation method. Experimental results demonstrate the effectiveness and robustness of our proposed method that can compete with the well-known interactive methods in real-time.
A Fast Rate-Distortion Optimization Algorithm for H.264/AVC

In H.264/AVC video coding standard, rate distortion optimization (RDO) is a very efficient technique to decide the coding mode of the macroblock and improves the coding performance greatly. But at the same time, RDO introduces a great deal of computation complexity because it needs to do the transform and entropy coding to get accurate distortion and bit-rate to calculate the Lagrange cost. In order to avoid the expensive computational cost, in this paper , a simple and accurate bit estimation model is proposed and used in RDO scheme. Combined with distortion measure method in transform domain and the fast intra mode RDO method, a lot of computation of RDO is reduced. Experimental results prove that our proposed fast RDO algorithm can reduce about 60% total encoding time and save about 76% computation time of RDO module with little degradation in coding performance.
Object segmentation from wide baseline video

In this paper, we propose an automatic approach to segment object from stereo videos, for which the viewpoints are widely apart. We first present a novel saliency analysis to emphasize the foreground object. The saliency map is estimated by combining the depth information recovered by feature matching and the boundary information revealed by color segmentation. The object mask is extracted initially based on the saliency map and then refined by graph-cut segmentation, where color and motion information are efficiently incorporated in both data and smoothness terms. Moreover, a background image is gradually reconstructed during video segmentation, based on which an additional constraint is imposed on the data term to further improve the video segmentation. The proposed method is tested on stereo videos with widely separated viewpoints and severe background clutters. Good experimental results demonstrate the feasibility of the proposed method.
Unsupervised Segmentation of Defocused Video Based on Matting Model

In this paper, an unsupervised segmentation algorithm based on matting model is proposed to extract the focused objects in the low depth of field (DOF) video images. The proposed algorithm is fully automatic and can be used to partition the video image into focused objects and defocused background. This method consists of three stages. The first stage is to generate the saliency map from the input image. In the second stage, bilateral and morphological filtering are employed to smooth and lift the saliency regions. Then a trimap with three regions is calculated by an adaptive thresholding method. The third stage involves the Poisson matting scheme to extract the boundaries of the focused objects accurately. Experimental evaluation on test sequences shows that the proposed method is capable of segmenting the focused region quite effectively and accurately.
Adaptive block-size transform based just-noticeable difference profile for videos

In this paper, we propose a novel adaptive block-size transform (ABT) based just-noticeable difference (JND) model for videos. Firstly, the ABT-based spatial JND profile is extended to spatial-temporal JND model for videos by considering temporal contrast sensitivity function (TCSF), eye movement, and the motion information of the objects in video sequence. Furthermore, a metric named motion characteristics distance (MCD) is proposed to depict the motion characteristics similarity between a macroblock and its corresponding sub-blocks. Based on the proposed MCD and the obtained spatial image content information, a novel balanced strategy is proposed to determine which transform size is employed to generate the resulting JND model. Experimental results have demonstrated that our proposed scheme could tolerate more distortions while preserving better perceptual quality than other JND profiles, which means that the proposed model consists well with human vision system (HVS). Moreover, for the balanced strategy, experiments have shown that temporal motion characteristics accord very well with the spatial image content information, which has demonstrated the efficiency of our proposed balanced strategy.
Advances in visual content analysis and adaptation for multimedia communications [Guest editorial]

The eight tutorial papers in this special section focus on ways to effectively improve the usability, efficiency, and robustness of multimedia communications in general and visual content access, adaptation, and delivery in particular. Visual analysis and processing methods must be able to provide automatic or semi-automatic solutions for: high accuracy content segmentation and representation; content-aware robust visual data compression; adapting or transcoding data for various channels and terminals; scalability across the visual content; and content and network-aware cross-layer optimized video transmission.
A novel geometric filter for affine invariant features

Invariant local image features have proven to be very successful in computer vision tasks involving partial occlusion and various image deformations. Even though the image features can be extracted in a high repeatability, their local appearance alone usually does not bring enough discriminative power to support a reliable matching, resulting in a relatively high number of outliers in the correspondence set. To reject these mismatches, various geometric filters have been proposed for different image features. In this paper, we present a novel and efficient geometric filter for the state-of-the-art affine invariant features. The proposed method detects the mismatches by examining the consistency of local affine geometry between neighboring matches of affine invariant features. Experimental results show that the proposed geometric filter not only achieves a higher inlier ratio than the standard Hough clustering, but also presents superior robustness to severe clutters, significant viewpoint changes and non-rigid deformation.
Error detection and correction of vector quantised digital images

The performance of a low bit-rate digital image codec based on a product vector quantiser (VQ) is examined in a noisy transmission channel environment. The VQ codec is a separating gain/shape type providing good data compression at about 1.25 bits per pixel. For small image subblock sizes, there is sufficient interblock correlation in the mean, standard deviation and shape information to facilitate crude error detection mechanisms to be implemented. We reinforce these error detection capabilities with a subblock boundary continuity testing algorithm. Erroneous subblocks are then estimated by extrapolating intensity contours from neighbouring subblocks. Using such an error detection and correction technique, noisy recovered VQ images can be considerably improved. At a bit error rate of 0.01, we obtained an improvement in the normalised mean square error of about 0.013, or equivalently about 6 dB in signal-to-noise ratio.
Error resilient transmission of SPIHT coded images over fading channels

The paper introduces a method of transmitting error resilient SPIHT coded images over highly error prone Rayleigh fading channels. First, the source significance of the SPIHT coded output is obtained. Based on the significance of the bits, the channel coding is varied accordingly. Channel coding consists of a mixture of rate compatible punctured convolutional (RCPC) codes and interleaving to combat the burst errors produced by the fading channel. An additional error concealment technique is also introduced into the SPIHT decoder to improve its results in cases where errors cause the corruption of the average luminance level during decoding. Comparison with using RS block codes at a total transmission rate of 1.0 bits/pixel is carried out over fading channels with very high error rates to show the superiority of this method over methods using burst error codes.
Reduction of blocking artifacts in image and video coding

The discrete cosine transform (DCT) is the most popular transform for image and video compression. Many international standards such as JPEG, MPEG, and H.261 are based on a block-DCT scheme. High compression ratios are obtained by discarding information about DCT coefficients that is considered to be less important. The major drawback is visible discontinuities along block boundaries, commonly referred to as blocking artifacts. These often limit the maximum compression ratios that can be achieved. Various postprocessing techniques have been published that reduce these blocking effects, but most of them introduce unnecessary blurring, ringing, or other artifacts. In this paper, a novel postprocessing algorithm based on Markov random fields (MRFs) is proposed. It efficiently removes blocking effects while retaining the sharpness of the image and without introducing new artifacts. The degraded image is first segmented into regions, and then each region is enhanced separately to prevent blurring of dominant edges. A novel texture detector allows the segmentation of images containing both texture and monotone areas. It finds all texture regions in the image before the remaining monotone areas are segmented by an MRF segmentation algorithm that has a new edge component incorporated to detect dominant edges more reliably. The proposed enhancement stage then finds the maximum a posteriori estimate of the unknown original image, which is modeled by an MRF and is therefore Gibbs distributed. A very efficient implementation is presented. Experiments demonstrate that our proposed postprocessor gives excellent results compared to other approaches, from both a subjective and an objective viewpoint. Furthermore, it will be shown that our technique also works for wavelet encoded images, which typically contain ringing artifacts.
Spread Spectrum Image Watermarking Based on Perceptual Quality Metric

Efficient image watermarking calls for full exploitation of the perceptual distortion constraint. Second-order statistics of visual stimuli are regarded as critical features for perception. This paper proposes a second-order statistics (SOS)-based image quality metric, which considers the texture masking effect and the contrast sensitivity in Karhunen-Loève transform domain. Compared with the state-of-the-art metrics, the quality prediction by SOS better correlates with several subjectively rated image databases, in which the images are impaired by the typical coding and watermarking artifacts. With the explicit metric definition, spread spectrum watermarking is posed as an optimization problem: we search for a watermark to minimize the distortion of the watermarked image and to maximize the correlation between the watermark pattern and the spread spectrum carrier. The simple metric guarantees the optimal watermark a closed-form solution and a fast implementation. The experiments show that the proposed watermarking scheme can take full advantage of the distortion constraint and improve the robustness in return.
Constant distortion rate control for H.264/AVC high definition videos with scene change

In this paper, we propose a constant distortion rate control algorithm for H.264/AVC high definition videos with scene change. With the rate and distortion information of each frame, the frame scene complexity is modeled and the scene change is detected in the first pass encoding. In the second-pass encoding, the expected distortion of each GOP under the constraint of target bits is derived based on the first-pass statistics. And the quantization step (Q-step) of each frame can be obtained with a linear distortion-quantizer (D-Q) model by considering the parameter update of this model at scene change. The experimental results show that our proposed algorithm can provide more constant quality among different video scenes, compared to traditional joint model (JM) rate control algorithm.
Learning to Extract Focused Objects From Low DOF Images

This paper proposes an approach to extract focused objects (i.e., attention objects) from low depth-of-field images. To recognize the focused object, we decompose the image into multiple regions, which are described by using three types of visual descriptors. Each descriptor is extracted from a representation of some aspects of local appearance, e.g., a spatially localized texture, color, or geometrical property. Therefore, the focus detection of a region can be achieved by the classification of extracted visual descriptors based on a binary classifier. We employ a boosting algorithm to learn the classifier with a cascade of decision structure. Given a test image, initial segmentation can be achieved using obtained classification results. Finally, we apply a post-processing technique to improve the results by incorporating region grouping and pixel-level segmentation. Experimental evaluation on a number of images demonstrates the performance advantages of the proposed method, when compared with state-of-the-art methods.
Rate-distortion modeling for binary shape in MPEG-4

In this paper, we explore the rate-distortion (R-D) characteristics of binary shape information in the MPEG-4 standard. The shape coding scheme is a block based context-based arithmetic (CAE) encoding approach and distortion is introduced by down- and up-sampling. Generally, the shape bit rate and non-normalized distortion increase in proportion with the number of border blocks. The more complex object suffers from more distortion under the same resolution scale. Consequently, we propose a rate-distortion model based on the parameters derived from the border blocks and a block-based shape complexity (SC) for the video object. The computational complexity is much lower than other existing methods. Experimental results show that it can accurately predict the bit rate and distortion of the binary shape for rate control purpose.