nils-herrmann/clean_scirepeval huggingface dataset:
An Unsupervised Method to Extract Video Object via Complexity Awareness and Object Local Parts

Existing unsupervised video object segmentation generates object information from the whole video, which ignores analysis of the local clips. However, we observe that local clips and their relationships are also useful for the video object segmentation. For example, the simple background clips can be used to improve the segmentation of complex background clips. In this paper, we propose a novel unsupervised segmentation framework to segment the primary object based on two aspects, i.e., the complexity awareness of video clips and their segmentation propagation. The first one is used to select the simple clips with smooth backgrounds and the second one generates an object prior from the simple clips and propagates the object prior to help and improve the segmentation of the complex clips. A complexity awareness method using the static cues and the dynamic cues are proposed to evaluate the complexity of the video frames. A new object prior learning model based on the local part structure is designed and a local part-based prior propagation is proposed for the complex clip segmentation. To verify our method, we collect a new challenging video segmentation data set, in which each video contains diverse backgrounds. Experimental results demonstrate that our method outperforms several state-of-the-art methods both on a classical data set and our new data set.
Unsupervised Salient Object Segmentation Based on Kernel Density Estimation and Two-Phase Graph Cut

In this paper, we propose an unsupervised salient object segmentation approach based on kernel density estimation (KDE) and two-phase graph cut. A set of KDE models are first constructed based on the pre-segmentation result of the input image, and then for each pixel, a set of likelihoods to fit all KDE models are calculated accordingly. The color saliency and spatial saliency of each KDE model are then evaluated based on its color distinctiveness and spatial distribution, and the pixel-wise saliency map is generated by integrating likelihood measures of pixels and saliency measures of KDE models. In the first phase of salient object segmentation, the saliency map based graph cut is exploited to obtain an initial segmentation result. In the second phase, the segmentation is further refined based on an iterative seed adjustment method, which efficiently utilizes the information of minimum cut generated using the KDE model based graph cut, and exploits a balancing weight update scheme for convergence of segmentation refinement. Experimental results on a dataset containing 1000 test images with ground truths demonstrate the better segmentation performance of our approach.
Video coding for ISDN applications using predictive classified vector quantization

A video coding algorithm based on predictive classified vector quantization (PCVQ) is described. For teleconferencing applications, it achieves bit per pixel (BPP) values of 0.1856 for the Y plane, 0.0811 for the U plane, and 0.0855 for the V plane of a commonly used color image sequence called Miss America with acceptable subjective quality. This is equivalent to a bit rate of 89.262 kb/s for a frame size of 144*176 pixels and a frame rate of 10 frames/s. Together with its simple structure, these results demonstrate that it can be used for video transmission over ISDN (integrated services digital network) channels.<<ETX>>
Visual Quality Evaluation of Image Object Segmentation: Subjective Assessment and Objective Measure

A visual quality evaluation of image object segmentation as one member of the visual quality evaluation family has been studied over the years. Researchers aim at developing the objective measures that can evaluate the visual quality of object segmentation results in agreement with human quality judgments. It is also significant to construct a platform for evaluating the performance of the objective measures in order to analyze their pros and cons. In this paper, first, we present a novel subjective object segmentation visual quality database, in which a total of 255 segmentation results were evaluated by more than thirty human subjects. Then, we propose a novel full-reference objective measure for an object segmentation visual quality evaluation, which involves four human visual properties. Finally, our measure is compared with some state-of-the-art objective measures on our database. The experiment demonstrates that the proposed measure performs better in matching subjective judgments. Moreover, the database is available publicly for other researchers in the field to evaluate their measures.
Adaptive partition size temporal error concealment for H.264

Existing temporal error concealment methods for H.264 often decide the partition size of the lost macroblock (MB) before recovering the motion information, without actual quality comparison between different partition modes. In this paper, we propose to select the best partition mode by minimizing the Weighted Double-Sided External Boundary Matching Error (WDS-EBME), which jointly measures the inter-MB boundary discontinuity, inter-partition boundary discontinuity and intrapartition block artifacts in the recovered MB. The proposed method estimates the best motion vectors for each of the candidate partition modes, calculates the overall WDS-EBME values for them, and selects the partition mode with the smallest overall WDS-EBME to recover the lost MB. We also propose a progressive concealment order for the 4times4 partition mode. Test results show that the adaptive partition size method always outperforms the fixed partition size methods. Both the adaptive and fixed partition size methods are much superior to the temporal error concealment (TEC) method in the H.264 reference software.
Rate-perceptual-distortion optimization (RpDO) based picture coding — Issues and challenges

Perceptual image and video coders have been reported extensively in the literature with a recent survey listing sixteen representative PVCs (perceptual video coders) for single-view video coding, ten of which conform to H.263, MPEG-4 or H.264/AVC international standard. Successful approaches to perceptual picture coding are classified as perceptual predictive coding, perceptual quantization, rate-perceptual-distortion optimization (RpDO) and perception-based pre-, loop- and post-filtering. This contribution analyzes and examines the performance and tradeoffs of representative perceptual as well as traditional waveform based picture coding methods in terms of their effectiveness in quality constrained coding and RpDO criterion in the context of visual quality controlled or regulated visual communication, broadcasting, entertainment and consumer electronic products, systems and services or applications. Fundamental flaws of performance benchmarking using a rate-driven coding strategy are interrogated for visual quality assured services or user experience. It is revealed that while existing perceptual distortion measures (PDMs) successfully grade distortions corresponding to HVS (human visual system) perception, their quantitative correspondence, or lack of it, to discernible levels by human visual perception in terms of just-not-noticeable-difference (JNND), just-noticeable-difference level 1 (JND1), JND level 2 (JND2), etc. has not been thoroughly investigated nor established, and this motivates the related initial work reported in this paper. Issues and constraints of existing picture coding standards are discussed in the context of developing a future picture coding standard framework for constant visual picture quality systems and services.
HDTV coding using hybrid MRVQ/DCT

The authors develop a technique to compress HDTV images to below 135 Mb/s at transparent quality, i.e., visually indistinguishable from the original. This is at least equal to the contribution quality. The proposed scheme consists of a lossy and a lossless coder. The lossy coder combines the advantages of two image coding methods, namely, mean/residue vector quantization (MRVQ) and discrete cosine transform (DCT) coding, and the lossless coder is an arithmetic coder (AC), which has been shown to outperform the commonly used Huffman code . The authors describe the various components of the hybrid HDTV image coding scheme and how they are implemented.
Shot boundary refinement for long transition in digital video sequence

Shot boundary detection, or scene change detection, is a technique used in the initial step of video indexing. With the recent introduction of object-based shot boundary detection technique, the deficiency of traditional detection techniques in handling long transitions has been overcome. However, such a technique merely detects the existence of a transition without giving accurate positions of the transition start and end for long transitions. Shot boundary refinement fills this gap by pinpointing the exact location of the shot boundary regardless of the transition types. This paper presents the theories behind the transition and classifies existing transition types. A novel shot boundary refinement algorithm is then constructed which works independently of the contents and transition length, and is accurate under most transition types. The results of testing using different indicators and different transition types show that the boundaries of long transitions are detected with minimal errors. The success of shot boundary refinement brings us one step closer in developing an accurate, fully automatic indexing system.
An efficient algorithm for H.264/AVC high definition video coding

Quality consistency has been an important issue in video applications. However, the quality fluctuation cannot be avoided in single pass video coding due to the variation of the video content, flexible coding structure, different frame types, etc. In this paper, we propose an efficient method to reduce the quality fluctuation for H.264/AVC high definition (HD) video coding. We aim to reduce the distortion variation among frames at different types. A single-pass distributed control algorithm is introduced which provides different control mechanism for different frame type. For P frame, the quantization parameter is decided by the relationship between the quantization step and the given target bit rate and the buffer status to satisfy the bit rate constraints. For B frame, the quantization parameter is determined by the relationship between the quantization step and the given target distortion to reduce the quality fluctuation between P and B frames. Extensive simulation results are provided to demonstrate the efficiency of the proposed method. When compared to the traditional method used in current H.264/AVC video coding, the quality fluctuations are significantly reduced. Moreover, the bit rate constraints of the high definition video coding are satisfied.
Fast patch-wise image retargeting

Content-aware image retargeting adjusts images to arbitrary sizes and preserves visually salient content. Previous algorithms formulate the problem in terms of either pixel level or mesh level structures, deforming salient objects inconsistently. To improve retargeting quality and reduce complexity, we introduced a patch-wise method to generate sparse image grids based on visual saliency and gradient magnitude. Three energy functions were optimized to warp the grids and generate retargeted images. Experimental results on the public database RetargetMe show that this patch-wise retargeting algorithm has lower complexity and performs slightly better than other algorithms using much denser grids.
Automatic video sequence segmentation using object tracking

Emerging video standards address content-based functionalities and interactive multimedia. MPEG-4 segments the frames into video object planes (VOP) with each VOP corresponding to one semantically meaningful object. In this paper a new automatic video sequence segmentation algorithm is presented that can extract such VOPs. A binary model for moving objects is automatically derived and tracked in subsequent frames using the generalised Hausdorff distance. The model is updated every frame to accommodate for rotation and changes in shape. For that an update mechanism consisting of two components is proposed. To improve the segmentation and reduce the computational complexity the stationary background is filtered by a novel technique. Experimental results demonstrate the performance of our proposed algorithm.
Image display techniques using the cosine transform

Transform coding has the ability to rapidly display a picture of low resolution by inverse transformation of a few of the low sequency coefficients only. Picture resolution is then progressively built up as more coefficients are received and decoded. In this paper, different sequences of transmitting the coefficients are investigated to find one which achieves the best subjective quality with the fewest coefficients transmitted and requiring the least overhead information. Five schemes are examined on two source images with different statistics to compare the effectiveness of the schemes to achieve the above objectives on each of the images.
Reduced-Reference Image Quality Assessment Using Reorganized DCT-Based Image Representation

In this paper, a novel reduced-reference (RR) image quality assessment (IQA) is proposed by statistical modeling of the discrete cosine transform (DCT) coefficient distributions. In order to reduce the RR data rates and further exploit the identical nature of the coefficient distributions between adjacent DCT subbands, the DCT coefficients are reorganized into a three-level coefficient tree. Subsequently, generalized Gaussian density (GGD) is employed to model the coefficient distribution of each reorganized DCT subband. The city-block distance is employed to measure the difference between the two images. Experimental results demonstrate that only a small number of RR features is sufficient for representing the image perceptual quality. The proposed method outperforms the RR WNISM and even the full-reference (FR) quality metric PSNR.
Scalable coding of subband images with quadtree-based classified vector quantization

Scalable coding is useful when a variety of resolutions are simultaneously required or when resilience to errors is necessary. This paper describes spatial and SNR scalable coding schemes which are based on subband coding with quadtree-based classified vector quantization. It is expected that the proposed scalable coding techniques can be efficiently adapted for simulcast coding and error resilient coding over ATM networks when used with interframe coding and motion compensation techniques. It is also shown that the proposed techniques are superior to the JPEG standard in which hierarchical (spatial scalability) and progressive (SNR scalability) modes are used for the comparison.
Motion-Depth: RGB-D Depth Map Enhancement with Motion and Depth in Complement

Low-cost RGB-D imaging system such as Kinect is widely utilized for dense 3D reconstruction. However, RGB-D system generally suffers from two main problems. The spatial resolution of the depth image is low. The depth image often contains numerous holes where no depth measurements are available. This can be due to bad infra-red reflectance properties of some objects in the scene. Since the spatial resolution of the color image is generally higher than that of the depth image, this paper introduces a new method to enhance the depth images captured by a moving RGB-D system using the depth cues from the induced optical flow. We not only fill the holes in the raw depth images, but also recover fine details of the imaged scene. We address the problem of depth image enhancement by minimizing an energy functional. In order to reduce the computational complexity, we have treated the textured and homogeneous regions in the color images differently. Experimental results on several RGB-D sequences are provided to show the effectiveness of the proposed method.
3D subband VLBR video coding scheme

A video coder for very low bit rate applications is designed and simulated. It employs a subband analysis technique to first split the video signal into four temporal subbands. Motion detection is performed to classify the co-sited macroblocks of the subbands into temporal activity (TA) macroblocks or no temporal activity (NTA) macroblocks. The base temporal subband is coded using a H.261-like (European COST 211 ter SIM3) coder with the motion estimation replaced by a reduced-motion search algorithm. The higher temporal subbands are vector-quantized. The NTA-macroblocks (except in the baseband) are not coded at all. Simulation results are presented at 9.6 kbits/s at five frames per second for a typical videoconferencing sequence. Comparisons are made with results obtained by the SIM3 coder.<<ETX>>
Parametric interpolation filter for motion compensated prediction

Recently, adaptive interpolation filter (AIF) has received increasing attention for motion-compensated prediction (MCP). The existing methods code the filter coefficients individually and the accuracy of coefficients and the size of side information are conflicting. This paper studies the effect of making trade-off between the two conflicting aspects and proposes the parametric interpolation filter (PIF), which represents filters by five parameters instead of individual coefficients and approximats the optimal filter by tuning the parameters. The experimental results show that PIF approaches the efficiency of the optimal filter and outperforms the related work.
Fast lossless multi-resolution motion estimation for scalable wavelet video coding

In this paper, we propose a fast lossless multi-resolution motion estimation algorithm in shift-invariant wavelet domain by using wavelet matching error characteristic based partial distortion elimination (MR-WMEC-PDE). Due to its multi-resolution nature, the proposed approach can be applied to scalable video coding. The proposed algorithm can achieve the same estimate accuracy as full search algorithm (FSA), while needing much less computation requirement than FSA
Spatial-temporal decorrelation for image/video coding

Modern image/video compression techniques greatly help to store and transmit digital images and video data. Discrete wavelet transform is used in JPEG2000 because of its scalability and tolerable degradation. In H.264/AVC, predictive coding is employed to remove spatial redundancy before discrete cosine transform. In this paper, we propose a new encoder structure combining both advantages from JPEG2000  and H.264/AVC , which firstly utilizes the proposed spatial transform to decompose a single image into several sub-images and then employs motion compensation to convert the conventional spatial decorrelation into temporal decorrelation. Experimental results show that our proposed method outperforms the state-of-the-art image coding algorithms and achieves better rate-distortion performance.
Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer

In this paper, we propose an efficient blind image quality assessment (BIQA) algorithm, which is characterized by a new feature fusion scheme and a k-nearest-neighbor (KNN)-based quality prediction model. Our goal is to predict the perceptual quality of an image without any prior information of its reference image and distortion type. Since the reference image is inaccessible in many applications, the BIQA is quite desirable in this context. In our method, a new feature fusion scheme is first introduced by combining an image's statistical information from multiple domains (i.e., discrete cosine transform, wavelet, and spatial domains) and multiple color channels (i.e., Y, Cb, and Cr). Then, the predicted image quality is generated from a nonparametric model, which is referred to as the label transfer (LT). Based on the assumption that similar images share similar perceptual qualities, we implement the LT with an image retrieval procedure, where a query image's KNNs are searched for from some annotated images. The weighted average of the KNN labels (e.g., difference mean opinion score or mean opinion score) is used as the predicted quality score. The proposed method is straightforward and computationally appealing. Experimental results on three publicly available databases (i.e., LIVE II, TID2008, and CSIQ) show that the proposed method is highly consistent with human perception and outperforms many representative BIQA metrics.
Fully scalable multiview wavelet video coding

This paper presents a 2-D pipeline-based locally adaptive inter-view-temporal lifting scheme for scalable multi-view video coding. Instead of global temporal and inter-view correlation analysis based on the whole picture, the proposed method adopts locally adaptive inter-view-temporal correlation analysis based on the macroblocks. In addition, in order to reduce the memory requirement and computational complexity and remove both of temporal and view boundary effects, a new 2-D pipeline-based lifting scheme is proposed. Experimental results show that the proposed scheme always outperforms the other tested schemes in coding efficiency, while providing several advantages, such as view scalability, pipeline-based processing, and moderate complexity.
Fast and Efficient Method for Block Edge Classification and Its Application in H.264/AVC Video Coding

Edge is an important feature in video classification which finds applications in video representation and coding. In H.264/AVC, intra-prediction mode decision (a computationally intensive process) is based on the orientation of the edges in the macroblock. In this paper, we first investigate the difference properties derived from three coefficients in the non-normalized Haar transform (NHT) domain and present a fast and efficient method to classify block edge using these properties. The proposed method significantly reduces the number of computational operations in the edge models determination with no multiplications and less addition operations. The use of edge classification for fast intra-prediction mode decision in H.264/AVC video coding is then presented. The experimental results show the effectiveness of the proposed method.
Concatenated sequency majority multiplexing DS-SSMA for synchronous digital communication in indoor wireless narrowband channels

A novel synchronous digital spread spectrum multiple access scheme using concatenated sequency majority multiplexing together with direct sequence spread spectrum multiple access is proposed for the indoor wireless narrowband channel environment. Sequency majority multiplexing and concatenated sequency majority multiplexing are binary digital signal multiplexing schemes using sequency diversity instead of the conventional time or frequency diversity. These sequency majority multiplexing schemes exhibit some attractive transmission error recovery abilities when their channel multiplexing capacity is not fully utilised. This paper presents performance analysis and simulation studies of the sequency majority multiplexing and the concatenated sequency majority multiplexing in the additive white Gaussian noise and frequency non-selective flat fading narrowband channels. From these analysis, we proposed a novel approach to synchronous narrowband digital spread spectrum communication systems utilising, the concatenated sequency majority multiplexing scheme together with direct sequence spread spectrum multiple access. In this paper, some theoretical and experimental results are presented for our proposed synchronous narrowband digital spread spectrum scheme with the binary phase shift keying (BPSK) channel modulation technique. The proposed scheme shows promising performance gains and thus it is an attractive candidate for synchronous narrowband digital spread spectrum communication systems in the indoor wireless environment.