nils-herrmann/clean_scirepeval huggingface dataset:
Composite modeling of optical flow for artifacts reduction

Because of the outstanding contribution in removing pixel correlation, block-based transform and quantization has been widely accepted in the state-of-art image/video coding standards. However, artifacts are introduced, which result in reducing the subjective quality of the decoded image/video. In this paper, a composite model of the optical flow velocity is mathematically derived and proved to further reduce the blocking artifacts by compensating the DC surface of the decoded image according to the model. The functional relationship among the optical flow velocity of the DC surface of the decoded image, the value discontinuity across the block boundary in the decoded image and the quantity of DC surface adjustment is analyzed and mathematically modeled as a composite function with second-degree polynomial. Four feature points on the model with the corresponding criteria in artifacts reduction are compared to show a good consistency between the model and the actual filtering effects. Finding a point on the model with a better trade-off between the optical flow smoothness and the edge smoothness is promising in producing a more pleasing and natural looking image/video.
Screen-camera calibration using a thread

In this paper, we propose a novel screen-camera calibration algorithm which aims to locate the position of the screen in the camera coordinate system. The difficulty comes from the fact that the screen is not directly visible to the camera. Rather than using an external camera or a portable mirror like in previous studies, we propose to use a more accessible and cheaper calibrating object, i.e., a thread. The thread is manipulated so that our algorithm can infer the perspective projections of the four screen corners on the image plane. The 3-dimentional (3D) position of each screen corner is then determined by minimizing the sum of squared projection errors. Experiments show that compared with the previous studies our method can generate similar calibration results without the additional hardware.
The implementation of object-based shot boundary detection using edge tracing and tracking

With the upcoming development of MPEG-7, research in shot boundary detection has become more popular. Traditionally, shot boundary detection was done by measuring the differences between the features of adjacent frames and threshold was constructed based on a trial and error basis. Such a method does not work well under gradual transition. In this paper, we describe a new object-based detection, which utilizes information from multiple frames through a time stamp transferring mechanism. The concept of edge object tracker is developed to track fast moving objects as well as allowing the unbounded object to be traced across the gradual transition scenes. Our result shown that object-based detection is more superior to traditional indicators for detecting gradual transitions. Such technique works very well with frames containing skipped blocks and is independent of the transition period, which cannot be handled by indicators based on two-frame comparison.
3-D object recognition using fuzzy quaternions

A new discrimination method based on fuzzy subset theory is proposed. This approach uses Zernike moment invariants of the silhouette of the unknown object to form a set of fuzzy-weighted quantities called fuzzy quaternions. These are matched against those of known objects at predetermined viewpoints. To speed up the recognition process, a modified simplex version of the Nelder-Mead algorithm is utilised. Results obtained demonstrate the potential of the fuzzy quaternion as a viable basis for discrimination.
A flexible Bayesian framework for image segmentation

This paper presents a new Bayesian framework for image segmentation. The major contribution is a novel optimization strategy that can be applied to any cost function derived from the MAP criterion. Classical Bayesian techniques normally minimize this cost using ICM together with a K-label approach that assigns each pixel a label m/spl isin/{0,1,...,K-1}. Several shortcomings of this approach are pointed out. Our proposed method first extracts initial seeds that represent the interior of regions. The boundary location is then determined by a modified HCF method that labels pixels in the order of decreasing confidence. There is no need for an initial estimate of the segmentation, and no parameter K is required. Moreover, the presented framework can be viewed as a combination of the elegant morphological segmentation approach with the spatial continuity constraints inherent to Markov random fields in Bayesian techniques. Experimental results demonstrate the significant improvements achieved by our optimization strategy.
Dense depth map generation using sparse depth data from normal flow

In this paper, we address the problem of dense depth map generation from two successive image frames in a video. We first recover the camera motion from the observable normal flow pattern using our previously proposed apparent flow constraints. Once the camera motion is estimated, sparse depth data can be directly recovered from the flow pattern. We utilize a hierarchical approach to generate an initial dense depth map from the sparse depth data. This depth map is further enhanced through the refinement of the associated optical flow field in a variational framework. Experimental results show that the proposed method can provide high-quality depth maps. We also have a faster computational time than the conventional optical flow approach.
A hybrid image coding scheme using CVQ and DCT

The authors describe a hybrid image coding scheme called the classified hybrid image coder (CHIC). CHIC combines the classified vector quantization (CVQ) and the discrete cosine transform (DCT) coding schemes to achieve a better performance than would be possible if they were to be used separately. The image is subdivided into 16*16 pixel blocks (vectors) and transformed using DCT. Each vector is classified into an edge vector and a shade vector. Classification is based on the distribution and value of the DCT coefficients in each vector. Shade vectors are coded using CVQ. Edge vectors are further classified according to their edge strengths. In each of these classes, the coefficients are quantized and then coded with an adaptive arithmetic coder (AAC). The performance of the AAC is further improved by further classifying the DCT coefficients into different groups, based on their different frequency ranges. The AAC uses a set of training images to generate the initial histogram tables.<<ETX>>
Joint temporal error control for H.264

In this paper, we propose a joint temporal error control (JTEC) method for H.264 which combines RDO-based macroblock (MB) classification at the encoder and adaptive partition size error concealment at the decoder. The encoder classifies the MBs by evaluating the sensitivity of the MBs as the RD cost between the concealment error and the bits needed for the additional motion information. Additional motion information can be transmitted for the error sensitive MBs such as the original motion vector or motion vector index. The decoder utilizes the additional motion information for the sensitive blocks. Non-sensitive blocks are concealed by the Adaptive Partition Size (APS) method. APS selects the best partition mode for error concealment by minimizing the Weighted Double-Side External Boundary Matching Error (WDS-EBME), which jointly measures the inter-MB boundary discontinuity, inter-partition boundary discontinuity and intra-partition block artifacts in the recovered MB. A progressive concealment method is developed for the 4x4 partition mode.
Optimal bit allocation for MPEG-4 multiple video objects

In this paper, we present an optimal bit allocation algorithm for multiple video objects (MVO's) in MPEG-4 video coding. We combine the rate-distortion models for texture and shape information to achieve the accurate rate control. To optimize the bit allocation for multiple video objects, we use the dynamic programming (DP). The simulation results demonstrate that the proposed algorithm outperforms the MPEG-4 verification model not only in terms of buffer stability but the objective quality.
Using mid-high level cues to detect salient object

This paper proposes a novel saliency object detection method by using the mid-level and high-level visual cues. In the mid-level objectness evaluation, we generate three complementary saliency maps, such as the multi-scale segmentation cue, the background cue and the spatial color distribution cue. The first cue is used to highlight the objects via the local region segment. The second cue uses the background priors to detect the saliency information. The third cue is to capture the spatial color distribution. For the high-level visual cue, we propose an objectness evaluation model to distinguish the object and the background. All the saliency cues are finally combined to achieve the saliency detection. The experimental results show that the proposed method outperforms the state-of-the-art saliency object detection methods.
3D Object-based Scalable Wavelet Video Coding with Boundary Effect Suppression

This paper extends the lifting-based motion threading technique from the frame-based coding to the object-based coding, attracted by the unique advantages of the object-based coding that do not exist in other coding schemes. The boundary effects, which exist in spatial and temporal wavelet transforms due to the manner of object-based coding, are well suppressed by 3D shape adaptive discrete wavelet transform (3D SA-DWT) via the lifting implementation in a unified framework. Based on the proposed object-based motion threading technique, a 3D object-based scalable wavelet video coder is developed. Experimental results show that the proposed coder achieves good performance compared with the existing object-based coders, and provides the scalability functionality at the same time.
Video object plane segmentation using a morphological motion filter and Hausdorff object tracking

This paper considers the decomposition of video sequences into so-called video object planes, which is required for the content-based representation of visual objects in MPEG-4. A new segmentation algorithm is described that identifies physical objects using a morphological motion filter. For the object of interest, a two-dimensional binary model is derived based on areas in the scene that are moving differently from the background. This model is updated each frame to pick up possible rotation and changes in shape of the object. Temporal correspondence is established by a Hausdorff object tracker: The binary model sequences guide the actual video object plane extraction. Since the model points correspond to edges detected by the Canny operator; a high object boundary location is achieved. Experimental results demonstrate that our proposed technique can successfully extract the physical object from video sequences.
Globally Measuring the Similarity of Superpixels by Binary Edge Maps for Superpixel Clustering

This paper proposes an edge-based superpixel similarity measurement, which globally evaluates the similarity between superpixels by binary edge maps. The basic idea is to assess whether the superpixels are surrounded by the same edges. To this end, we first describe the edge spatial distributions by directional regions and then use the directional regions to represent the surrounding relationships of superpixels and edges by their traverse relationships, which form the histogram feature. Finally, the similarity is simply calculated by the distances between the features. To verify the proposed similarity measurement, we use our global similarity measurement to perform superpixel clustering. Two clustering methods, the directed graph clustering (DGC) and spectral clustering (ultrametric contour map) are combined to achieve the clustering process. The combination of our global similarity measurement and DGC to form a new three-layer-based superpixel generation method, which can quickly generate the superpixel from edge maps, is highlighted. We verify the global similarity measurement by the BSDS500 dataset. The experimental results demonstrate that the proposed global similarity measurement can improve the clustering accuracy in terms of larger intersection-over-union-criterion-based values. The code can be downloaded from https://github.com/FanmanMeng/Superpixel-Similarity-Measurement.
Hierarchical Parsing Net: Semantic Scene Parsing From Global Scene to Objects

This paper proposes a novel Hierarchical Parsing Net (HPN) for semantic scene parsing. Unlike previous methods, which separately classify each object, HPN leverages global scene semantic information and the context among multiple objects to enhance scene parsing. On the one hand, HPN uses the global scene category to constrain the semantic consistency between the scene and each object. On the other hand, the context among all objects is also modeled to avoid incompatible object predictions. Specifically, HPN consists of four steps. In the first step, we extract scene and local appearance features. Based on these appearance features, the second step is to encode a contextual feature for each object, which models both the scene-object context (the context between the scene and each object) and the interobject context (the context among different objects). In the third step, we classify the global scene and then use the scene classification loss and a backpropagation algorithm to constrain the scene feature encoding. In the fourth step, a label map for scene parsing is generated from the local appearance and contextual features. Our model outperforms many state-of-the-art deep scene parsing networks on five scene parsing databases.
An adaptive and parallel scheme for hd video de-interlacing

It is very challenging to de-interlace HD videos in real time, as both high efficiency and low complexity should be fulfilled, which, however, are conflicting. This paper presents a de-interlacer to resolve the conflict specially for H.264 coded videos. It adapts to spatially and temporally local activities by making full use of the syntax element (SE) values in bit-streams, which give many hints of the motions and textures of video sequences. Accuracy analysis is also introduced to deal with the disparity between the SE values and the real motions and textures. The experimental results show the proposed de-interlacer provides better visual quality than common ones and can de-interlace 1080i sequences in real time on PCs.
Motion trajectory based visual saliency for video quality assessment

In this paper, we propose a novel visual saliency detection method for video sequences by considering the object motion trajectories. Firstly, each frame of the video sequence is described in a new Quaternion Representation (QR), which comprises the spatial image content and the temporal motion characteristics. Based on the QR, Quaternion Fourier Transform (QFT) is employed to construct the visual salien-cy of the video sequence. Finally, the detected visual salien-cy map is incorporated with several video quality metrics. Compared with other visual saliency models, the proposed method can improve the performances of video quality metrics. It further confirms that the proposed visual saliency model can accurately depict the Human Vision System (HVS) properties.
Perceptual image compression via adaptive block- based super-resolution directed down-sampling

In this paper, we propose a novel perceptual image coding scheme via adaptive block-based super-resolution directed down-sampling. At the encoder side, for each macroblock of a given image, Rate Distortion Optimization (RDO) determines whether it is encoded at the original or down-sampled resolution. The down-sampling process is directed by super-resolution, which generates the down-sampled block by minimizing the reconstruction errors between the original macroblock and the one restored by the corresponding super-resolution method. At the decoder side, in order to reduce the complexity, the super-resolution method reconstructs the full-resolution macroblock in the DCT domain together with the inverse DCT. Experimental results have demonstrated that the proposed method can produce higher quality images in terms of both PSNR and visual quality compared with the existing methods.
A review of object segmentation quality subjective assessment methods

In the past decades, researchers have made great efforts to develop object segmentation algorithms and constantly improving the segmentation quality. Compared with object segmentation algorithms, object segmentation quality assessment is less studied. Especially, how to evaluate segmentation quality by subjective assessment, which is the most reliable way to understand and analyze segmentation quality in terms of human perception has not attracted much attention. In this paper, we review three object segmentation subjective assessment methods and propose possible future directions for research in object segmentation quality subjective assessment.
Automatic segmentation of objects of interest in video: a unified framework

Automatically extracting interesting objects from videos is a very challenging task. Traditional video segmentation algorithms assume that objects of interest are the moving objects, whereas they neglect the fact that some stationary objects are also likely to attract human interest. To remove this limitation, this study provides a unified video object segmentation framework by taking human visual attention perception into account. It is implemented by three major steps. First, the JSEG algorithm (Y. Deng et al,  vol.23, p.800-810, 2001) is adopted to partition each frame into homogenous regions. Following that, based on a number of visual features that have been proven to be able to influence attention, two visual attention models are proposed to calculate the attention value for each region. Finally, a combination of the spatial segmentation and the visual attention models produces all the objects of interest with and without motion. Simulation results on standard video sequences demonstrate its effectiveness.
A robust Markovian segmentation based on highest confidence first (HCF)

A new robust method to segment images based on Markov random fields (MRF) is presented. The algorithm does not require the number of classes or regions K as input, which is normally difficult to determine in advance. There is also no need for an initial estimate obtained by an algorithm such as K-means. Further, each region is connected during the whole segmentation process leading to more reliable estimates of the regions' mean gray levels and to fewer wrong detected boundaries. In addition, a novel way to incorporate edge information into the segmentation process is proposed resulting in a better detection of small objects. Experimental results demonstrate the performance of our technique.
Rate-constrained arbitrarily shaped video object coding with object-based rate control

The authors develop an object-based rate control algorithm for MPEG-4 arbitrarily shaped video object coding. There are three main contributions. An improved quadratic rate-distortion model for texture information is derived, which is more accurate for arbitrarily shaped video objects. Secondly, an accurate rate-distortion model for shape information is introduced. Combining the rate-distortion models for texture and shape information, an object level rate control is proposed. To achieve constant bit rate, the quantiser is adjusted based on buffer fullness. The tradeoff between picture quality and buffer fullness is considered. Compared with the rate control algorithm for the MPEG-4 verification model, the proposed scheme can achieve a more stable buffer without sacrificing picture quality.
Automatic scale selection for corners and junctions

In this paper, we aim to address the problem of automatic scale selection for image features like corners and junctions. Image neighborhood of these features usually contain the background and multiple foreground surfaces, thus can not be correctly described by a single scale. We show that the proposed Fan Laplacian-of-Gaussian (FLOG) kernel is capable to select the appropriate scales for independent image partitions that can represent meaningful physical surfaces attached to the corner or junction. Support for the proposed method is given in terms of theoretical investigation and experiments on real images.
Video segmentation for content-based coding

To provide multimedia applications with new functionalities, the new video coding standard MPEG-4 relies on a content-based representation. This requires a prior decomposition of sequences into semantically meaningful, physical objects. We formulate this problem as one of separating foreground objects from the background based on motion information. For the object of interest, a 2D binary model is derived and tracked throughout the sequence. The model points consist of edge pixels detected by the Canny operator. To accommodate rotation and changes in shape of the tracked object, the model is updated every frame. These binary models then guide the actual video object plane (VOP) extraction. Thanks to our new boundary postprocessor and the excellent edge localization properties of the Canny operator, the resulting VOP contours are very accurate. Both the model initialization and update stages exploit motion information. The main assumption underlying our approach is the existence of a dominant global motion that can be assigned to the background. Areas that do not follow this background motion indicate the presence of independently moving physical objects. Two alternative methods to identify such objects are presented. The first one employs a morphological motion filter with a new filter criterion, which measures the deviation of the locally estimated optical flow from the corresponding global motion. The second method computes a change detection mask by taking the difference between consecutive frames. The first version is more suitable for sequences with little motion, whereas the second version is better at dealing with faster moving or changing objects. Experimental results demonstrate the performance of our algorithm.
Visual Horizontal Effect for Image Quality Assessment

In this paper, an image quality metric is proposed by modeling the visual Horizontal Effect (HE) and saliency property over structural distortions. Specifically, Structrue SIMilarity (SSIM) is firstly performed to obtain the structural distortion map. Subsequently, the obtained distortion map is refined by the visual HE model, which depicts visual sensitivities of oriented stimuli over different oriented contents. Finally, in order to describe the local Human Visual System (HVS) conspicuities, a saliency pooling strategy is proposed to generate the resulting image quality index. The experimental results have demonstrated that the proposed method outperforms SSIM and Visual Information Fidelity (VIF), which indicates that the obtained similarity index is more consistent with the perceptual evaluation of image quality.
A facial expression model with generative albedo texture

A facial expression model (FEM) is developed which can synthesize various face shapes and albedo textures. The face shape varies with individuals and expressions. FEM synthesizes these shape variations by using a bilinear face model built from the Face Warehouse Database. On the other hand, the generative albedo texture is directly extracted from a neutral face model — the Basel Face Model. In this paper, we elaborate the model construction process and demonstrate its application in face reconstruction and expression tracking.