nils-herrmann/clean_scirepeval huggingface dataset:
Weighted Adaptive Lifting-Based Wavelet Transform for Image Coding

In this paper, a new weighted adaptive lifting (WAL)-based wavelet transform is presented. The proposed WAL approach is designed to solve the problems existing in the previous adaptive directional lifting (ADL) approach, such as mismatch between the predict and update steps, interpolation favoring only horizontal or vertical direction, and invariant interpolation filter coefficients for all images. The main contribution of the proposed approach consists of two parts: one is the improved weighted lifting, which maintains the consistency between the predict and update steps as far as possible and preserves the perfect reconstruction at the same time; another is the directional adaptive interpolation, which improves the orientation property of the interpolated image and adapts to statistical property of each image. Experimental results show that the proposed WAL-based wavelet transform for image coding outperforms the conventional lifting-based wavelet transform up to 3.06 dB in PSNR and significant improvement in subjective quality is also observed. Compared with the ADL-based wavelet transform, up to 1.22-dB improvement in PSNR is reported.
Reduced reference video quality assessment based on spatial HVS mutual masking and temporal motion estimation

In this paper, an effective reduced reference (RR) video quality assessment (VQA) is proposed by depicting both the spatial and temporal statistical characteristics of the video signals. For each video frame, spatial information change (SIC) is employed to depict the energy variation. A novel mutual masking strategy based on the extracted SIC is proposed to accurately simulate the human visual system (HVS) texture masking property. For adjacent video frames, the temporal relationship is depicted by block-based motion estimation (BME). The generalized Gaussian density (GGD) function is employed to depict the histogram natural statistic of the residual frame after BME. The city-block distance (CBD) is used to measure the distance between histograms of the original and distorted video sequence. By pooling the measurements from both spatial and temporal perspectives, an efficient RR VQA is constructed. With the evaluations on the public video quality database, the proposed RR VQA demonstrated to be more effective than the representative RR VQAs and even the full-reference (FR) VQAs, such as peak signal-to-noise ratio (PSNR) and structure similarity index (SSIM) in matching the subjective ratings. Furthermore, the proposed RR VQA demonstrated to be much more effective and efficient, requiring only a very small number of bits for the RR feature representation.
Examination of features used for reference indicator construction in shot boundary refinement

Shot boundary refinement is the first stage of long transition analysis. Its purpose is to extract the exact transition start and end in a video sequence. This is done by first constructing a reference indicator and then picking up the sharp gradient changes within such an indicator. The parameter used in constructing the reference indicator can vary, and different features can be used, with reference frames set at different positions. This paper examines the effect of varying these parameters, and its implication when applied to transitions with different special effects.
A temporal just-noticeble distortion profile for video in DCT domain

Just-noticeable distortion (JND) profile is an efficient model to represent the perceptual redundancies. Human eyes usually are not sensitive to the distortion below the JND threshold. In order to extend the spatial JND profile to video images by considering the temporal effects, a temporal JND modulation factor is introduced in this paper, which not only incorporates the temporal contrast sensitivity function (CSF) and the smooth pursuit eye movement (SPEM), but also considers the directionality of the motion in video sequences. This factor can be easily combined with other existing subband spatial JND models to achieve a complete spatio-temporal JND profile. Experimental results show that the proposed model is consistent with the human visual system (HVS). Compared with the other JND profiles, the proposed model can tolerate more distortion and has much better perceptual quality at the same time. Since the proposed JND model is DCT based, it can be easily applied in many related areas, such as compression, watermarking, error protection, perceptual distortion metric, and so on.
Automated Inspection Of IC Bonding Wires Using Hough Transforf4

In this paper, we describe an algorithm developed to inspect bonding wires which connect the die to the package lead frame. The wire inspection algorithm first generates the wire candidate points using a simple one-dimensional gradient thresholding technique before calculating the Hough arrays for each of the candidate points. The latter operation involves the line-to-point Hough transformation in which wires are to be represented by straight lines having parameters ( p , e ) , where p is the distance of the line to the center and e is the inclination angle of the normal to the line. Apart from being able to count the wires and hence capable of detecting wire count aberrations, this algorithm can also detect crossing wires.
Depth enhancement based on hybrid geometric hole filling strategy

Depth map is a crucial component in various 3D applications. However, current available depth maps usually suffer from low resolution, high noise, random and structural depth missing problems due to theoretical, systematic or hardware limitations. In this paper, we propose a novel method to enhance depth map with the guidance of aligned color image, tackling these problems in a whole framework, where a hybrid strategy on filling hole geometrically by the combination of joint bilateral filtering and segment-based surface structure propagation is introduced. Our experimental results prove the proposed method outperforms existing methods.
Model-based face reconstruction using SIFT flow registration and spherical harmonics

In this paper, we propose a robust method for face reconstruction using a single color image. A 3D morphable model is used to reconstruct a smooth 3D face shape. To find the correspondence between model vertices and image pixels, landmarks are updated using SIFT flow which is illumination and rotation invariant. To reconstruct more detailed information, depth values are refined using a shape from shading method which approximates lighting condition by spherical harmonics. We test the proposed method on a set of real world images and compare reconstructed results with depth maps captured by a depth camera. The average error is around 3.3 mm.
Post shot boundary detection technique: flashlight scene determination

Shot boundary detection, or scene change detection, is a technique used in the initial phase of video indexing. One of the problems in detection is the discrimination of abrupt scene change from flashlight scenes. Current method of detecting flashlight is based upon object edges, in which performance can be affected by the contents of the scene. To overcome this, we present a novel method that utilises the edge direction, reducing erroneous matching with increasing dilation radius. This will improve the accuracy of similarity testing and reduce the amount of erroneously matched edges by at least four times. Our experiment in discriminating flashlight effects from abrupt scene change frame pairs shows that our technique produces a perfect detection, which cannot be achieved by feature-based detection. Such contribution is important as it improves the indexing of real life video used in the upcoming video database standard MPEG-7.
Towards rate-distortion tradeoff in real-time color video coding

In this paper, we address to the key problem in real-time video coding, the rate-distortion tradeoff. We found that the distribution of the integer transform coefficients in H.264 color video coding can be approximated by Laplacian distribution but the rate-distortion characteristics should be modelled for the luminance and chrominance components separately. So we propose separable rate-distortion models for color video coding. The feedback from the encoder buffer is analyzed by a control-theoretic adaptation approach. To achieve smooth video quality and satisfy the delay constraints in real-time applications, a novel rate distortion tradeoff controller is employed. Simulations results demonstrate that the proposed approach can achieve smooth video quality without sacrificing overall quality while efficiently utilize the bandwidth without buffer overflow or underflow
Recent advances and challenges of visual signal quality assessment

While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this paper, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D images/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.
Integrated shot boundary detection using object-based technique

Shot boundary detection is a technique used in the first stage of video parsing, whereby shots are extracted before objects are segmented and indexed. Most traditional techniques detect transitions by testing for the feature differences between frames, and tend to fail under long transition with large disturbance. This problem is solved in this paper by considering shot boundary from a different perspective and treating transitions as periods with temporal object discontinuity. A novel concept known as object-based shot boundary detection, which utilises frame lifetime for detection, is introduced. This technique is superior to traditional techniques as it is transition length independent disturbance resistant, transition type independent and does not need a trial-and-error threshold. Object-based detection logically becomes a preferable technique to be used in fully automatic shot extraction systems for the upcoming MPEG-7 platform.
2-D Order-16 Integer Transforms for HD Video Coding

In this paper, the spatial properties of high-definition (HD) videos are investigated based on a large set of HD video sequences. Compared with lower resolution videos, the prediction errors of HD videos have higher correlation. Hence, we propose using 2-D order-16 transforms for HD video coding, which are expected to be more efficient to exploit this spatial property, and specifically propose two types of 2-D order-16 integer transforms, nonorthogonal integer cosine transform (ICT) and modified ICT. The former resembles the discrete cosine transform (DCT) and is approximately orthogonal, of which the transform error introduced by the nonorthogonality is proven to be negligible. The latter modifies the structure of the DCT matrix and is inherently orthogonal, no matter what the values of the matrix elements are. Both types allow selecting matrix elements more freely by releasing the orthogonality constraint and can provide comparable performance with that of the DCT. Each type is integrated into the audio and video coding standard (AVS) Enhanced Profile (EP) and the H.264 high profile (HP), respectively, and used adaptively as an alternative to the 2-D order-8 transform according to local activities. At the same time, many efforts have been devoted to further reducing the complexity of the 2-D order-16 transforms and specially for the modified ICT, a fast algorithm is developed and extended to a universal approach. Experimental results show that 2-D order-16 transforms provide significant performance improvement for both AVS enhanced profile and H.264 high profile, which means they can be efficient coding tools especially for HD video coding.
Learn to segment attention object from low DoF image

In this paper, a novel segmentation algorithm is proposed to extract attention object (i.e., focus object) from Low depth of field image. In order to recognize the focus object, we first decompose the image into multiple segments that are described by visual words. Each visual word is computed from a filter bank to represent the high frequency components. The boosting method is then used to generate a strong classifier for each training image. Given a test image, we employ the voting algorithm to achieve the attention decision according to obtained strong classifiers. To extract focus objects from the test image, two-level segmentation method is proposed, which includes region and pixel levels segmentation. Experimental evaluation on test images shows that the proposed method is capable of segmenting the attention object quite effectively.
Parametric Interpolation Filter for HD Video Coding

Recently, adaptive interpolation filter (AIF) for motion-compensated prediction (MCP) has received increasing attention. This letter studies the existing AIF techniques, and points out that making tradeoff between the two conflicting aspects: the accuracy of coefficients and the size of side information, is the major obstacle to improving the performance of the AIF techniques that code the filter coefficients individually. To overcome this obstacle, parametric interpolation filter (PIF) is proposed for MCP, which represents interpolation filters by a function determined by five parameters instead of by individual coefficients. The function is designed based on the fact that high frequency energies of HD video source are mainly distributed along the vertical and horizontal directions; the parameters are calculated to minimize the energy of prediction error. The experimental results show that PIF outperforms the existing AIF techniques and approaches the efficiency of the optimal filter.
Perceptual video coding: Challenges and approaches

Investigation on the human perception can play an important role in video signal processing. Recently, there has been great interest in incorporating the human perception in video coding systems to enhance the perceptual quality of the represented visual signal. However, the limited understanding of the human visual system and high complexity of computational models of human visual system make it a challenging task. Furthermore, the hybrid video coding structure brings difficulties to integrate computational models with coding components to fulfill the requirements. In this paper, we review the physiological characteristics of human perception and address the most relevant aspects to video coding applications. Moreover, we discuss the computational models and metrics which guide the design and implementation of the video coding system, as well as the recent advances in perceptual video coding. To introduce this overview with the latest technologies and most promising directions in perceptual video coding, we focus on three key areas. Specifically, we cover 1) visual attention and sensitivity modeling, with which we concentrate on the computational models of bottom-up and top-down attention, contrast sensitivity functions and masking effects, and fovea based manipulations; 2) perceptual quality optimization for constrained video coding, with which we discuss how to achieve maximum perceptual quality whilst satisfying various constraints; and 3) the impact of the human perception on advanced video applications, including emerging immersive multimedia services, and compression of high dynamic range video content and 3D video. For each aspect, we discuss the major challenges, highlight significant approaches, and outline future research directions.
Rank Learning Based No-Reference Quality Assessment of Retargeted Images

In this paper, we first propose a novel no-reference (NR) image quality assessment (IQA) method for retargeted image based on the rank learning approach. Firstly, image features for each retargeted image are extracted, which should not only represent the image characteristics but also be sensitive to the retargeted distortions. Specifically, the image feature should be able to capture the shape distortions, which are the commonly encountered distortions of the retargeted image. Based on the extracted image features, the rank learning method is employed to train a model to discriminate the perceptual quality of the retargeted image. Experimental results demonstrate that the proposed method can effectively depict the perceptual quality of the retargeted image, which can even perform comparably with the full-reference (FR) quality assessment methods.
Sequency, frequency, code or time? [DS-SS diversity systems]

This paper presents novel results from computer simulations for different hybrid multiplexing direct sequence spread spectrum systems operation in an indoor wireless environment. Prominent hybrid multiplexing direct sequence spread spectrum proposals using code, frequency and time diversity approaches have been analysed and compared to novel approaches proposed by the authors using sequency diversity. The results in this paper have shown that under certain practical wireless scenarios, the proposed sequency majority multiplexing direct sequence spread spectrum schemes exhibit very significant improvements over the compared current prominent hybrid multiplexing multiple access schemes.
Automatic video segmentation and tracking for content-based applications

Advanced multimedia applications have to provide content-related functionalities such as search and retrieval of meaningful objects, detection and analysis of events, and understanding of scenes, which allow the user to access and manipulate the multimedia content with greater flexibility. This greatly depends on automatic techniques for extracting such objects from multimedia data. In this article we intend to provide a tutorial on the state-of-the-art in video segmentation and tracking technology with particular attention paid to the recent developments in attention-based object extraction. Performance results are included to highlight this emerging technology
Video content dependent directional transform for intra frame coding

The mode-dependent directional transform (MDDT) employed Karhunen-Loève Transform (KLT) for compressing directional residue signal of intra prediction along its direction. The transform bases were derived from the singular value decomposition (SVD) of residue signals coming from all kinds of video sequences, which were expected to be efficient for most of video sequences. However, the advantage of KLT comes from the concept of a “signal content dependent transform”. MDDT and its variants failed to exploit such a concept, so they did not fully exploit the efficiency of KLT. In this paper, a video content feature is firstly defined as the histogram of the residue produced by intra prediction. Secondly, one KLT basis is computed for each feature of each mode from off-line experiments. Thus, multiple KLT bases identified by their features are provided to each mode instead of only one basis in MDDT. One of them is selected during encoding process by matching the feature of signal being processed to the predefined features. The experiments show that the average improvement of 0.17dB PSNR and 2.23% bits saving can be achieved by the proposed video content dependent directional transform (CDDT) comparing to the state-of-the-art MDDT.
Segmentation and Tracking Multiple Objects Under Occlusion From Multiview Video

In this paper, we present a multiview approach to segment the foreground objects consisting of a group of people into individual human objects and track them across the video sequence. Depth and occlusion information recovered from multiple views of the scene is integrated into the object detection, segmentation, and tracking processes. Adaptive background penalty with occlusion reasoning is proposed to separate the foreground regions from the background in the initial frame. Multiple cues are employed to segment individual human objects from the group. To propagate the segmentation through video, each object region is independently tracked by motion compensation and uncertainty refinement, and the motion occlusion is tackled as layer transition. The experimental results implemented on both our sequences and other's sequence have demonstrated the algorithm's efficiency in terms of subjective performance. Objective comparison with a state-of-the-art algorithm validates the superior performance of our method quantitatively.
Quality Enhancement in H.264 Transform Domain Downsizing

In the H.264 coding standard, an integer cosine transform (ICT) is adopted for transform coding. As the ICT is an integer approximation to the discrete cosine transform (DCT), its performance is different from that of the DCT. When the scheme of discarding high frequency coefficients is employed in the ICT domain downsizing, the output quality is worse than when done in the DCT domain. In this paper, the authors propose a scheme to enhance the output quality by converting the downsizing operation back to the DCT domain. The experimental results show that the proposed schemes present quality enhancement over the ICT domain downsizing.
Free-Energy Principle Inspired Video Quality Metric and Its Use in Video Coding

In this paper, we extend the free-energy principle to video quality assessment (VQA) by incorporating with the recent psychophysical study on human visual speed perception (HVSP). A novel video quality metric, namely the free-energy principle inspired video quality metric (FePVQ), is therefore developed and applied to perceptual video coding optimization. The free-energy principle suggests that the human visual system (HVS) can actively predict “orderly” information and avoid “disorderly” information for image perception. Basically, “orderly” is associated with the skeletons and edges of objects, and “disorderly” mostly concerns textures in images. Based on this principle, an image is separated into orderly and disorderly regions, and processed differently in image quality assessment. For videos, visual attention, or fixation, is associated with the objects with significant motion according to HVSP, resulting in a motion strength factor in the FePVQ so that the free-energy principle is extended into spatio-temporal domain for VQA. In addition, we investigate the application of the FePVQ in perceptual rate distortion optimization (RDO). For this purpose, the FePVQ is realized with low computational cost by using the relative total variation model and the block-wise motion vectors of video coding to simulate the free-energy principle and the HVSP, respectively. The experimental results indicate that the proposed FePVQ is highly consistent with the HVS perception. The linear correlation coefficient and Spearman's rank-order correlation coefficient are up to 0.8324 and 0.8281 on the LIVE video database. Better perceptual quality of encoded video sequences is achieved by FePVQ-motivated RDO in video coding.
A region-based algorithm for enhancement of images degraded by blocking effects

A new region-based method for removing blocking artifacts in block coded images is presented. The proposed algorithm segments the degraded image and each region obtained by the segmentation process is then low-pass filtered separately. This prevents blurring of edges because the low-pass filter is never applied across region boundaries. Experiments show that the proposed algorithm is comparable to other techniques at removing blocking artifacts, however, the undesired blurring introduced by the low-pass filter is significantly smaller.