nils-herrmann/clean_scirepeval huggingface dataset:
Low-rank approximation based LineCast for video broadcasting

Recent years have witnessed the success of the proposed line-based distributed coding and transmission system for broadcasting satellite images (LineCast). The LineCast achieves high compression efficiency and provides smooth multicast capability with soft broadcast and coset coding, in which the precision of side information is crucial. However, the generated side information is rough by averaging several candidates based on template matching. In this paper, we propose a low-rank approximation based LineCast scheme for video broadcasting (LRALineCast) to generate more precise side information by exploiting the cross-frame correlation. The side information generation problem is thus reformulated as a low-rank matrix completion problem, where the approximately low-rank matrix is rearranged by the similar lines searched by template matching. LRALineCast achieves good broadcast performance and avoids the annoying cliff effect by utilizing soft broadcast transmission. The experimental results show that LRALineCast outperforms the state-of-the-art 2D broadcasting scheme and LineCast with up to 2dB coding gain.
Multi-scale face hallucination based on frequency bands analysis

In this paper, a multi-scale face hallucination method is proposed to produce high-resolution (HR) face images from low-resolution (LR) ones according to the specific face characteristics and priors based on frequency bands analysis. In the first scale, the middle-resolution (MR) images are generated based on a patch-based learning method in DCT domain. In this scale, the DC coefficients and AC coefficients are estimated separately. In the second scale, a DCT upsampling for low frequency band restoration and a high frequency band restoration are combined to generate the final high-resolution face images. Extensive experiments show that the proposed algorithm achieves significant improvement.
Block-Based Compressive Sensing Coding of Natural Images by Local Structural Measurement Matrix

Gaussian random matrix (GRM) has been widely used to generate linear measurements in compressive sensing (CS) of natural images. However, in practice, there actually exist two problems with GRM. One is that GRM is non-sparse and complicated, leading to high computational complexity and high difficulty in hardware implementation. The other is that regardless of the characteristics of signal the measurements generated by GRM are also random, which results in low efficiency of compression coding. In this paper, we design a novel local structural measurement matrix (LSMM) for block-based CS coding of natural images by utilizing the local smooth property of images. The proposed LSMM has two main advantages. First, LSMM is a highly sparse matrix, which can be easily implemented in hardware, and its reconstruction performance is even superior to GRM at low CS sampling sub rate. Second, the adjacent measurement elements generated by LSMM have high correlation, which can be exploited to greatly improve the coding efficiency. Furthermore, this paper presents a new framework with LSMM for block-based CS coding of natural images, including measurement generating, measurement coding and CS reconstruction. Experimental results show that the proposed framework with LSMM for block-based CS coding of natural images greatly enhances the existing CS coding performance when compared with other state-of-the-art image CS coding schemes.
Fast encoder decision for dependent texture coding in 3D-AVS

As a 3D extension of the Audio Video coding Standard (AVS), 3D-AVS is used to improve the coding efficiency of multi-view video. However, the improved performance has also resulted in increased coding complexity. Taking into account the real-time request, how to reduce the complexity of the encoder is becoming very critical. In this paper, a fast encoder decision algorithm including two strategies is proposed. The first one is an early skip/direct mode decision algorithm, and the second one is an adaptive CU minimum splitting algorithm. Both of the two algorithms are applied to dependent texture views to reduce unnecessary visits by exploiting inter-view correlations between the dependent view and the base view. Experimental results show that the proposed algorithm can save 34.2% encoding time with 0.07% BD-rate loss on the overall texture views compared to RFD-4.0 (3D-AVS test model) under the common test condition (CTC). Both of the two strategies have been adopted into the 3D-AVS reference software and disabled in the configuration file under CTC.
Convolutional Neural Networks Based Intra Prediction for HEVC

Traditional intra prediction methods for HEVC rely on using the nearest reference lines for predicting a block, which ignore much richer context between the current block and its neighboring blocks and therefore cause inaccurate prediction especially when weak spatial correlation exists between the current block and the reference lines. To overcome this problem, in this paper, an intra-prediction convolutional neural network (IPCNN) is proposed for intra prediction, which exploits the rich context of the current block and therefore is capable of improving the accuracy of predicting the current block. Meanwhile, the reconstruction of the three nearest blocks can also be refined. To the best of our knowledge, this is the first paper that directly applies CNNs to intra prediction for HEVC. Experimental results validate the effectiveness of applying CNNs to intra prediction and the proposed method can achieve 0.70% bitrate reduction compared to HEVC reference software HM-14.0.
From relation between filter-based MRFs model and sparsity based method to the pursuit of natural images space

In the pursuit of natural image prior, responses to the specific filter bank and the character of sparse representation are of the most important clues. Based on these clues, many effective and successful algorithms are proposed and widely used in low vision tasks. Up to now, the corresponding researches with these clues are developed in relatively independent ways. In this paper, taking K-SVD as an example of sparse representation and Fields of experts (FoE) as an example of responses to the specific filter bank, we demonstrate the inherent relationship between them. The filters of FoE stand for the components that fire rarely on natural images, while the redundant dictionary of K-SVD depicts the primary components to some extent. They are two complementary pursuits of natural images space. We further bridge the gap between these two methods by proposing a method to get adaptive filters for FoE from the redundant dictionary of K-SVD. Instead of pursuing the state-of-the-art performance, our research gives a suggestive and unique view point from the essence of natural image space pursuit.
Joint gaze-correction and beautification of DIBR-synthesized human face via dual sparse coding

Gaze mismatch is a common problem in video conferencing, where the viewpoint captured by a camera (usually located above or below a display monitor) is not aligned with the gaze direction of the human subject, who typically looks at his counterpart in the center of the screen. This means that the two parties cannot converse eye-to-eye, hampering the quality of visual communication. One conventional approach to the gaze mismatch problem is to synthesize a gaze-corrected face image as viewed from center of the screen via depth-image-based rendering (DIBR), assuming texture and depth maps are available at the camera-captured viewpoint(s). Due to self-occlusion, however, there will be missing pixels in the DIBR-synthesized view image that require satisfactory filling. In this paper, we propose to jointly solve the hole-filling problem and the face beautification problem (subtle modifications of facial features to enhance attractiveness of the rendered face) via a unified dual sparse coding framework. Specifically, we first train two dictionaries separately: one for face images of the intended conference subject, one for images of “beautiful” human faces. During synthesis, we simultaneously seek two code vectors - one is sparse in the first dictionary and explains the available DIBR-synthesized pixels, the other is sparse in the second dictionary and matches well with the first vector up to a restricted linear transform. This ensures a good match with the intended target face, while increasing proximity to “beautiful” facial features to improve attractiveness. Experimental results show naturally rendered human faces with noticeably improved attractiveness.
Spatially directional predictive coding for block-based compressive sensing of natural images

A novel coding strategy for block-based compressive sensing named spatially directional predictive coding (SDPC) is proposed, which efficiently utilizes the intrinsic spatial correlation of natural images. At the encoder, for each block of compressive sensing (CS) measurements, the optimal prediction is selected from a set of prediction candidates that are generated by four designed directional predictive modes. Then, the resulting residual is processed by scalar quantization (SQ). At the decoder, the same prediction is added onto the de-quantized residuals to produce the quantized CS measurements, which is exploited for CS reconstruction. Experimental results substantiate significant improvements achieved by SDPC-plus-SQ in rate distortion performance as compared with SQ alone and DPCM-plus-SQ.
Sparsity-based joint gaze correction and face beautification for conferencing video

A well-known problem in video conferencing is gaze mismatch. Instead of relying exclusively on online captured data for rendering, a recent work first trains offline dictionaries using a large image database of movie and TV stars to learn "beautiful" features. During real-time conferencing, one can then simultaneously correct gaze and beautify the subject's facial components in single images by seeking sparse linear combination of pre-trained dictionary atoms for face reconstruction. Extending on this work, we focus on joint gaze correction / face beautification for video. First, we define a large search space invariant to scale, shift and rotation for facial feature beautification based on SIFT. We then address two practical issues unique to video: i) how beautified results can be temporally consistent across group of pictures (GOP), and ii) how blinking eyes can be beautified even though the training database contains only open-eye facial images. Experimental results show that our method achieves the desired temporal consistency, and the blinking process is smooth and natural.
Wyner-Ziv Switching Scheme for Multiple Bit-Rate Video Streaming

This paper proposes a Wyner-Ziv (WZ) switching scheme for multiple bit-rate (MBR) video streaming over networks. Identical video content is encoded into a set of normal streams, which are generated by conventional hybrid video coding with multiple bit rates, so that streaming can dynamically switch among these normal streams according to available bandwidth. At encoder side, the WZ codec generates a switching stream by compressing the reconstructed frames of a certain normal stream that will be switched to, no matter which normal stream it switches from. At decoder side, for switching to the same frame, the same WZ switching stream is used to reconstruct the switching-to frame by taking the switching-from frame as the side information. The number of required WZ bits depends on the inherent mutual correlation between two frames switching to and from. Since the WZ switching streams are generated independently of the normal switching-from streams, given normal streams that can switch from any one to another, the proposed scheme reduces the number of switching streams from to . Furthermore, switching streams do not deteriorate the coding efficiency of normal streams when no switching occurs. However a big problem here, similar to requesting bits in distributed video coding, is how many WZ bits should be transmitted when a switching happens because the streaming scenario does not tolerate too much extra delay caused by the requests back and forth. Therefore, a Laplacian model, which is proved in the simplified case, is proposed to characterize the correlation between switching-to and switching-from frames. It can be used to accurately estimate the number of WZ bits at the server side.
Up-sampling Dependent Frame Rate Reduction for Low Bit-Rate Video Coding

In low bit rate video coding, the frame rate of input sequence can be reduced to the half or even smaller portion by skipping or deleting frames before compression, and then the temporal resolution is restored via up-sampling at the decoder side. Numerous algorithms have been developed to address the problem of temporal resolution improvement. Actually, the quality of up-sampled frames depends on not only the performance of up-sampling method but also the information maintained in the down-sampled video sequence. To improve the quality of up-sampled frames and smooth the quality between the up-sampled and decompressed frames, this paper proposes an up-sampling dependent frame rate reduction.
Group-based sparse representation for low lighting image enhancement

The Group-based Sparse Representation (GSR) is able to sparsely represent natural images in the domain of group, which enforces the intrinsic local sparsity and nonlocal self-similarity of images simultaneously in a unified framework. And the GSR-driven L0 minimization method for image restoration has been proposed. This paper expands the application of GSR from image restoration to low lighting image enhancement. The GSR is not used to represent the natural images anymore, but representing the transmission map of the haze image and recovering it. Because the transmission map is very important for the low lighting image enhancement, the dark channel prior based enhancement method with the enhanced transmission map can get a better enhanced results. Different from other methods, we evaluate the quality of the enhanced images not only by qualitative analysis but also by quantitative results. Extensive experiments on low lighting image show that the GSR-based method gets a better enhancement result than many current state-of-the-art ones.
Hand gesture recognition based on skeleton of point clouds

In this paper, we present a method of recognizing hand gestures in the form of point clouds recorded by Kinect sensor. Firstly, through Laplacian-based contraction and further processing, we extract skeleton points from point clouds of hands. Then, we apply a novel partition-based descriptor and corresponded algorithm to classify these skeletons and, taking one step further, to recognize gestures. In the process of recognition, the issue of scale variant and rotation variant are solved. Finally, to test and verify performance of our method, we design a series of experiments. Experimental results proved both its accuracy and robustness. Besides, we believe the skeleton-based way of recognition owns potential for further exploration.
Viewpoint-independent hand gesture recognition system

In this paper, we creatively present a viewpoint-free hand gesture recognition system based on Kinect sensor. Through depth image, we build Point Clouds of user. Then, we estimate the current optimal viewpoint, i.e., the front, and project Point Clouds to that direction. Through that process we in great extent overcome the viewpoint-dependency issue. To match hand types, we propose an improved shape context to describe each hand gesture and use the Hungarian algorithm to calculate match degree. Our method is quite straightforward, however the experimental results prove that by this means gestures can be recognized independent of viewpoints with great accuracy. Besides, it is fast and robust, thus can be applied under various realistic scenarios in realtime.
Reduced reference stereoscopic image quality assessment based on entropy of classified primitives

Stereoscopic vision is a complex system which receives and integrates perceptual information from both monocular and binocular cues. In this paper, a novel reduced-reference stereoscopic image quality assessment scheme is proposed, based on the visual perceptual information measured by entropy of classified primitives (EoCP) and mutual information of classified primitives (MIoCP), named as DCprimary, sketch and texture primitives respectively, which is in accordance with the hierarchical progressive process of human visual perception. Specifically, EoCP of each-view image are calculated as monocular cue, and MIoCP between two-view images is derived as binocular cue. The Maximum (MAX) mechanism is applied to determine the perceptual information. The perceptual information differences between the original and distorted images are used to predict the stereoscopic image quality by support vector regression (SVR). Experimental results on LIVE phase II asymmetric database validate the proposed metric achieves significantly higher consistency with subjective ratings and outperforms state-of-the-art stereoscopic image quality assessment methods.
Multilayer method based on multi-resolution feature extracting and MVC dimension reducing method for sign language recognition

Hidden Markov model (HMM) has been successfully used in the sign language recognition (SLR). However, due to large vocabulary of the sign language, traditional one-layer HMM method is becoming limited with the increasing number of training samples. It is tardy when recognizing which cannot meet the real time requirement. In this paper, we present a multi-resolution feature extracting method and a reducing dimension method of maximum variance criterion (MVC), which has better performance in sign language recognition system than traditional reducing dimension methods of PCA or ICA. Our multilayer sign language recognition system increases the recognition accuracy by 3.42%, as well as reduces the recognition time by 0.992 second in average, compared with traditional HMM based system.
Image Entropy of Primitive and visual quality assessment

Recently, the concept of Entropy of Primitive (EoP) has been proposed to measure the image visual information. Some successful EoP based application also be developed. In this paper, we further explore the concept of EoP and propose an improved version: the L1 norm based EoP. Our EoP takes full account of the properties of a dictionary's layered structure and the characteristic of a basis pursuit method. Experimental results show that the L1 norm based EoP is superior to the L0 norm based one in measuring the image visual information. The curve of L1 norm based EoP holds a more consistent monotonicity with SSIM, its values is not trapped in the local convergence and the convergence value is less than that of the L0 norm based one. With the convergence characteristics of EoP, we further explore its application in stereoscopic image quality assessment (SIQA). With EoP as monocular cue and mutual information of primitive (MIP) as binocular cue, the relative entropy between the original stereoscopic image and the distorted one is used to compute the quality score by a prediction function which is trained using support vector regression (SVR). Extensive experimental results show that our new EoP based SIQA outperforms many state-of-the-art on the LIVE phase II databases.
Three-tiered network model for image hallucination

In this paper, we propose a novel three-tiered network model for image hallucination based on the learnt knowledge composed of image patches relating low and high resolution. A common problem of previous hallucination methods is that irregularities are usually introduced into the constructed high-resolution images. We remove the irregularities in three steps. First, the hallucination with primal sketch priors is performed to construct a coarse high-frequency component. Second, enhancement is implemented to enforce local compatibility between the patches in the constructed component. Third, a Markov network is utilized to refine the enhanced high-frequency component. Experiments demonstrate that our model can hallucinate higher-quality images than existing methods.
Stereoscopic video quality assessment based on stereo just-noticeable difference model

In this paper, we propose a full reference Stereoscopic Video Quality Assessment (SVQA) algorithm based on the Stereo Just-Noticeable Difference (SJND) model. Firstly, SJND mimic the human binocular visual system characteristics from four factors, including: sensitivity of luminance contrast, spatial masking, temporal masking and binocular masking. Secondly, based on the SJND model, the full reference SVQA is developed, by capturing spatio-temporal distortions and binocular perceptions. Finally, experimental results have demonstrated that the proposed SVQA outperforms other four current evaluation methods and has a good consistency with the observers' subjective perception.
Region-of-interest based coding scheme for synthesized video

In many multimedia applications, such as online speech, video chat and online conference, multiple source videos are synthesized in a single scene for explicit presentation and the synthesized video is compressed for transmission. The source video with important contents deserves more compression resources for quality preservation under the bandwidth constraint. To address this problem, a region-of-interest (ROI) based coding scheme for synthesized video is proposed in this paper aiming at achieve better and consistent quality for ROI source videos with the bitrate meeting the constraint bandwidth. In the proposed coding scheme, ROI based rate-distortion (R-D) models are established, in which different R-D models are built for different source video. Then an objective function is defined with respect to the video quality and the consistency of video quality. By minimizing the objective function, the optimal quantization parameters for the ROI and non-ROI source videos are obtained. The experimental results show that the proposed coding scheme achieves better and consistent quality for ROI source videos.
Building Emerging Pattern (EP) Random forest for recognition

The Random forest classifier comes to be the working horse for visual recognition community. It predicts the class label of an input data by aggregating the votes of multiple tree classifiers. However, the classification performances of these tree classifiers are different. The random forest classifier ignores the difference by simply assigning them equal weights in voting for the final classification decision. Also, the random forest classifier only casts votes from individual tree classifiers without considering their compositions which would be more accurate. In this paper, we propose to tackle the two points by discovering weighted decision rules from the tree classifiers' output sets on training data. By treating the outputs of the tree classifiers on each data as a digital itemset, we want to find discriminative patterns (either containing the output of a single tree classifier or a set of tree classifiers) from the itemsets of training data. We employ an efficient data mining algorithm, the Emerging Pattern (EP) Mining, to search such discriminative patterns and weight them according to their discriminative powers. A set of decision rules are built from these discovered patterns and the final outputs of the Random Forest are made using these decision rules. We call the proposed classifier Emerging Pattern (EP) Random Forest. Experimental results on action categorization problems confirm that the proposed method really improve the performance of the traditional Random Forest classifier.