nils-herrmann/clean_scirepeval huggingface dataset:
A learning-based video compression on low-quality data by unscented kalman filters with Gaussian process regression

With the ever increasing concern of vision-based video analysis and coding over resource-limited systems, this paper proposes a novel video coding scheme that exploits low-quality video data and formulates as an inverse learning based video reconstruction from online training by diverse stochastic processes. Given a sparsely sampled incomplete data, the intrinsic nonlocal and spatio-temporal geometric regularity related to online training examples in the key frames are considered as a state-dependent uncertainty estimation problem using Gaussian Process (GP) regression. Unlike non-parametric or exemplar-based sampling methods, we consider non-parametric system models for sequential state estimation by using the Unscented Kalman Filter (UKF) as the state estimator. It inherits the unscented transform for linearization to the transition function and the observation function. Once an approximate motion and observation model is available, it can naturally be incorporated to make a further performance improvement.
Transductive Video Segmentation on Tree-Structured Model

This paper presents a transductive multicomponent video segmentation algorithm, which is capable of segmenting the predefined object of interest in the frames of a video sequence. To ensure temporal consistency, a temporal coherent parametric min-cut algorithm is developed to generate segmentation hypotheses based on visual cues and motion cues. Furthermore, each hypothesis is evaluated by an energy function from foreground resemblance, foreground/background divergence, boundary strength, and visual saliency. In particular, the state-of-the-art R-convolutional neural network descriptor is leveraged to encode the visual appearance of the foreground object. Finally, the optimal segmentation of the frame can be attained by assembling the segmentation hypotheses through the Monte Carlo approximation. In particular, multiple foreground components are built to capture the variances of the foreground object in shapes and poses. To group the frames into different components, a tree-structured graphical model named temporal tree is designed, where visually similar and temporally coherent frames are arranged in branches. The temporal tree can be constructed by iteratively adding frames to the active nodes by probabilistic clustering. In addition, each component, consisting of frames in the same branch, is characterized by a support vector machine classifier, which is learned in a transductive fashion by jointly maximizing the margin over the labeled frames and the unlabeled frames. As the frames from the same video sequence follow the same distribution, the transductive classifiers achieve stronger generalization capability than inductive ones. Experimental results on the public benchmarks demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art supervised and unsupervised video segmentation methods.
Delay-rate-distortion optimized rate control for wireless video communication

In this paper, we extend the traditional rate-distortion optimization (RDO) for the end-to-end wireless video communication system and develop a novel delay-rate-distortion optimization (dRDO) based rate control (RC) algorithm, by investigating the allocation of end-to-end delay to different delay components. Tradeoffs regarding, respectively, source coding delay versus buffering delay and available source coding rate versus redundant rate incurred by channel coding, are coupled in the proposed dRDO algorithm. It targets at minimizing the average total end-to-end distortion under the transmission rate and end-to-end delay constraints, by joint selection of search range and quantization step size in source coding and channel code rate in channel coding. Experimental results demonstrate the superiority of the proposed dRDO algorithm over existing schemes.
Lifetime and Distortion Optimization With Joint Source/Channel Rate Adaptation and Network Coding-Based Error Control in Wireless Video Sensor Networks

In this paper, we study joint performance optimization on network lifetime and video distortion for an energy-constrained wireless video sensor network (WVSN). To seek an appropriate tradeoff between maximum network lifetime and minimum video distortion, a framework for joint source/channel rate adaptation is proposed, where the video encoding rate, link rate, and power consumption are jointly considered, formulating a weighted convex optimization problem. In the context of lossy wireless channels, an efficient error control scheme that couples network coding and multipath routing is explored. Moreover, an integrated power consumption model, including power dissipation on video compression, error control, and data communication, is specifically developed for the video sensor node. By primal decomposition, the original problem is decomposed into a two-level optimization procedure, with the high-level procedure for source adaptation (source rate optimization) and the low-level procedure for channel adaptation (network resource allocation). Finally, a fully decentralized iterative algorithm is developed to resolve the target optimization problem. Extensive simulation results evaluate the convergence performance of the proposed algorithm and demonstrate the best tradeoff performance.
Strip based media retargeting via combing multi-operators

Based on the recently prioritized objectives and the statistical findings, we propose a novel media retargeting approach which considers content loss and deformation in a visual friendly manner. First, we divide an input image into several strips according to the average strip energy density and the line structure information. Then, we find the maximum resizing lengths which makes the deformation acceptable and choose the proper retargeting operator for each strip according to the energy information. The process above will be repeated iteratively until a cropping window can be selected which makes the content loss acceptable. The experimental results show that our proposed method can protect more important content information than traditional cropping and make less deformation than previous content-aware media retargeting method.
Image compression with structure-aware inpainting

This paper carves out a way to image compression that is motivated by the recent advancement in image inpainting. An image coding approach is proposed in which a number of regions of the input image are skipped at the encoder and are recovered through the inpainting process at the decoder. Furthermore, a structure-aware inpainting (SAI) method is developed to restore the skipped structural regions by taking advantage of the available portion of the decoded image. A binary structure map is extracted and compressed into the generated bit-stream to indicate the skipped regions with salient structures. By making use of the decoded texture information together with the structure map, the SAI method can recover the skipped structural regions as well as the non-structural ones effectively at the decoder. Compared with JPEG, our proposed image compression scheme allows smaller file, with the potential of up to 50% bit-saving capability, at similar visual quality levels
Delay–Rate–Distortion Optimized Rate Control for End-to-End Video Communication Over Wireless Channels

In addition to the rate-distortion (R-D) behavior, in a real-time wireless video communication system, the end-to-end delay would also significantly affect the overall video reception quality. To analyze, control, and optimize the R-D behavior under the end-to-end delay constraint, in this paper we extend the traditional R-D optimization (RDO) for the wireless video communication system and formulate a novel delay-RDO-based rate control problem, by investigating the allocation of end-to-end delay to different delay components. It aims at minimizing the average total end-to-end distortion under the transmission rate and end-to-end delay constraints, by a joint selection of both the source coding and the channel coding parameters. The wireless channel is represented by a finite-state Markov channel model characterizing the time-varying process and predicting the future channel condition. As applicable solutions, a practical algorithm based on the Lagrange multiplier approaches, Karush-Kuhn-Tucker conditions, and sequential quadratic programming methods is developed. The experimental results demonstrate the superiority of the proposed algorithm over the existing schemes.
Prioritized Flow Optimization with Generalized Routing for Scalable Multirate Multicasting

This paper addresses the performance optimization for scalable video coding and multicast over networks. Multi-path video streaming, network coding based routing, and network flow control are jointly optimized to maximize a network utility function defined over heterogeneous receivers. Importantly, contextual priors of scalable video layers are imposed on the flow routing optimization problem, seeking to guarantee the transmission cost for each layer in an incremental order and find jointly optimal multicast paths and associated rates. Through a primal decomposition and the primal-dual approach, a decentralized algorithm with two-level optimization update is developed to solve the target convex optimization problem. Numerical and simulation results validate the convergence and network performance of the proposed algorithm.
A Cross-Resolution Leaky Prediction Scheme for In-Band Wavelet Video Coding With Spatial Scalability

In most existing in-band wavelet video coding schemes, over-complete wavelet transform is used for the motion-compensated temporal filtering (MCTF) of each spatial subband. It can overcome the shift-variance of critical sampling wavelet transform and improve the coding efficiency of the in-band scheme. However, a dilemma exists in the current implementations of in-band MCTF (IBMCTF), which is whether or not to exploit the spatial highpass subbands in motion compensation of the spatial lowpass subband. The absence of the spatial highpass subbands will result in significant quality loss in the reconstructed full-resolution video, whereas the presence of the spatial highpass subbands may bring serious mismatch error in the decoded low-resolution video since the corresponding highpass subbands may be unavailable at the decoder. In this paper, we first analyze the mismatch error propagation in decoding the low-resolution video. Based on our analysis, we then propose a frame-based cross-resolution leaky prediction scheme for IBMCTF. It can make a good tradeoff between alleviating the low-resolution mismatch and improving the full-resolution coding efficiency. Experimental results show that the proposed scheme can dramatically reduce the mismatch error by 0.3-2.5 dB for low resolution, while the performance loss is marginal for high resolution.
Dynamic Spectrum Access and Power Allocation for Cooperative Cognitive Radio Networks

In the existing cooperative cognitive radio, primary users are generally assumed to be more than capable of supporting their target throughput, thereby the rate enhancement or the cooperation from secondary users actually has less attraction to primary users. Instead, they might be more interested in the benefits in other format. This paper presents a new cooperative cognitive radio framework, where primary users are willing to cooperatively relay data for secondary users with their under-utilized resource to earn the revenue. An auction model with multiple auctioneers, multiple bidders and hybrid divisible/indivisible commodities is proposed to solve channel bands and cooperative transmit power competitions among secondary users. For maximizing the utility, secondary users can select receiving primary user's assistance (i.e., work in cooperative transmission mode) and purchase both spectrum and power from the primary user, or select direct transmission mode and just buy spectrum from the primary user. Then, the convergence of the proposed auction scheme to a Walrasian equilibrium is mathematically proved. Finally, the performance of the proposed scheme is verified by the simulation results.
An error resilience scheme on an end-to-end distortion analysis for video transmission over Internet

In this paper, an end-to-end rate-distortion (R-D) estimation recursive model is proposed, on the basis of the statistical analysis of the spatial and temporal prediction as well as a general error concealment method by the video decoder. It takes the source quantization error, channel error, and the subsequent error spatial-temporal propagation into account. Then a corresponding integrated error resilience scheme, which adopts a global optimal intra-macroblock (MB) refreshment ratio and visual acuity selection procedure, is introduced. Moreover, it employs a combined source-channel bit allocation strategy, which could estimate an instantaneous available source transmission rate under a given time-varying network condition. The network experiments show that this scheme is effective and applicable for a better and more consistent picture quality in the receiver end.
Joint Spectrum and Power Auction With Multiauctioneer and Multibidder in Coded Cooperative Cognitive Radio Networks

In the existing cooperative cognitive radio (CR), qualities of service (e.g., rate or outage probability) of primary users can be improved by cooperative transmission from secondary users. However, as owners of the spectrum, primary users' traffic demands are relatively easy to satisfy. The rationale is that they would be more interested in the benefits of other formats (e.g., revenue), rather than the enhanced rate. In this paper, we propose a new cooperative CR framework, where primary users assist in the transmissions of secondary users. In exchange for this concession, primary users receive payments from secondary users for the spectrum and cooperative transmit power being used in cooperation. An auction-theoretic model with multiple auctioneers, multiple bidders, and multiple commodities is developed for a joint spectrum and cooperative power allocation. Since the spectrum and power are two heterogeneous but correlated commodities, their properties are specifically considered in auction strategy design. Finally, we mathematically prove the convergence of the proposed auction game, and show with numerical results, that the proposed auction is beneficial to both primary and secondary users.
A Simulation-Based 3D Axon Axis Extraction in Confocal Fluorescence Microscopy Images

Neuron axon analysis is a fundamental mean to investigate nervous system in neurobiology and often requires collecting a great amount of statistical information. Automated extraction of axons axis in 3D microscopic images posts a key problem in the field of neuron axon analysis. To address tortuous axons with several touchings in 3D volumes, a novel simulation based approach with regard to volume modeling and scenario mapping is presented. Given the simulation model, the properties of dataset and the nature of axon (e.g., continuity) are converted to physics laws including the principle of minimum potential energy, law of buoyancy, and law of inertia. Experimental results on axon volumes show the proposed algorithm can effectively handle axon touching problem and actually extract multiple axon axes.
Delay—Power-Rate-Distortion Model for Wireless Video Communication Under Delay and Energy Constraints

Smart mobile phones are capable of performing video coding and streaming over wireless networks, but are often constrained by the end-to-end delay requirement and energy supply. To achieve optimal performance under the delay and energy constraints, in this paper we extend the traditional rate-distortion (R-D) model and the previously proposed delay R-D model to a novel delay-power-rate-distortion (d-P-R-D) model by including another two dimensions (the encoding time and encoder power consumption), which quantifies the relationship among source encoding delay, rate, distortion, and power consumption for IPPPP coding mode in H.264/AVC. We have verified the accuracy of our proposed d-P-R-D model through experiments. Based on the proposed d-P-R-D model, we develop a novel rate-control (RC) algorithm, which minimizes the encoding distortion under the constraints of rate, delay, and power. The experimental results demonstrate the superiority of the proposed RC algorithm over the existing scheme. Therefore, the d-P-R-D model and the model-based RC provide a theoretical basis and a practical guideline for the cross-layer system design and performance optimization in wireless video communication under delay and energy constraints.
Scalable Video Compression Framework With Adaptive Orientational Multiresolution Transform and Nonuniform Directional Filterbank Design

Although wavelet-based scalable video coding becomes the state-of-the-art video compression engine for its adaptability to heterogeneous networks and clients, a large number of attempts have been made to integrate local directionality onto discrete wavelet transform to explore the intrinsic geometrical structures. Taking into consideration that the contours and textures scattered in different scales change their directional resolutions as their curvatures change, we investigate adaptive directional resolutions along scales to achieve the dual (scale and orientation) multiresolution transform. This paper proposes nonuniform directional frequency decompositions for video representation and approximation, and exploits the nonuniformity of orientation multiresolution distribution and designs nonuniform directional filter banks to make the geometrical transform more sparse and efficient. The nonuniform directional frequency decomposition under arbitrary scales is fulfilled by a non-symmetric binary tree (NSBT) topology structure with nonuniform directional filterbank design. In turn, the proposed scalable video coding framework, called DMSVC, is enriched with the dual multiresolution transform. Each temporal subband through motion compensated temporal filtering is further decomposed into multiscale subbands, and the highpass wavelet subspaces are divided into an arbitrary number of directional subspaces in alignment with the orientation distribution via phase congruency to establish NSBT. The paraunitary perfect reconstruction condition is provided through a polyphase identical form of filter bank. Comparing with the isolated wavelet basis, our transform provides a greater correlated set of localized and anisotropic basis functions. The spatio-temporal subband coefficients are coded by a 3-D ESCOT entropy coding algorithm which is adopted to match the structure of NSBT. Experimental results show that the reconstructed video frames DMSVC in the proposed DMSVC scheme have better visual quality than existing scalable video coding schemes. It could produce higher compression ratio on video sequences full of directional edges and textures.
Deformable Shape Preserving Video Retargeting With Salient Curve Matching

Video retargeting is dedicated to resizing the resolution of videos in a content aware manner, and it involves three critical challenges, namely, visual saliency preservation, deformation prevention, and temporal consistency persistence. Existing retargeting algorithms include seam carving based approaches, warping based approaches, and cropping based approaches, which mainly concentrate on saliency preservation and fail to prevent the deformation of salient shapes. This paper proposes a deformable shape preserving video retargeting scheme where salient curves extracted from frames are protected from deformation by minimizing the matching cost of curves in the original frames and the retargeted frames. Correspondingly, a curve matching algorithm is developed to generate the deformation cost which is invariant to translation, rotation and scaling with the Bookstein coordinate transform. In turn, the deformation cost of salient curves will be incorporated into the energy map of seam carving. Furthermore, the proposed scheme defines a temporal energy term to penalize the change of the relative position of curves in consecutive retargeted frames with respect to the original frames. Extensive experiments are validated by the visual comparison, user evaluation, deformation analysis and temporal consistency evaluation, which prove that the proposed scheme outperforms state-of-the-art video retargeting methods.
Optimization of variational methods via motion-based weight selection and keypoint matching

Variational method is a well-established technique that solves for a dense field, which is widely adopted in the estimation of optical flow field and remains the most accurate technique to date. However, one of the problems in variational method lies in that it is optimized in an iterative manner towards a single objective, but local details may be compromised owing to the “big picture”. In this paper, we address this problem in an optical flow framework by introducing two sparse local rectifications to the global numerical scheme, i.e., motion-based weight selection and keypoint matching. The selection of the weighting parameter in a self-adaptive and content-aware manner provides a more accurate estimation of the optical flow field near motion boundaries, and motion details and small structures are preserved in the optical flow field by keypoint matching in the initialization of the optical flow field. Experimental results using the Middlebury dataset show that the proposed algorithm achieves higher accuracy compared to the original TV-ℓ1 optical flow algorithm and many state-of-the-art methods.
Discriminative structured set prediction modeling with max-margin Markov network for optimal lossless image coding

In this paper, we investigate and propose a novel prediction model for lossless image coding in which the optimal correlated prediction for block of pixels are simultaneously obtained in the sense of the least code length. It not only utilizes the spatial statistical correlation for the optimal prediction directly based on 2-D contexts, but also formulates the data-driven structural interdependencies to make the prediction error coherent with the underlying probability distribution for coding. Besides the discriminative adaptive pixel-wise prediction, the Markov network is adaptively derived to maintain the coherence of prediction in the blocks and seek the concurrent optimization of set of prediction by relating the loss function to actual code length. The prediction error is shown to be asymptotically upper bounded by the training error under the decomposable loss function. For validation, we apply the proposed model into lossless image coding and experimental results show that the proposed scheme outperforms the best prediction scheme reported.
An Improved Update Operator for H.264 Scalable Extension

This paper presents an improved update operator for H.264 scalable extension, which is a new joint standardization effort of ISO/IEC and ITU-T. Energy distributed update (EDU) is utilized to reduce decoder complexity. The relationship between EDU and the optimum update in B. Girod et al., (2005) is revealed. Moreover, for the constrained motion-compensated lifting structure employed in H.264 scalable extension, we derive a set of new update coefficients to improve coding performance based on distortion analysis. Experimental results show that the proposed update operator can improve coding efficiency with less complexity and meanwhile, frame-wise quality fluctuation is also reduced