nils-herrmann/clean_scirepeval huggingface dataset:
Maximizing user utility in video streaming applications

We study the design tradeoffs involved in video streaming in networks with QoS guarantees. We approach this problem by using a utility function to quantify the benefit a user derives from the received video sequence. This benefit is expressed as a function of the total distortion. In addition, we also consider the cost, in network resources, of a video streaming system. The goal of the network user is then to obtain the most benefit for the smallest cost. We formulate this utility maximization problem as a joint constrained optimization problem. The difference between the utility and the network cost is maximized subject to the constraint that the decoder buffer does not underflow. We present a deterministic dynamic programming approach to find the optimal tradeoff for both the constant bit rate (CBR) and renegotiated constant bit rate (RCBR) service classes. Experimental results demonstrate the benefits and the performance of the proposed approach.
Energy efficient wireless video communications for the digital set-top box

In the future, digital set-top boxes may serve as the primary access point for wireless home networks, enabling mobile users to use videoconferencing as well as streaming applications on hand-held devices. In this scenario, an important issue that must be addressed is the limited energy supply of a mobile device. This is of course a relevant issue for any wireless device. We focus on methods for efficiently utilizing transmission energy in wireless video communications. We present a general framework for the problem of minimizing the transmission energy required to provide an acceptable level of video quality. We discuss two special cases in which communication resources are adjusted simultaneously with the source coding parameters in order to provide (i) packet loss adaptation and (ii) transmission rate adaptation.
EM-based simultaneous registration, restoration, and interpolation of super-resolved images

A maximum likelihood (ML) solution to the problem of obtaining high-resolution images from sequences of noisy, blurred, and low-resolution images is presented. In our formulation, the registration parameters of the low-resolution images, the degrading blur, and noise variance are unknown. Our algorithm has the advantage that all unknown parameters are obtained simultaneously using all of the available data. An efficient implementation is presented in the frequency domain, based on the expectation maximization (EM) algorithm. Simulations demonstrate the effectiveness of the algorithm.
Regularized estimation of occluded displacement vector fields

Occluded regions and motion boundaries introduce displacement vector field (DVF) discontinuities that must be reconciled to accurately estimate image flow. In this work, the robust regularized estimation of the DVF is considered in the presence of these discontinuities. A robust convex estimation criterion is presented that preserves motion boundaries and allows for a globally optimal estimate of the DVF. A new class of robust convex measures is introduced for edge preserving regularization and an occlusion weighted gradient is proposed as mechanism for managing DVF discontinuities due to occlusion. Results using synthetic image sequences are presented.
Feature space video stream consistency estimation for dynamic stream weighting in audio-visual speech recognition

Most current audio-visual automatic speech recognition (AV- ASR) systems use static weights to leverage between audio and visual information during information fusion. State of the art research has led to using audio reliability metrics for dynamically changing the fusion weights in order to successfully improve overall recognition results. So far, however, incorporating visual reliability metrics into these audio reliability metric based systems have not significantly improved performance. We introduce a new approach to this problem by inferring the "consistency" between the audio and visual information and leveraging the existing audio reliability metrics to create a video reliability metric. Our approach is formulated in the extracted feature space and, thus, does not rely on analyzing the actual video signal itself. The framework presented in this work competes with the audio-only reliability metric based systems and shows promise to consistently outperform.
Robust and Low-Rank Representation for Fast Face Identification With Occlusions

In this paper, we propose an iterative method to address the face identification problem with block occlusions. Our approach utilizes a robust representation based on two characteristics in order to model contiguous errors (e.g., block occlusion) effectively. The first fits to the errors a distribution described by a tailored loss function. The second describes the error image as having a specific structure (resulting in low-rank in comparison with image size). We will show that this joint characterization is effective for describing errors with spatial continuity. Our approach is computationally efficient due to the utilization of the alternating direction method of multipliers. A special case of our fast iterative algorithm leads to the robust representation method, which is normally used to handle non-contiguous errors (e.g., pixel corruption). Extensive results on representative face databases (in constrained and unconstrained environments) document the effectiveness of our method over existing robust representation methods with respect to both identification rates and computational time.
Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation

The application of regularization to ill-conditioned problems necessitates the choice of a regularization parameter which trades fidelity to the data with smoothness of the solution. The value of the regularization parameter depends on the variance of the noise in the data. The problem of choosing the regularization parameter and estimating the noise variance in image restoration is examined. An error analysis based on an objective mean-square-error (MSE) criterion is used to motivate regularization. Two approaches for choosing the regularization parameter and estimating the noise variance are proposed. The proposed and existing methods are compared and their relationship to linear minimum-mean-square-error filtering is examined. Experiments are presented that verify the theoretical results.
Robust Recovery of Temporally Smooth Signals From Under-Determined Multiple Measurements

In this paper, we consider the problem of recovering jointly sparse vectors from underdetermined measurements that are corrupted by both additive noise and outliers. This can be viewed as the robust extension of the Multiple Measurement Vector (MMV) problem. To solve this problem, we propose two general approaches. As a benchmark, the first approach preprocesses the input for outlier removal and then employs state-of-the-art technologies for signal recovery. The second approach, as the main contribution of this paper, is based on formulation of an innovative regularized fitting problem. By solving the regularized fitting problem, we jointly remove outliers and recover the sparse vectors. Furthermore, by exploiting temporal smoothness among the sparse vectors, we improve noise robustness of the proposed approach and avoid the problem of over-fitting. Extensive numerical results are provided to illustrate the excellent performance of the proposed approach.
Iterative color image restoration algorithms

Iterative techniques for restoring colour images are presented. These iterative algorithms have been developed for the restoration of monochromatic images. The colour images are modeled as three spatially related monochromatic images or channels. The interchannel correlation is incorporated into the iterative restoration algorithms in an indirect and in a direct way. Experimental results with simulated and real photographically blurred images are presented. Based on experiments the authors conclude that the incorporation of the interchannel correlation into the algorithms does not necessarily improve the quality of the restored images over the algorithms that ignore the interchannel correlation, but they result in considerable computation savings.<<ETX>>
X-Ray fluorescence image super-resolution using dictionary learning

X-Ray fluorescence (XRF) scanning of works of art is becoming an increasingly popular non-destructive analytical method. The high quality XRF spectra is necessary to obtain significant information on both major and minor elements used for characterization and provenance analysis. However, there is a trade-off between the spatial resolution of an XRF scan and the Signal-to-Noise Ratio (SNR) of each pixel's spectrum, due to the limited scanning time. In this paper, we propose an XRF image super-resolution method to address this trade-off, thus obtaining a high spatial resolution XRF scan with high SNR. We use a sparse representation of each pixel using a dictionary trained from the spectrum samples of the image, while imposing a spatial smoothness constraint on the sparse coefficients. We then increase the spatial resolution of the sparse coefficient map using a conventional super-resolution method. Finally the high spatial resolution XRF image is reconstructed by the high spatial resolution sparse coefficient map and the trained spectrum dictionary.
Content-aware resource allocation and packet scheduling for video transmission over wireless networks

A cross-layer packet scheduling scheme that streams pre-encoded video over wireless downlink packet access networks to multiple users is presented. The scheme can be used with the emerging wireless standards such as HSDPA and IEEE 802.16. A gradient based scheduling scheme is used in which user data rates are dynamically adjusted based on channel quality as well as the gradients of a utility function. The user utilities are designed as a function of the distortion of the received video. This enables distortion-aware packet scheduling both within and across multiple users. The utility takes into account decoder error concealment, an important component in deciding the received quality of the video. We consider both simple and complex error concealment techniques. Simulation results show that the gradient based scheduling framework combined with the content-aware utility functions provides a viable method for downlink packet scheduling as it can significantly outperform current content-independent techniques. Further tests determine the sensitivity of the system to the initial video encoding schemes, as well as to non-real-time packet ordering techniques.
Nonstationary AR modeling and constrained recursive estimation of the displacement field

An approach to constrained recursive estimation of the displacement vector field (DVF) in image sequences is presented. An estimate of the displacement vector at the working point is obtained by minimizing the linearized displaced frame difference based on a set of observations that belong to a causal neighborhood (mask). An expression for the variance of the linearization error (noise) is obtained. Because the estimation of the DVF is an ill-posed problem, the solution is constrained by considering an autoregressive (AR) model for the DVF. A nonstationary AR model of the DVF is also considered. Additional information about the solution is incorporated into the algorithm using a causal oriented smoothness constraint. A set theoretic regularization approach based on this formulation results in a weighted constrained least-squares estimation of the DVF. The algorithm shows an improved performance with respect to accuracy, robustness of occlusion, and smoothness of the estimated DVF when applied to typical videoconferencing scenes.
Directionally adaptive cubic-spline interpolation using optimized interpolation kernel and edge orientation for mobile digital zoom system

This paper presents a novel directionally adaptive cubic-spline interpolation method which is applicable to mobile camera digital zoom systems. The problems of conventional (linear and cubic-spline) and advanced interpolation exhibit blurring and jagging artifacts in the digitally zoomed image. To solve this problem, the proposed method performs directionally adaptive interpolation using the optimal interpolation kernel according to the edge orientation. Experimental results show that the proposed method successfully enlarges images with reduced interpolation artifacts compared with both conventional and advanced interpolation methods. Objective evaluation reveals that the proposed method gives higher peak signal to noise ratio (PSNR) and structural similarity (SSIM) figures.
Automated Recovery of Compressedly Observed Sparse Signals From Smooth Background

We propose a Bayesian based algorithm to recover sparse signals from compressed noisy measurements in the presence of a smooth background component. This problem is closely related to robust principal component analysis and compressive sensing, and is found in a number of practical areas. The proposed algorithm adopts a hierarchical Bayesian framework for modeling, and employs approximate inference to estimate the unknowns. Numerical examples demonstrate the effectiveness of the proposed algorithm and its advantage over the current state-of-the-art solutions.
Noise reduction filters for dynamic image sequences: a review

In this paper, a thorough review is presented of noise reduction filters for digital image sequences. Detailed descriptions of several spatiotemporal and temporal noise reduction algorithms are provided. To aid in comparing between these different algorithms, we classify them based on their support (i.e., 3-D or 1-D filter) and whether or not motion compensation is employed. Several algorithms from each of the four categories are implemented and tested on real sequences degraded to various signal-to-noise ratios. These experimental results are discussed and analyzed to determine the overall advantages and disadvantages of the four general classifications, as well as, the individual filters.
Maximizing user utility in video streaming applications

We study some of the design tradeoffs of video streaming systems in networks with QoS guarantees. We approach this problem by using a utility function to quantify the benefit a user derives from the quality of the received video sequence. We also consider the cost to the network user for streaming the video sequence. We have formulated this utility maximization problem as a joint constrained optimization problem where we maximize the difference between the utility and the network cost, subject to the constraint that the decoder buffer does not underflow. In this manner, we can find the optimal tradeoff between video quality and network cost. We present a deterministic dynamic programming approach for both the constant bit rate and renegotiated constant bit rate service classes. Experimental results demonstrate the benefits and the performance of the proposed approach.
Channel protection for H.264 compression in transportation video surveillance applications

The compression of video and subsequent partial loss of the compressed bitstream can dramatically reduce the accuracy of automated tracking algorithms. This is problematic for centralized applications such as transportation surveillance systems, where remotely captured and compressed video is transmitted over lossy wireless links to a central location for tracking. We propose a low-complexity method for protecting compressed video against channel loss such that the tracking accuracy of decoded and concealed video is maximized. Our algorithm leverages a previous method of video processing that removes components of low tracking interest before compression to minimize bitrate, and uses some of the bitrate savings to introduce redundancy into the transmitted bitstream to reduce the probability of information loss. We show using a common tracker and loss concealment algorithm that our system allows for up to 100% increased tracking accuracy at a given bitrate, or 90% bitrate savings for comparable tracking quality.
Passive millimeter wave image classification with large scale Gaussian processes

Passive Millimeter Wave Images (PMMWIs) are being increasingly used to identify and localize objects concealed under clothing. Taking into account the quality of these images and the unknown position, shape, and size of the hidden objects, large data sets are required to build successful classification/detection systems. Kernel methods, in particular Gaussian Processes (GPs), are sound, flexible, and popular techniques to address supervised learning problems. Unfortunately, their computational cost is known to be prohibitive for large scale applications. In this work, we present a novel approach to PMMWI classification based on the use of Gaussian Processes for large data sets. The proposed methodology relies on linear approximations to kernel functions through random Fourier features. Model hyperparameters are learned within a variational Bayes inference scheme. Our proposal is well suited for real-time applications, since its computational cost at training and test times is much lower than the original GP formulation. The proposed approach is tested on a unique, large, and real PMMWI database containing a broad variety of sizes, types, and locations of hidden objects.
Spline-based boundary loss concealment

Object-based video coding requires the transmission of the object shape. This shape is sent as a binary /spl alpha/-plane. In lossy packet-based networks, such as the Internet, this information has a nonnegligible probability of not arriving at the receiver, and hence its loss needs to be concealed. In this paper we propose a shape concealment technique utilizing Hermite splines. The algorithm has the following steps: (I) the received boundary is detected and the lost boundary parts are grouped using the packet loss pattern. (II) for each of these lost boundary parts, the received boundary points that border the area of the lost boundary parts are collected. These boundary points are then modelled by a second order Hermite spline. This model is subsequently used to match the velocity along the received boundary with the velocity of the concealing cubic Hermite spline. (Ill) since in most cases there are more than one concealing splines we draw every spline combination that does not result in an intersection and keep all possible results until the end. (IV) if there are more than one possible solutions we select the one that results in one overall closed nonintersecting boundary and fill the interior of the boundary to get the concealed /spl alpha/-plane. Experimental results which demonstrate and compare the performance of the proposed concealment method are given at the end of the paper.
Recursive displacement estimation and restoration of noisy-blurred image sequences

A recursive model-based maximum a posteriori (MAP) estimator that simultaneously estimates the displacement vector field (DVF) and intensity field from a noisy-blurred image sequence is developed. By simultaneously estimating these two fields, information is made available to each filter regarding the reliability of estimates that they are dependent upon. Nonstationary models are used for the DVF and the intensity field in the proposed estimator, thus avoiding the smoothing of boundaries present in both. The advantage of the proposed SDIE (simultaneous displacement and intensity field estimation) algorithm is that the error inherent in estimating the DVF is taken into account in the filtering of the intensity field. A second advantage is that, through the use of the nonstationary VCGM (vector coupled Gauss-Markov) and STCGM (spatiotemporal coupled Gauss-Markov) models, boundaries in both the DVF and the intensity fields are preserved. The performance of the proposed SDIE algorithm was shown to be superior to that of the Wiener-based PR algorithm and the 2-D Kalman filter in estimating the DVF and intensity field, respectively, from a noisy-blurred image sequence.<<ETX>>
Context based optimal shape coding

This paper investigates how context-based DPCM techniques can be used in conjunction with an operationally rate-distortion (ORD) optimized shape coder. Object contours are approximated by connected second-order spline segments, each defined by three consecutive control points, and, in the case of the inter mode, by segments of the motion-compensated reference contours. In scalable intra mode and non-scalable inter mode shape coding, consecutive control points are encoded predictively, using an angle and run framework, with respect to corresponding contexts in the reference frame and the base layer, respectively. We employ a novel criterion for selecting global object motion vectors in the inter mode, which further improves efficiency.
Efficient motion compensated frame rate upconversion using multiple interpolations and median filtering

Methods for motion compensated frame rate upconversion exploit motion in order to generate interpolated frames that are temporally located between the available original frames of a video sequence. When the interpolated frames are inserted into the original sequence, the resulting sequence has a higher visual quality due to the increased frame rate. We propose a method for motion compensated frame rate upconversion which reduces upconversion artifacts by combining multiple intermediate interpolations utilizing median filtering. We use an efficient block based motion estimation method which makes use of motion vectors extracted from an H.264 bitstream. Low computational complexity makes the method suitable for real time operation.
A rate-distortion optimal alternative to matching pursuit

This paper presents a method to find the operational rate-distortion optimal solution for an overcomplete signal decomposition. The idea of using overcomplete dictionaries, or frames, is to get a sparse representation of the signal. Traditionally, suboptimal algorithms, such as matching pursuit (MP), are used for this purpose. When using frames in a lossy compression scheme, the major issue is to find the best possible rate-distortion (RD) tradeoff. Given the frame and the variable length code (VLC) table embedded in the entropy coder, the solution to the problem of establishing the best RD tradeoff is highly complex. The proposed approach reduces this complexity significantly by structuring the solution approach such that the dependent quantizer allocation problem reduces to an independent one. In addition, the use of a solution tree further reduces the complexity. It is important to note that this large reduction in complexity is achieved without sacrificing optimality. The optimal rate-distortion solution depends on the selection of the frame and the VLC table embedded in the entropy coder. Thus, frame design and VLC optimization is part of this work. We experimentally demonstrate that the new approach outperforms rate-distortion optimized (RDO) matching pursuit, previously proposed by Gharavi-Alkhansari.
Noise robust spatial gradient estimation for use in displacement estimation

An important component of any spatial temporal gradient motion estimation algorithm is the accuracy by which spatial gradients are calculated. When an image sequence is corrupted by noise, the problem of determining these spatial gradients becomes extremely difficult. This is immediately apparent, since the magnitude response of the derivative operator is |/spl omega/|. In other words, the components of an image are amplified upon differentiation in proportion to their frequency value. Thus, high-frequency noise terms will dominate any low-frequency features in the differentiated image. If this corrupted differentiated image is then used within a spatio-temporal gradient motion estimator, the noise will erroneously influence the estimated motion vector. The problem of estimating the spatial gradient is treated as an inverse problem with noise. Formulating the problem in this manner results in a recursive gradient estimator that suppresses the effects of noise.