nils-herrmann/clean_scirepeval huggingface dataset:
High-resolution image reconstruction of low-resolution DCT-based compressed images

The problem of recovering a high-resolution image from a sequence of low-resolution DCT-based compressed images is considered in this paper. The presence of the compression system complicates the recovery problem, as the operation reduces the amount of frequency aliasing in the low-resolution frames and introduces a non-linear quantization process. The effect of the quantization error and resulting inaccurate sub-pixel motion information is modeled as a zero-mean additive correlated Gaussian noise. A regularization functional is introduced not only to reflect the relative amount of registration error in each low-resolution image but also to determine the regularization parameter without any prior knowledge in the reconstruction procedure. The effectiveness of the proposed algorithm is demonstrated experimentally.
A phone-viseme dynamic Bayesian network for audio-visual automatic speech recognition

This work extends and improves a recently introduced (Dec. 2007) dynamic Bayesian network (DBN) based audio-visual automatic speech recognition (AV-ASR) system. That system models the audio and visual components of speech as being composed of the same sub-word units when, in fact, this is not psycholinguistically true. We extend the system to model the audio and visual streams as being composed of separate, yet related, sub-word units. We also introduce a novel stream weighting structure incorporated into the model itself. In doing so, our system makes improvements in word error rate (WER) and overall recognition accuracy in a large vocabulary continuous speech recognition task (LVCSR). The ldquobestrdquo performing proposed system attains a WER of 66.71%whereas the ldquobestrdquo baseline system performs at a WER of 64.30%. The proposed system also improves accuracy to 45.95% from 39.40%.
Channel modeling and its effect on the end-to-end distortion in wireless video communications

A major limitation faced by a mobile user is their dependence on a limited battery supply. For wireless video communications, joint source coding and transmission power management (JSCPM) has recently been considered as a means of efficiently allocating transmission energy. In order to reduce complexity, the design of many of these adaptive resource allocation algorithms utilizes simplified channel models that do not account for the burstiness of the channel. We analyze the effects of such channel model simplifications on the end-to-end distortion. We present a channel model that is based on information theoretic considerations, which captures the bursty nature of wireless channels and accounts for packet lengths when calculating the probability of loss. Given the source coding and transmission parameters derived using a simplified channel model, our goal is to analyze how the end-to-end distortion is affected when a more realistic complex channel model is used to simulate losses. Experimental results suggest that the performance gain predictions for JSCPM using a simpler channel model are also valid when more sophisticated channel simulations are used, provided that a number of additional steps are taken after the optimization to account for the complex characteristics of wireless channels.
Joint data detection and channel tracking for OFDM systems using the variational bayes method

This paper addresses the problem of data detection in OFDM systems operating under a time-varying multipath fading channel. Optimal detection in such a scenario is infeasible, which makes the introduction of approximations necessary. The variational Bayes method is employed here, which approximates the joint data and channel distribution as a separable one, greatly simplifying the problem. The data detection part of the resulting algorithm provides soft data estimates that are used for channel tracking. The channel itself is modeled as an auto-regressive process allowing for a Kalman-like tracking algorithm. According to the developed algorithm, both data and channel estimates are exchanged and updated in an iterative manner. Performance of the proposed algorithm is evaluated by simulations.
Spatial-Spectral Representation for X-Ray Fluorescence Image Super-Resolution

X-ray fluorescence (XRF) scanning of works of art is becoming an increasing popular nondestructive analytical method. The high quality XRF spectra is necessary to obtain significant information on both major and minor elements used for characterization and provenance analysis. However, there is a tradeoff between the spatial resolution of an XRF scan and the signal-to-noise ratio (SNR) of each pixel's spectrum, due to the limited scanning time. In this project, we propose an XRF image super-resolution method to address this tradeoff; thus, obtaining a high spatial resolution XRF scan with high SNR. We fuse a low-resolution XRF image and a conventional RGB high-resolution image into a product of both high spatial and high spectral resolution XRF image. There is no guarantee of a one to one mapping between XRF spectrum and RGB color since, for instance, paintings with hidden layers cannot be detected in visible but can in X-ray wavelengths. We separate the XRF image into the visible and nonvisible components. The spatial resolution of the visible component is increased utilizing the high-resolution RGB image, whereas the spatial resolution of the non-visible component is increased using a total variation super-resolution method. Finally, the visible and nonvisible components are combined to obtain the final result.
Tracking-optimized quantization for H.264 compression in transportation video surveillance applications

We propose a tracking-aware system that removes video components of low tracking interest and optimizes the quantization during compression of frequency coefficients, particularly those that most influence trackers, significantly reducing bitrate while maintaining comparable tracking accuracy. We utilize tracking accuracy as our compression criterion in lieu of mean squared error metrics. The process of optimizing quantization tables suitable for automated tracking can be executed online or offline. The online implementation initializes the encoding procedure for a specific scene, but introduces delay. On the other hand, the offline procedure produces globally optimum quantization tables where the optimization occurs for a collection of video sequences. Our proposed system is designed with low processing power and memory requirements in mind, and as such can be deployed on remote nodes. Using H.264/AVC video coding and a commonly used state-of-the-art tracker we show that while maintaining comparable tracking accuracy our system allows for over 50% bitrate savings on top of existing savings from previous work.
A rate-distortion optimal video pre-processing algorithm

Pre-processing algorithms improve the quality of a compression system by removing unimportant data before encoding. This enhances both the visual quality and coding efficiency of the system. We cast the pre-processing problem in the operational rate-distortion framework. Filtering the displaced frame difference is the focus, and the proposed method couples the choice of the quantization scale to the response of the prefilter. Coding errors are then addressed by penalizing significant differences between coded blocks. Finally, experimental results illustrate the efficacy of the method within the context of an MPEG-2 coding scenario.
Sequential construction of 3D-based scene description

A technique for constructing a sequential 3D-based scene description is proposed in this paper. Multiple small baseline 3D scene descriptions from a single moving camera or an array of cameras are used to sequentially construct a large baseline 3D scene description while maintaining the field of view of a small baseline system. A Bayesian framework using a disparity-space image (DSI) technique for disparity estimation is presented. The cost function for large baseline image matching is designed based not only on the photometric matching error, the smoothness constraint, and the ordering constraint, but also on the previous disparity estimates from smaller baseline stereo image pairs as a prior model. Texture information is registered along the scan path of the camera(s). Experimental results demonstrate the effectiveness of this technique in visual communication applications.
Bayesian K-SVD Using Fast Variational Inference

Recent work in signal processing in general and image processing in particular deals with sparse representation related problems. Two such problems are of paramount importance: an overriding need for designing a well-suited overcomplete dictionary containing a redundant set of atoms—i.e., basis signals—and how to find a sparse representation of a given signal with respect to the chosen dictionary. Dictionary learning techniques, among which we find the popular K-singular value decomposition algorithm, tackle these problems by adapting a dictionary to a set of training data. A common drawback of such techniques is the need for parameter-tuning. In order to overcome this limitation, we propose a fully-automated Bayesian method that considers the uncertainty of the estimates and produces a sparse representation of the data without prior information on the number of non-zeros in each representation vector. We follow a Bayesian approach that uses a three-tiered hierarchical prior to enforce sparsity on the representations and develop an efficient variational inference framework that reduces computational complexity. Furthermore, we describe a greedy approach that speeds up the whole process. Finally, we present experimental results that show superior performance on two different applications with real images: denoising and inpainting.
Block based video alignment with linear time and space complexity

Video retrieval and video copy detection are well studied problems. The goal is to find the matching video in a database from a given query video. Typically, these query videos are short and aligning the query video is of secondary importance. Short sequences can be aligned using dynamic time warping. But, since time and memory usage increases quadratically with the length of the sequences, such process is not suitable for the alignment of two full length movies. A typical feature film is between 70 and 210 minutes long. Our goal is to find an accurate frame-by-frame alignment of a full length original film and a copy that has inserted and deleted sequences (e.g., commercial breaks or censorship), as well as differences in quality, format and framerate. We propose a fast, robust and memory efficient video sequence alignment algorithm which has linear space and time complexity.
Robust line detection using a weighted MSE estimator

In this paper we introduce a novel line detection algorithm based on a weighted minimum mean square error (MSE) formulation. This algorithm has been developed to enable an autonomous robot to follow a white line drawn on the floor, but is general in nature and widely applicable to line detection problems. Traditional approaches to line detections consist of two stages, an edge detection stage and a line detection stage using the edge detection result. There are several problems with this approach. First, the initial edge detection stage is sensitive to noise. Second, the second stage does not use all the information available in the image and there fore incorrect decisions made by the first stage cannot be corrected in the second stage. The proposed algorithm achieves its robustness by operating in one step, using all pixels of the image (correctly weighted) and not using any thresholds. The detected line is the solution of a weighted MSE problem. The following three questions are answered in the paper: (I) what mathematical model should be used for the line? (II) how should the weighted MSE problem be set up so that the optimal solution results in the parameters of the line model? And (III), how should the pixels in the image be weighted such that a weighted MSE optimal solution results in a robust line detection? Experimental results demonstrate the performance of the algorithm in noiseless and noisy conditions.
A game theoretic approach to video streaming over peer-to-peer networks

We address the problem of content-aware, foresighted resource reciprocation for media streaming over peer-to-peer (P2P) networks. The envisioned P2P network consists of autonomous and self-interested peers trying to maximize their individual utilities. The resource reciprocation among such peers is modeled as a stochastic game and peers determine the optimal strategies for resource reciprocation using a Markov Decision Process (MDP) framework. Unlike existing solutions, this framework takes the content and the characteristics of the video signal into account by introducing an artificial currency in order to maximize the video quality in the entire network.
Multiscale adaptive image restoration in the wavelet domain

This paper introduces a method for spatially adaptive image restoration based on the detail coefficients of the wavelet transform of a noisy blurred image. A multiscale recursive smoothing filter is applied to the wavelet coefficients ordered onto quadtree structures along different orientations. These coefficients are first prefiltered by a constrained least squares filter in order to remove the spatial correlations due to the blur. An optimal way of choosing the regularization parameters used for this prefiltering operation is introduced here. This is based on an analysis of the detection operations performed by the multiscale filter in order to model edge and non-edge regions of the image differently. Results show that this approach offers a highly adaptive means of preserving edges in a restored image.<<ETX>>
Application-Centric Routing for Video Streaming over Multi-hop Wireless Networks

Routing for video transmissions over multi-hop wireless networks has gained increasing research interest in recent years. However, most existing works only focus on how to satisfy the network-oriented QoS, such as, throughput, delay, and packet loss rate rather than the user perceived quality. Although there are some research efforts which use application-centric video quality as the routing metric, the calculation of video quality is based on some predefined rate-distortion function or model without exploring the impact of video coding and decoding (including error concealment) on network path selection and the resulting received video quality. Moreover, unlike network-centric routing metrics, such as, hop count, average delay or average success probability of packet transmission, video distortion cannot be calculated either additively or multiplicatively in a hop-by-hop fashion due to the dependency among packets introduced by error concealment. As a result, most existing works use either exhaustive search or heuristic methods to find the optimal path, which leads to high computational complexity or suboptimal solutions to the routing problem of video transmission. In this paper, we propose an application-centric routing framework for real-time video transmission over multi-hop wireless networks, where expected video distortion is used as the routing metric. The major contributions of this work are: 1) the development of an efficient routing algorithm with the routing metric in terms of the expected video distortion being calculated on-the-fly, and 2) the development of a quality-driven cross-layer optimization framework to enhance the flexibility and robustness of routing by the joint optimization of routing path selection and video coding, thereby maximizing the user perceived video quality under a given video playback delay constraint. Both theoretical and experimental results demonstrate that the proposed quality-driven application-centric routing approach can achieve a superior performance over existing network-centric routing approaches.
Minmax optimal shape coding using skeleton decomposition

In this paper, we consider the rate-distortion optimal encoding of shape information using a skeleton decomposition and the minimum maximum (minmax) distortion criterion. For bit budget constrained video communication applications, whose goal is to achieve as low as possible but almost constant distortion, the minmax criterion is the natural choice. We propose a 4D DAG (directed acyclic graph) shortest path algorithm implemented by dynamic programming to solve the minimum rate problem, and also provide a solution for the dual minimum distortion problem. Experimental results indicate that our algorithm has an outstanding performance compared with existing methods.
Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images

In this paper a solution is provided to the problem of obtaining a high resolution image from several low resolution images that have been subsampled and displaced by different amounts of sub-pixel shifts. In its most general form, this problem can be broken up into three sub-problems: registration, restoration, and interpolation. Previous work has either solved all three sub-problems independently, or more recently, solved either the first two steps (registration and restoration) or the last two steps together. However, none of the existing methods solve all three sub-problems simultaneously. This paper poses the low resolution to high resolution problem as a maximum likelihood (ML) problem which is solved by the expectation-maximization (EM) algorithm. By exploiting the structure of the matrices involved, the problem ran be solved in the discrete frequency domain. The ML problem is then the estimation of the sub-pixel shifts, the noise variances of each image, the power spectra of the high resolution image, and the high resolution image itself. Experimental results are shown which demonstrate the effectiveness of this approach.
Greedy Bayesian double sparsity dictionary learning

This work presents a greedy Bayesian dictionary learning (DL) algorithm where not only the signals but also the dictionary representation matrix accept a sparse representation. This double-sparsity (DS) model has been shown to be superior to the standard sparse approach in some image processing tasks, where sparsity is only imposed on the signal coefficients. We present a new Bayesian approach which addresses typical shortcomings of regularization-based DS algorithms: the prior knowledge of the true noise level and the need of parameter tuning. Our model estimates the noise and sparsity levels as well as the model parameters from the observations and frequently outperforms state-of-the-art dictionary based techniques by taking into account the uncertainty of the estimates. Additionally, we introduce a versatile notation which generalizes denoising, inpainting and compressive sensing problem formulations. Finally, theoretical results are validated with denoising experiments on a set of images.
A hidden Markov model based visual speech synthesizer

This paper describes a hidden Markov model (HMM) based visual synthesizer designed to assist persons with impaired hearing. This synthesizer builds on results in the area of audio-visual speech recognition. We describe how a correlation HMM can be used to integrate independent acoustic and visual HMMs for speech-to-visual synthesis. Our results show that an HMM correlating model can significantly improve synchronization errors versus techniques which compensate for rate differences through scaling.
Frame rate and viseme analysis for multimedia applications

In the future, multimedia technology will be able to provide video frame rates equal to or better than 30 frames per second (FPS). Until that time the hearing impaired community will be using band limited communication systems over unshielded twisted pair copper wiring. As a result, multimedia communication systems will use a coder/decoder (CODEC) to compress the video and audio signals for transmission. For these systems to be usable by the hearing impaired community, the algorithms within the CODEC have to be designed to account for the perceptual boundaries of the hearing impaired. We investigate the perceptual boundaries of speech reading and multimedia technology, which are the constraints that effect speech reading performance. We analyze and draw conclusions on the relationship between viseme groupings, accuracy of viseme recognition, and presentation rate. These results are critical in the design of multimedia systems for the hearing impaired.
An iterative weighted regularized algorithm for improving the resolution of video sequences

This paper introduces an iterative regularized approach to increase the resolution of a video sequence. A multiple input smoothing convex functional is defined and used to obtain a globally optimal high resolution video sequence. A mathematical model of multiple inputs is described by using the point spread function between the original and bilinearly interpolated images in the spatial domain, and motion estimation between frames in the temporal domain. An iterative algorithm is utilized for obtaining the solution. The regularization parameter is updated at each iteration step from the partially restored video sequence. Experimental results demonstrate the capability of the proposed approach.
A quality-driven decision engine for live video transmission under service-oriented architecture

Service-oriented architecture provides a solution to the increasing network complexity due to ever-growing heterogeneous networks. As the most significant component of SOA, the decision engine is to create a workflow, defined as a sequence of individual data processing entities, for providing end-to-end QoS of a given task. Although the workflow of video transmission is generally known, existing solutions are often monolithic. Furthermore, there is no decision engine to select a workflow based on the best user-perceived quality. In this article we propose a service-oriented decision engine framework, which consists of a decision engine, a performance evaluation component, and other major SOA components to support real-time video transmission over wireless multihop networks, aiming to provide the best user-perceived video quality under application-centric QoS constraints. Based on the investigation of the state of- the-art research efforts on SOA, some key issues for wireless live video transmission are discussed, and a case study for live video transmission is given to illustrate the proposed scheme. The superior performance of the proposed service-oriented decision engine is validated by experimental results.
Parallel processing architectures for iterative image restoration

Mesh and mesh-of-pyramids implementations of iterative image restoration algorithms are proposed. These implementations are based on a single-step algorithm as well as on a multistep iterative algorithm derived from the single step regularized iterative restoration algorithm. One processor is assigned to each picture element, with local memory depending on the support of the restoration filter. The implementations consist of interprocessor communication and intraprocessor computations. The efficiency of the proposed VLSI algorithms is judged by establishing lower bounds on AT/sup 2/, where A is the area of the VLSI and T is its computation time.<<ETX>>
Video completion using block matching for video stabilization

This paper presents a video completion algorithm using block matching for video stabilization. In order to fill in missing pixels, the proposed algorithm consists of three steps: i) mosaicking for covering the missing static, planar regions, ii) estimation of local motion vectors using the hierarchical Lucas-Kanade optical flow method, and iii) selection of the most similar patch in both spatial and temporal neighbors. The proposed video completion algorithm can be applied in the wide areas of consumer electronics including camcorders, smart phone cameras, tablet cameras, and smart glasses.