nils-herrmann/clean_scirepeval huggingface dataset:
A recursive shape error concealment algorithm

The encoding of shape information is a distinguishing feature of MPEG-4. In error prone communication networks, it is important and efficient to conceal shape errors spatially, so as to avoid propagation of errors in the video frames. The proposed system first defines the missing area's four rectangular neighbors. It then detects in the neighbors the lines that intersect the borders and redefines them in a geometric coordinate system. These lines are paired under some optimization objective so that they connect smoothly in the missing area. Each pair is extrapolated and corrected recursively until a close curve is formed. Test results on various sequences under various error rates are presented and compared with other shape concealment methods.
Joint Source-Channel Video Transmission

This book deals with the problem of joint source-channel video transmission, i.e., the joint optimal allocation of resources at the application layer and the other network layers, such as data rate adaptation, channel coding, power adaptation in wireless networks, quality of service (QoS) support from the network, and packet scheduling, for efficient video transmission. Real-time video communication applications, such as videoconferencing, video telephony, and on-demand video streaming, have gained increased popularity. However, a key problem in video transmission over the existing Internet and wireless networks is the incompatibility between the nature of the network conditions and the QoS requirements (in terms, for example, of bandwidth, delay, and packet loss) of real-time video applications. To deal with this incompatibility, a natural approach is to adapt the end-system to the network. The joint source-channel coding approach aims to efficiently perform content-aware cross-layer resource allocation, thus increasing the communication efficiency of multiple network layers. Our purpose in this book is to review the basic elements of the state-of-the-art approaches toward joint source-channel video transmission for wired and wireless systems. In this book, we present a general resource-distortion optimization framework, which is used throughout the book to guide our discussions on various techniques of joint source-channel video transmission. In this framework, network resources from multiple layers are assigned to each video packet according to its level of importance. It provides not only an optimization benchmark against which the performance of other sub-optimal systems can be evaluated, but also a useful tool for assessing the effectiveness of different error control components in practical system design. This book is therefore written to be accessible to researchers, expert industrial R&D engineers, and university students who are interested in the cut ing edge technologies in joint source-channel video transmission.  Elements of a Video Communication System / Joint Source-Channel Coding / Error-Resilient Video Coding / Channel Modeling and Channel Coding / Internet Video Transmission / Wireless Video
Comparison of low- and high-level visual features for audio-visual continuous automatic speech recognition

We compare two different groups of visual features that can be used in addition to audio to improve automatic speech recognition (ASR), high- and low-level visual features. Facial animation parameters (FAPs), supported by the MPEG-4 standard for the visual representation of speech, are used as high-level visual features. Principal component analysis (PCA) based projection weights of the intensity images of the mouth area are used as low-level visual features. PCA is also applied on the FAPs. We develop an audio-visual ASR (AV-ASR) system and compare its performance for two different visual feature groups, following two approaches. The first approach assumes the same dimensionality for both high- and low-level visual features, while, in the second approach, the percentage of statistical variance described by the visual features used is the same. Multi-stream hidden Markov models (HMMs) and a late integration approach are used to integrate audio and visual information and perform continuous AV-ASR experiments. Experiments were performed at various SNRs (0-30 dB) with additive white Gaussian noise on a relatively large vocabulary database (approximately 1000 words). Conclusions are drawn on the trade off between the dimensionality of the visual features and the amount of speechreading information contained in them and its influence on the AV-ASR performance.
Rate-distortion optimized product code forward error correction for video transmission over IP-based wireless networks

The problem of encoding and transmitting a video sequence over an IP-based wireless network, consisting of both wired and wireless links, is addressed. To combat the different types of packet loss in the heterogeneous network, the use of a product code forward error correction (FEC) scheme capable of providing unequal error protection is considered. At the transport layer, Reed-Solomon (RS) coding is used to provide inter-packet protection. In addition, rate-compatible punctured convolutional (RCPC) coding is used at the link layer to provide unequal intra-packet protection. Optimal bit allocation is performed in a rate-distortion optimized joint source-channel coding and power allocation framework to achieve the best video quality. Simulation results illustrate the advantage of the proposed product code FEC scheme over previously studied approaches.
An operational rate-distortion optimal single-pass SNR scalable video coder

In this paper, we introduce a new methodology for signal-to-noise ratio (SNR) video scalability based on the partitioning of the DCT coefficients. The DCT coefficients of the displaced frame difference (DFD) for inter-blocks or the intensity for intra-blocks are partitioned into a base layer and one or more enhancement layers, thus, producing an embedded bitstream. Subsets of this bitstream can be transmitted with increasing video quality as measured by the SNR. Given a bit budget for the base and enhancement layers the partitioning of the DCT coefficients is done in a way that is optimal in the operational rate-distortion sense. The optimization is performed using Lagrangian relaxation and dynamic programming (DP). Experimental results are presented and conclusions are drawn.
Shape-from-Shifting: Uncalibrated Photometric Stereo with a Mobile Device

Surface shape scanning techniques, such as laser scanning and photometric stereo, are widespread analytical tools used in the field of cultural heritage. Compared to regular 2D RGB photos, 3D surface scans provide higher fidelity of an object's surface shape which assist conservators, art historians, and archaeologists in understanding how these artworks and artifacts are made and to digitally document them for purposes of conservation. However, current state-of-the-art 3D surface scanning tools used in art conservation are often expensive and bulky-such as light dome structures that are often over 1 m in diameter. In this paper, we introduce mobile shape-from-shifting (SfS): a simple, low-cost and streamlined photometric stereo framework for scanning planar surfaces with a consumer mobile device coupled to a low-cost add-on component. Our free-form mobile SfS framework relaxes the rigorous hardware and other complex requirements inherent to conventional 3D scanning tools. This is achieved by taking a sequence of photos with the on-board camera and flash of a mobile device. The sequence of captures are used to reconstruct high quality normal maps using nearlight photometric stereo algorithms, which are of comparable quality to conventional photometric stereo. We demonstrate 3D surface reconstructions with SfS on different materials and scales. Moreover, the mobile SfS technique can be used "in the wild" so that 3D scans may be performed in their natural environment, eliminating the need for transport to a laboratory setting. With the elegant design and low cost, we believe our Mobile SfS can greatly benefit the conservation community by providing a userfriendly and cost-effective solution for 3D surface scanning.
Motion compensated shape error concealment [MPEG-4]

The introduction of video objects (VOs) is one of the innovations of MPEG-4. The /spl alpha/-plane of a VO defines its shape at a given instance in time and hence determines the boundary of its texture. In packet-based networks, shape, motion, and texture are subject to loss. In this paper, we propose a post-processing shape error concealment technique that uses the motion compensated boundary information of the previously received /spl alpha/-plane. The proposed approach is based on matching received boundary segments in the current frame to the boundary in the previous frame. This matching is achieved by finding a maximally smooth motion vector field. After the current boundary segments are matched to the previous boundary, the missing boundary pieces are reconstructed by motion compensation. Experimental results demonstrating the performance of the proposed motion compensated shape error concealment method, and comparisons with our previously proposed spatial Hermite spline method are presented.
Image sequence filtering in quantum-limited noise with applications to low-dose fluoroscopy

Clinical angiography requires hundreds of X-ray images, putting the patients and particularly the medical staff at risk. Dosage reduction involves an inevitable sacrifice in image quality. In this work, the latter problem is addressed by first modeling the signal-dependent, Poisson-distributed noise that arises as a result of this dosage reduction. The commonly utilized noise model for single images is shown to be obtainable from the new model. Stochastic temporal filtering techniques are proposed to enhance clinical fluoroscopy sequences corrupted by quantum mottle. The temporal versions of these filters as developed here are more suitable for filtering image sequences, as correlations along the time axis can be utilized. For these dynamic sequences, the problem of displacement field estimation is treated in conjunction with the filtering stage to ensure that the temporal correlations are taken along the direction of motion to prevent object blur.
Two-dimensional channel coding for scalable H.264/AVC video

Efficient bit stream adaptation and resilience to packet losses are two critical requirements in scalable video coding for transmission over packet-lossy networks. These requirements have a greater significance in scalable H.264/AVC video bit streams since missing refinement information in a layer propagates to all lower layers in the prediction hierarchy and causes substantial degradation in video quality. This work proposes an algorithm to accurately estimate the overall distortion of the reconstructed frames due to enhancement layer truncation, drift/error propagation, and error concealment in the scalable H.264/AVC video. This ensures low computational cost since it bypasses highly complex pixel-level motion compensation operations. Simulation results show an accurate distortion estimation at various channel loss rates. The estimate is further integrated into a cross-layer optimization framework for optimized bit extraction and content-aware channel rate allocation. Experimental results demonstrate that precise distortion estimation enables our proposed transmission system to achieve a significantly higher average video PSNR compared to a conventional content independent system.
Compressive reconstruction for 3D incoherent holographic microscopy

Incoherent holography has recently attracted significant research interest due to its flexibility for a wide variety of light sources. In this paper, we use compressive sensing to reconstruct a three-dimensional volumetric object from its two-dimensional Fresnel incoherent correlation hologram. We show how compressed sensing enables reconstruction without out-of-focus artifacts, when compared to conventional back-propagation recovery. Finally, we analyze the reconstruction guarantees of the proposed approach both numerically and theoretically and compare that with coherent holography.
Dictionary-based multiple frame video super-resolution

In this paper, we propose a multiple-frame super-resolution (SR) algorithm based on dictionary learning and motion estimation. We adopt the use of multiple bilevel dictionaries which have also been used for single-frame SR. Multiple frames compensated through sub-pixel motion are considered. By simultaneously solving for a batch of patches from multiple frames, the proposed multiple-frame SR algorithm improves over single frame SR. We also propose a novel dictionary learning algorithm based on which dictionaries are trained from consecutive video frames, rather than still images or individual video frames, which further improves the performance of the developed video SR algorithm. Extensive experimental comparisons with state-of-the-art SR algorithms verifies the effectiveness of our proposed multiple-frame SR approach.
A rate-distortion optimal scalable vertex based shape coding algorithm

We present a rate-distortion (RD) optimized scalable vertex-based shape coding algorithm. Following the base layer, each successive enhancement layer refines a given shape approximation by optimally (within a layer) placing new vertices and perturbing existing vertices. An efficient low entropy distortion adaptive vertex coding strategy is employed to take advantage of information available from coarser layers. Based on the chosen vertex rate and distortion definitions, a resulting enhancement layer topology is solved by executing a directed acyclic graph (DAG) shortest path algorithm. Finally, an iterative VLC optimization scheme is employed to find both the optimized scalable code and the most efficient set of parameter VLC tables.
DNA Microarray Image Intensity Extraction using Eigenspots

DNA microarrays are commonly used in the rapid analysis of gene expression in organisms. Image analysis is used to measure the average intensity of circular image areas (spots), which correspond to the level of expression of the genes. A crucial aspect of image analysis is the estimation of the background noise. Currently, background subtraction algorithms are used to estimate the local background noise and subtract it from the signal. In this paper we use principal component analysis (PCA) to de-correlate the signal from the noise, by projecting each spot on the space of eigenvectors, which we term eigenspots. PCA is well suited for such application due to the structural nature of the images. To compare the proposed method with other background estimation methods we use the industry standard signal-to-noise metric xdev.
A general sparse image prior combination in super-resolution

In this paper the Super-Resolution (SR) image registration and reconstruction problem is studied within the Bayesian framework using a general sparse image prior combination. The representation of the proposed priors as Scale Mixtures of Gaussians (SMG), leads to the introduction of variational parameters, for which degenerate distributions are assumed. In the proposed method all the problem unknowns are automatically estimated using variational techniques. An experimental comparison between the proposed and state of the art methods has been performed, on both synthetic and real images.
Optimal source coding and transmission power management using a min-max expected distortion approach

We consider the problem of compressing a video sequence for transmission over a wireless channel. In our approach we jointly consider error resilience and concealment techniques, at the source coding level, and transmission power management at the physical layer. We formulate a minimum-maximum distortion problem, where our goal is to either (i) minimize the total transmission energy for a given maximum expected distortion, or (ii) minimize the maximum expected distortion at the receiver for a given maximum transmission energy. Experimental results show that simultaneously adjusting the source coding and transmission power is more energy efficient than considering these factors separately.
Joint source coding and data rate adaptation for energy efficient wireless video streaming

Rapid growth in wireless networks is fueling demand for video services from mobile users. While the problem of transmitting video over unreliable channels has received some attention, the wireless network environment poses challenges such as transmission power management that have received little attention previously in connection with video. Transmission power management affects battery life in mobile devices, interference to other users, and network capacity. We consider energy efficient transmission of a video sequence under delay and quality constraints. The selection of source coding parameters is considered jointly with transmitter power and rate adaptation, and packet transmission scheduling. The goal is to transmit a video frame using the minimal required transmission energy under delay and quality constraints. Experimental results are presented that illustrate the advantages of the proposed approach.
A multicamera setup for generating stereo panoramic video

Traditional visual communication systems convey only two-dimensional (2-D) fixed field-of-view (FOV) video information. The viewer is presented with a series of flat, nonstereoscopic images, which fail to provide a realistic sense of depth. Furthermore, traditional video is restricted to only a small part of the scene, based on the director's discretion and the user is not allowed to "look around" in an environment. The objective of this work is to address both of these issues and develop new techniques for creating stereo panoramic video sequences. A stereo panoramic video sequence should be able to provide the viewer with stereo vision at any direction (complete 360-degree FOV) at video rates. In this paper, we propose a new technique for creating stereo panoramic video using a multicamera approach, thus creating a high-resolution output. We present a setup that is an extension of a previously known approach, developed for the generation of still stereo panoramas, and demonstrate that it is capable of creating high-resolution stereo panoramic video sequences. We further explore the limitations involved in a practical implementation of the setup, namely the limited number of cameras and the nonzero physical size of real cameras. The relevant tradeoffs are identified and studied.
A model-based pel-recursive motion estimation algorithm

A model-based approach to pel-recursive motion estimation is presented. The derivation of the algorithm is similar to the Wiener-based pel-recursive motion-estimation algorithm. However, the proposed algorithm utilizes the spatio-temporal correlations in an image sequence by considering an autoregressive (AR) model for the motion-compensated frames. Therefore, depending on the support of the AR model, the estimation is based on two or more consecutive frames. Pel-recursive motion-estimation algorithms which appear in literature are special cases of this algorithm. Various implementation issues such as adaptive regularization and modeling of the motion field are considered. Based on experiments with typical videoconferencing scenes, the conclusion is that the proposed algorithm performs better than the two-frame Wiener-based pel-recursive algorithm with respect to accuracy, robustness, and smoothness of the velocity field.<<ETX>>
Image super-resolution from compressed sensing observations

In this work we propose a novel framework to obtain High Resolution (HR) images from Compressed Sensing (CS) imaging systems capturing multiple Low Resolution (LR) images of the same scene. The proposed CS Super Resolution (SR) approach combines existing CS reconstruction algorithms with an LR to HR approach based on the use of a Super Gaussian (SG) regularization term. The reconstruction is formulated as a constrained optimization problem which is solved using the Alternate Direction Methods of Multipliers (ADMM). The image estimation subproblem is solved using Majorization-Minimization (MM) while the CS reconstruction becomes an l1-minimization subject to a quadratic constraint. The performed experiments show that the proposed method compares favorably to classical SR methods at compression ratio 1, obtaining excellent SR reconstructions at ratios below one.
Deep multi-view models for glitch classification

Non-cosmic, non-Gaussian disturbances known as “glitches”, show up in gravitational-wave data of the Advanced Laser Interferometer Gravitational-wave Observatory, or aLIGO. In this paper, we propose a deep multi-view convolutional neural network to classify glitches automatically. The primary purpose of classifying glitches is to understand their characteristics and origin, which facilitates their removal from the data or from the detector entirely. We visualize glitches as spectrograms and leverage the state-of-the-art image classification techniques in our model. The suggested classifier is a multi-view deep neural network that exploits four different views for classification. The experimental results demonstrate that the proposed model improves the overall accuracy of the classification compared to traditional single view algorithms.
A VQ-based blur identification algorithm

The estimation of the point spread function (PSF) of the degradation system is often a necessary first step in the restoration of blurred images. A novel vector quantization (VQ)-based blur identification algorithm is presented. A number of codebooks are designed corresponding to various versions of the blurring function. Prototype images blurred by each candidate blur are used. Only the non-flat regions for specific frequency bands are represented by the entries in the codebooks. Given a noisy and blurred image, one of the codebooks is chosen based on a similarity measure, therefore providing the identification of the blur. Simulations are performed for various blurring functions and noise levels. The results demonstrate the effectiveness of the proposed algorithms.
Joint source-channel coding for scalable video over DS-CDMA multipath fading channels

We extend our previous work on joint source-channel coding to scalable video transmission over wireless direct-sequence code-division-multiple-access (DS-CDMA) multipath fading channels. A SNR scalable video coder is used and unequal error protection (UEP) is allowed for each scalable layer. At the receiver-end an adaptive antenna array auxiliary-vector (AV) filter is utilized that provides space-time RAKE-type processing and multiple-access interference suppression. The choice of the AV receiver is dictated by realistic channel fading rates that limit the data record available for receiver adaptation and redesign. Our problem is to allocate the available bit rate of the user of interest between source and channel coding and across scalable layers, while minimizing the end-to-end distortion of the received video sequence. The optimization algorithm that we propose utilizes universal rate-distortion characteristic curves that show the contribution of each layer to the total distortion as a function of the source rate of the layer and the residual bit error rate (the error rate after channel coding). These plots can be approximated using appropriate functions to reduce the computational complexity of the solution.