nils-herrmann/clean_scirepeval huggingface dataset:
Surface shape studies of the art of Paul Gauguin

Starting in the 1890s the artist Paul Gauguin (1848-1903) created a series of prints and transfer drawings using techniques that are not entirely understood. To better understand the artist's production methods, photometric stereo was used to assess the surface shape of a number of these graphic works that are now in the collection of the Art Institute of Chicago. Photometric stereo uses multiple images of Gauguin's graphic works captured from a fixed camera position, lit from multiple specific angles to create an interactive composite image that reveals textural characteristics. These active images reveal details of sequential media application upon experimental printing matrices that help resolve longstanding art historical questions about the evolution of Gauguin's printing techniques. Our study promotes the use of photometric stereo to capitalize on the increasing popularity of Reflectance Transformation Imaging (RTI) among conservators in the world's leading museums.
Speech-to-video synthesis using MPEG-4 compliant visual features

There is a strong correlation between the building blocks of speech (phonemes) and the building blocks of visual speech (visimes). In this paper, this correlation is exploited and an approach is proposed for synthesizing the visual representation of speech from a narrow-band acoustic speech signal. The visual speech is represented in terms of the facial animation parameters (FAPs), supported by the MPEG-4 standard. The main contribution of this paper is the development of a correlation hidden Markov model (CHMM) system, which integrates independently trained acoustic HMM (AHMM) and visual HMM (VHMM) systems, in order to realize speech-to-video synthesis. The proposed CHMM system allows for different model topologies for acoustic and visual HMMs. It performs late integration and reduces the amount of required training data compared to early integration modeling techniques. Temporal accuracy experiments, comparison of the synthesized FAPs to the original FAPs, and audio-visual automatic speech recognition (AV-ASR) experiments utilizing the synthesized visual speech were performed in order to objectively measure the performance of the system. The objective experiments demonstrated that the proposed approach reduces time alignment errors by 40.5% compared to the conventional temporal scaling method, that the synthesized FAP sequences are very similar to the original FAP sequences, and that synthesized FAP sequences contain visual speechreading information that can improve AV-ASR performance.
A hybrid source-channel coding scheme for object-based wireless video communications

In this paper, we study the joint source-channel coding of object-based video, and propose a data hiding scheme that improves the video error resilience by adaptively embedding the shape and motion information in the texture data. Within a rate-distortion theoretical framework, the source coding, channel coding, data embedding, and decoder error concealment are jointly optimized based on the knowledge of transmission channel conditions. The problem is solved using Lagrangian relaxation and dynamic programming. Experimental results indicate that the proposed hybrid source-channel coding scheme significantly outperforms methods without data hiding or unequal error protection
A novel cumulative distortion metric and a no-reference sparse prediction model for packet prioritization in encoded video transmission

In this paper we propose a new quality metric to estimate the impact of packet loss on the perceptual quality of encoded video sequences transmitted over error-prone networks. The proposed metric, henceforth referred to as Cumulative Distortion using Structural Similarity (CDSSIM), quantifies the overall structural distortion resulting from bidirectional error propagation in predictively coded, motion compensated videos. Furthermore, we present a No-Reference (NR) sparse regression model to predict the proposed CDSSIM metric using pre-defined features associated with slice loss. The Least Absolute Shrinkage and Selection Operator (LASSO) method is applied for two resolution formats with features extracted solely from the encoded bit-stream. Standardized statistical performance measures show that the model can predict the cumulative distortion to a high degree of accuracy. We further evaluate the results using a Quartile-Based Prioritization (QBP) scheme and demonstrate that the predicted data provides an effective way to prioritize packets for video streaming applications.
Hierarchical Bayesian image restoration from partially known blurs

In this paper, we examine the restoration problem when the point-spread function (PSF) of the degradation system is partially known. For this problem, the PSF is assumed to be the sum of a known deterministic and an unknown random component. This problem has been examined before; however, in most previous works the problem of estimating the parameters that define the restoration filters was not addressed. In this paper, two iterative algorithms that simultaneously restore the image and estimate the parameters of the restoration filter are proposed using evidence analysis (EA) within the hierarchical Bayesian framework. We show that the restoration step of the first of these algorithms is in effect almost identical to the regularized constrained total least-squares (RCTLS) filter, while the restoration step of the second is identical to the linear minimum mean square-error (LMMSE) filter for this problem. Therefore, in this paper we provide a solution to the parameter estimation problem of the RCTLS filter. We further provide an alternative approach to the expectation-maximization (EM) framework to derive a parameter estimation algorithm for the LMMSE filter. These iterative algorithms are derived in the discrete Fourier transform (DFT) domain; therefore, they are computationally efficient even for large images. Numerical experiments are presented that test and compare the proposed algorithms.
An optimal segmentation encoding scheme in the rate distortion sense

In this paper, we present two fast and efficient methods for the lossy encoding of object boundaries which are given as 8-connect chain codes. We approximate the boundary by a polygon and consider the problem of finding the polygon which leads to the smallest distortion for a given number of bits. We introduce two different approaches to find the best polygonal approximation when an additive distortion measure is employed. The first approach is based on Lagrangian relaxation and a shortest path algorithm. This scheme results in solutions which belong to the convex hull of the operational rate-distortion curve. The second algorithm is based on a tree pruning scheme and finds all optimal solutions. We present results of the proposed algorithms using an object from the "Miss America" sequence.
Preconditioning for Underdetermined Linear Systems with Sparse Solutions

Performance guarantees for the algorithms deployed to solve underdetermined linear systems with sparse solutions are based on the assumption that the involved system matrix has the form of an incoherent unit norm tight frame. Learned dictionaries, which are popular in sparse representations, often do not meet the necessary conditions for signal recovery. In compressed sensing (CS), recovery rates have been improved substantially with optimized projections; however, these techniques do not produce binary matrices, which are more suitable for hardware implementation. In this paper, we consider an underdetermined linear system with sparse solutions and propose a preconditioning technique that yields a system matrix having the properties of an incoherent unit norm tight frame. While existing work in preconditioning concerns greedy algorithms, the proposed technique is based on recent theoretical results for standard numerical solvers such as BP and OMP. Our simulations show that the proposed preconditioning improves the recovery rates both in sparse representations and CS; the results for CS are comparable to optimized projections.
Hybrid image segmentation using watersheds and fast region merging

A hybrid multidimensional image segmentation algorithm is proposed, which combines edge and region-based techniques through the morphological algorithm of watersheds. An edge-preserving statistical noise reduction approach is used as a preprocessing stage in order to compute an accurate estimate of the image gradient. Then, an initial partitioning of the image into primitive regions is produced by applying the watershed transform on the image gradient magnitude. This initial segmentation is the input to a computationally efficient hierarchical (bottom-up) region merging process that produces the final segmentation. The latter process uses the region adjacency graph (RAG) representation of the image regions. At each step, the most similar pair of regions is determined (minimum cost RAG edge), the regions are merged and the RAG is updated. Traditionally, the above is implemented by storing all RAG edges in a priority queue. We propose a significantly faster algorithm, which additionally maintains the so-called nearest neighbor graph, due to which the priority queue size and processing time are drastically reduced. The final segmentation provides, due to the RAG, one-pixel wide, closed, and accurately localized contours/surfaces. Experimental results obtained with two-dimensional/three-dimensional (2-D/3-D) magnetic resonance images are presented.
Hyperparameter estimation for emission computed tomography data

Although many statistical methods have been proposed for the restoration of tomographic images, their use in medical environments has been limited due to two important factors. These factors are the need for greater computational time than deterministic methods and the selection of the hyperparameters in the image models. Consequently, deterministic methods, like the classical filtered back-projection (FBP) and algebraic reconstruction (AR), are commonly used. In this work, we propose a method to estimate, from observed image data in emission tomography, the hyperparameters in a Generalized Gaussian Markov Random Field (GGMRF). We use the hierarchical Bayesian approach and evidence analysis to reconstruct the image and estimate the unknown hyperparameters. The method is tested on synthetic images.
Joint source-channel coding for motion-compensated DCT-based SNR scalable video

In this paper, we develop an approach toward joint source-channel coding for motion-compensated DCT-based scalable video coding and transmission. A framework for the optimal selection of the source and channel coding rates over all scalable layers is presented such that the overall distortion is minimized. The algorithm utilizes universal rate distortion characteristics which are obtained experimentally and show the sensitivity of the source encoder and decoder to channel errors. The proposed algorithm allocates the available bit rate between scalable layers and, within each layer, between source and channel coding. We present the results of this rate allocation algorithm for video transmission over a wireless channel using the H.263 Version 2 signal-to-noise ratio (SNR) scalable codec for source coding and rate-compatible punctured convolutional (RCPC) codes for channel coding. We discuss the performance of the algorithm with respect to the channel conditions, coding methodologies, layer rates, and number of layers.
Audiovisual Fusion: Challenges and New Approaches

In this paper, we review recent results on audiovisual (AV) fusion. We also discuss some of the challenges and report on approaches to address them. One important issue in AV fusion is how the modalities interact and influence each other. This review will address this question in the context of AV speech processing, and especially speech recognition, where one of the issues is that the modalities both interact but also sometimes appear to desynchronize from each other. An additional issue that sometimes arises is that one of the modalities may be missing at test time, although it is available at training time; for example, it may be possible to collect AV training data while only having access to audio at test time. We will review approaches to address this issue from the area of multiview learning, where the goal is to learn a model or representation for each of the modalities separately while taking advantage of the rich multimodal training data. In addition to multiview learning, we also discuss the recent application of deep learning (DL) toward AV fusion. We finally draw conclusions and offer our assessment of the future in the area of AV fusion.
On the Detection of OFDM Signals in the Presence of Strong Phase Noise

In this paper, the problem of optimum data detection in OFDM under severe phase noise (PHN) is addressed. Since direct maximization of the likelihood function is infeasible, the expectation-maximization (EM) algorithm is invoked as an iterative, feasible method of data detection. Two algorithms are proposed, one derived by direct application of the standard EM algorithm in the present context of OFDM data detection, whereas the other employs soft data estimates resulting in enhanced performance. The use of Kalman filtering is also examined as a means of complexity reduction and handling of the case of a non-negligible frequency offset (FO). The performance of the proposed algorithms is evaluated by simulations which show significant gains over the PHN-compensation method employed by OFDM-based standards.
A video compression scheme with optimal bit allocation among segmentation, motion, and residual error

We present a theory for the optimal bit allocation among quadtree (QT) segmentation, displacement vector field (DVF), and displaced frame difference (DFD). The theory is applicable to variable block size motion-compensated video coders (VBSMCVC), where the variable block sizes are encoded using the QT structure, the DVF is encoded by first-order differential pulse code modulation (DPCM), the DFD is encoded by a block-based scheme, and an additive distortion measure is employed. We derive an optimal scanning path for a QT that is based on a Hilbert curve. We consider the case of a lossless VBSMCVC first, for which we develop the optimal bit allocation algorithm using dynamic programming (DP). We then consider a lossy VBSMCVC, for which we use Lagrangian relaxation, and show how an iterative scheme, which employs the DP-based solution, can be used to find the optimal solution. We finally present a VBSMCVC, which is based on the proposed theory, which employs a DCT-based DFD encoding scheme. We compare the proposed coder with H.263. The results show that it outperforms H.263 significantly in the rate distortion sense, as well as in the subjective sense.
Shape error concealment based on a shape-preserving boundary approximation

In error-prone communications, packet loss results in missing information of shape, motion and texture of a video object (VO). Error concealment refers to the recovery of lost information at the decoder. In this paper, we propose a spatial shape error concealment technique. We consider a geometric representation of the shape of a VO consisting of its boundary, which can be extracted from the received a-plane. Some boundary parts are missing due to errors. We propose a method for modeling the received boundary based on a shape-preserving approximation that uses T-splines. Such an approximation provides a good estimation of the direction of a missing boundary segment, which we use to construct a concealment spline that joins smoothly with the received boundary parts.
Motion field prediction and restoration for low bit-rate video coding

Motion vector field (MVF) prediction methods are presented followed by a restoration method. These methods combined with a proposed motion compensated (MC) video coding scheme are suitable for low bit rate transmission. An expression is derived for the initial estimate of the working MVF based on the preceding MVF. Spatio-temporally adaptive regularization is applied using neighborhood information. The output MVF is used as the initial prediction estimate for a Kalman MVF restoration approach. By applying this method to both the encoder and decoder, the resulting MC MVF and image intensity temporal updates are coded and transmitted. The restoration method produces accurate estimates of the MVF, thus resulting in a significant transmission cost reduction. Experiments with standard video-conference image sequences demonstrate the improved performance of the proposed scheme.
Frequency-domain analysis of discrete wavelet transform coefficients and their adaptive shrinkage for anti-aliasing

We present an antialiasing method using combined wavelet-Fourier transform and spatially adaptive shrinkage of the transform coefficients. Traditional antialiasing methods employ a simple low-pass filter onto the entire image, so the resulting image loses not only aliasing artifacts but also high-frequency components such as edges and ridges. The proposed algorithm analyzes the property of the LL subband of the discrete wavelet transform (DWT), and reduces aliasing artifacts using patch-adaptive shrinkage of the DWT coefficients. More specifically, an antialiased LL subband is obtained using adaptive patch-based aliasing reduction. To detect an aliased region, we subtract the discrete Fourier transform (DFT) coefficients of the LL subband from the DFT coefficients of antialiased LL subband. The detected aliasing artifacts in the LH, HL, and HH subbands are reduced by patch-wise adaptive shrinkage of the transform coefficients. The resulting antialiased image is obtained using the inverse DWT. The aliasing artifacts can be efficiently reduced by adaptively shrinking wavelet transform coefficients for preserving high-frequency image details. The proposed antialiasing algorithm is suitable for removing aliasing artifacts which frequently occur in imaging sensors with limited resolution.
Rate distortion optimal signal compression using second order polynomial approximation

We present a time domain signal compression algorithm based on the coding of line segments which are used to approximate the signal. These segments are fitted in a way that is optimal in the rate distortion sense. The approach is applicable to many types of signals, but in this paper we focus on the compression of electrocardiogram (ECG) signals. As opposed to traditional time-domain algorithms, where heuristics are used to extract representative signal samples from the original signal, an optimization algorithm is formulated for sample selection using graph theory, with linear interpolation applied to the reconstruction of the signal. In this paper the algorithm is generalized by using second order polynomial interpolation for the reconstruction of the signal from the extracted signal samples. The polynomials are fitted in a way that guarantees minimum reconstruction error given an upper bound on the number of bits. The method achieves good performance compared both to the case where linear interpolation is used in reconstruction of the signal and to other state-of-the-art ECG coders.
A general formulation of the weighted smoothing functional for regularized image restoration

Proposes a general form of the weighted smoothing functional for regularized image restoration. The weighting matrices which introduce the spatial adaptivity are defined as a function of the (partially) restored image. As a result no prior knowledge about the image is required but the smoothing functional to be minimized is nonlinear with respect to the unknown image. Conditions for the convexity of the functional are established. An iterative algorithm is proposed for obtaining its minimum. Sufficient conditions for the convergence of the algorithm are established. Various forms of the weighting matrices are proposed. Experimental results demonstrate the effectiveness of the approach.<<ETX>>
An efficient rate-distortion optimal shape coding approach utilizing a skeleton-based decomposition

In this paper, we present a new shape-coding approach, which decouples the shape information into two independent signal data sets; the skeleton and the boundary distance from the skeleton. The major benefit of this approach is that it allows for a more flexible tradeoff between approximation error and bit budget. Curves of arbitrary order can be utilized for approximating both the skeleton and distance signals. For a given bit budget for a video frame, we solve the problem of choosing the number and location of the control points for all skeleton and distance signals of all boundaries within a frame, so that the overall distortion is minimized. An operational rate-distortion (ORD) optimal approach using Lagrangian relaxation and a four-dimensional direct acyclic graph (DAG) shortest path algorithm is developed for solving the problem. To reduce the computational complexity from O(N(5)) to O(N(3)), where N is the number of admissible control points for a skeleton, a suboptimal greedy-trellis search algorithm is proposed and compared with the optimal algorithm. In addition, an even more efficient algorithm with computational complexity O(N(2)) that finds an ORD optimal solution using a relaxed distortion criterion is also proposed and compared with the optimal solution. Experimental results demonstrate that our proposed approaches outperform existing ORD optimal approaches, which do not follow the same decomposition of the source data.
A general framework for frequency domain multi-channel signal processing

The authors provide a general framework for performing processing of stationary multichannel (MC) signals that is linear shift-invariant within channel and shift varying across channels. Emphasis is given to the restoration of degraded signals. It is shown that, by utilizing the special structure of semiblock circulant and block diagonal matrices, MC signal processing can be easily carried out in the frequency domain. The generalization of many frequency-domain single-channel (SC) signal processing techniques to the MC case is presented. It is shown that in MC signal processing each frequency component of a signal and system is presented, respectively, by a small vector and a matrix (of size equal to the number of channels), while in SC signal processing each frequency component in both cases is a scalar.
A multi-camera motion capture system for remote healthcare monitoring

This paper presents a multi-camera motion capture system aiming to provide caregivers with timely access to the patient's health status through mobile communication devices. The major components include video capture, object detection, video coding and transmission, error concealment, and video analysis. Our contribution is twofold. First, several novel ideas are developed, including fast object detection, and content-aware and adaptive video coding and transmission. Second, all components are seamlessly integrated in a unified optimization framework dedicated for online data transmission. In the scenario, the subject walked on a treadmill with four tripod cameras capturing the video from different viewpoints. After video compression and transmission over a wireless sensor network, the remote receiver recovered the videos and performed multi-view motion capture for gait analysis. Experimental results show that the presented system design achieves better video quality than traditional video coding and transmission scheme, while the requirement for a low-cost, noninvasive and real-time healthcare monitoring system is accommodated.