nils-herrmann/clean_scirepeval huggingface dataset:
Sound source separation via computational auditory scene analysis-enhanced beamforming

The paper describes a new approach to sound source separation, called computational auditory scene analysis-enhanced beamforming (CASA-EB). It achieves increased signal separation performance by combining beamforming with an auditory model (CASA). The motivation for combining them is that they are complementary, i.e., they use independent signal attributes, and as a result, CASA-EB can separate more types of mixtures. The efficacy of this approach is demonstrated by experimental results.
A rate control method for H.263 temporal scalability

In this paper a novel methodology for temporal scalability within the H.263 video coding standard is presented. Temporal scalability is defined as the transmission of previously dropped frames in the form of a scalable enhancement layer to increase the overall encoded frame rate. The proposed methodology extends the base layer rate control to the enhancement layer and incorporates an adaptive technique for enhancement layer frame selection. Three criteria important in the selection of enhancement frames are identified, allowing us to adaptively choose frames such that the overall temporal resolution of the encoded video sequence is enhanced. Experimental results are provided and compared to a non-adaptive technique where enhancement frames are selected solely on the decay of the enhancement layer buffer.
Advances in Efficient Resource Allocation for Packet-Based Real-Time Video Transmission

Multimedia applications involving the transmission of video over communication networks are rapidly increasing in popularity. Such applications can greatly benefit from adapting video coding parameters to network conditions as well as adapting network parameters to better support the application requirements. These two dimensions can both be viewed as allocating source and network resources to improve video quality. We highlight recent advances in optimal resource allocation for real-time video communications over unreliable and resource constrained communication channels. More specifically, we focus on point-to-point coding and delivery schemes in which the sequences are encoded on the fly. We present a high-level framework for resource-distortion optimization. The framework can be used for jointly considering factors across network layers, including source coding, channel resource allocation, and error concealment. For example, resources can take the form of transmission energy in a wireless channel, and transmission cost in a DiffServ-based Internet channel. This framework can be used to optimally trade off resource consumption with end-to-end video quality in packet-based video transmission. After giving an overview of this framework, we review recent work in two areas-energy efficient wireless video transmission and resource allocation for Internet-based applications.
Low-Complexity Tracking-Aware H.264 Video Compression for Transportation Surveillance

In centralized transportation surveillance systems, video is captured and compressed at low processing power remote nodes and transmitted to a central location for processing. Such compression can reduce the accuracy of centrally run automated object tracking algorithms. In typical systems, the majority of communications bandwidth is spent on encoding temporal pixel variations such as acquisition noise or local changes to lighting. We propose a tracking-aware, H.264-compliant compression algorithm that removes temporal components of low tracking interest and optimizes the quantization of frequency coefficients, particularly those that most influence trackers, significantly reducing bitrate while maintaining comparable tracking accuracy. We utilize tracking accuracy as our compression criterion in lieu of mean squared error metrics. Our proposed system is designed with low processing power and memory requirements in mind, and as such can be deployed on remote nodes. Using H.264/AVC video coding and a commonly used state-of-the-art tracker we show that our algorithm allows for over 90% bitrate savings while maintaining comparable tracking accuracy.
A regularized mixed norm multichannel image restoration approach

We develop a deterministic regularized mixed norm multichannel image restoration algorithm. A functional which combines the least mean squares (LMS), the least mean fourth (LMF), and a smoothing functional using both within- and between-channel deterministic information is proposed. One parameter is defined to control the relative contribution between the LMS and the LMF norms, and a second one (regularization parameter) is defined to control the degree of smoothness of the solution. They are both updated at each iteration step. The novelty of the proposed algorithm is that no knowledge about the noise distribution for each channel is required, and the parameters are adjusted based on the partially restored image.
Maximum a Posteriori Video Super-Resolution Using a New Multichannel Image Prior

Super-resolution (SR) is the term used to define the process of estimating a high-resolution (HR) image or a set of HR images from a set of low-resolution (LR) observations. In this paper we propose a class of SR algorithms based on the maximum a posteriori (MAP) framework. These algorithms utilize a new multichannel image prior model, along with the state-of-the-art single channel image prior and observation models. A hierarchical (two-level) Gaussian nonstationary version of the multichannel prior is also defined and utilized within the same framework. Numerical experiments comparing the proposed algorithms among themselves and with other algorithms in the literature, demonstrate the advantages of the adopted multichannel approach.
Optimized Bit Extraction Using Distortion Modeling in the Scalable Extension of H.264/AVC

The newly adopted scalable extension of H.264/AVC video coding standard (SVC) demonstrates significant improvements in coding efficiency in addition to an increased degree of supported scalability relative to the scalable profiles of prior video coding standards. Due to the complicated hierarchical prediction structure of the SVC and the concept of key pictures, content-aware rate adaptation of SVC bit streams to intermediate bit rates is a nontrivial task. The concept of quality layers has been introduced in the design of the SVC to allow for fast content-aware prioritized rate adaptation. However, existing quality layer assignment methods are suboptimal and do not consider all network abstraction layer (NAL) units from different layers for the optimization. In this paper, we first propose a technique to accurately and efficiently estimate the quality degradation resulting from discarding an arbitrary number of NAL units from multiple layers of a bitstream by properly taking drift into account. Then, we utilize this distortion estimation technique to assign quality layers to NAL units for a more efficient extraction. Experimental results show that a significant gain can be achieved by the proposed scheme.
Low bit rate video coding using robust motion vector regeneration in the decoder

In this paper, we present a novel coding technique that makes use of the nonstationary characteristics of an image sequence displacement field to estimate and encode motion information. We utilize an MPEG style codec in which the anchor frames in a sequence are encoded with a hybrid approach using quadtree, DCT, and wavelet-based coding techniques. A quadtree structured approach is also utilized for the interframe information. The main objective of the overall design is to demonstrate the coding potential of a newly developed motion estimator called the coupled linearized MAP (CLMAP) estimator. This estimator can be used as a means for producing motion vectors that may be regenerated at the decoder with a coarsely quantized error term created in the encoder. The motion estimator generates highly accurate motion estimates from this coarsely quantized data. This permits the elimination of a separately coded displaced frame difference (DFD) and coded motion vectors. For low bit rate applications, this is especially important because the overhead associated with the transmission of motion vectors may become prohibitive. We exploit both the advantages of the nonstationary motion estimator and the effective compression of the anchor frame coder to improve the visual quality of reconstructed QCIF format color image sequences at low bit rates. Comparisons are made with other video coding methods, including the H.261 and MPEG standards and a pel-recursive-based codec.
An HMM-based speech-to-video synthesizer

Emerging broadband communication systems promise a future of multimedia telephony, e.g. the addition of visual information to telephone conversations. It is useful to consider the problem of generating the critical information useful for speechreading, based on existing narrowband communications systems used for speech. This paper focuses on the problem of synthesizing visual articulatory movements given the acoustic speech signal. In this application, the acoustic speech signal is analyzed and the corresponding articulatory movements are synthesized for speechreading. This paper describes a hidden Markov model (HMM)-based visual speech synthesizer. The key elements in the application of HMMs to this problem are the decomposition of the overall modeling task into key stages and the judicious determination of the observation vector's components for each stage. The main contribution of this paper is a novel correlation HMM model that is able to integrate independently trained acoustic and visual HMMs for speech-to-visual synthesis. This model allows increased flexibility in choosing model topologies for the acoustic and visual HMMs. Moreover the propose model reduces the amount of training data compared to early integration modeling techniques. Results from objective experiments analysis show that the propose approach can reduce time alignment errors by 37.4% compared to conventional temporal scaling method. Furthermore, subjective results indicated that the purpose model can increase speech understanding.
Maximum likelihood image identification and restoration based on the EM algorithm

Simultaneous iterative identification and restoration have been treated. The image and the noise have been modeled as multivariate Gaussian processes. Maximum-likelihood estimation has been used to estimate the parameters that characterize the Gaussian processes, where the estimation of the conditional mean of the image represents the restored image. Likelihood functions of observed images are highly nonlinear with respect to these parameters. Therefore, it is in general very difficult to maximize them directly. The expectation-maximization (EM) algorithm has been used to find these parameters.<<ETX>>
Analysis of High-Dimensional Phase Space via Poincaré Section for Patient-Specific Seizure Detection.

In this paper, the performance of the phase space representation in interpreting the underlying dynamics of epileptic seizures is investigated and a novel patient-specific seizure detection approach is proposed based on the dynamics of EEG signals. To accomplish this, the trajectories of seizure and nonseizure segments are reconstructed in a high dimensional space using time-delay embedding method.  Principal Component Analysis (PCA) was used in order to reduce the dimension of the reconstructed phase spaces. The geometry of the trajectories in the lower dimensions is then characterized using Poincaré section and seven features were extracted from the obtained intersection sequence. Once the features are formed, they are fed into a two-layer classification scheme, comprising the Linear Discriminant Analysis (LDA) and Naive Bayesian classifiers. The performance of the proposed method is then evaluated over the CHB-MIT benchmark database and the proposed approach achieved 88.27% sensitivity and 93.21% specificity on average with 25% training data. Finally, we perform comparative performance evaluations against the state-of-the-art methods in this domain which demonstrate the superiority of the proposed method.
Motion estimation of skeletonized angiographic images using elastic registration

An approach for estimating the motion of arteries in digital angiographic image sequences is proposed. Binary skeleton images are registered using an elastic registration algorithm in order to estimate the motion of the corresponding arteries. This algorithm operates recursively on the skeleton images by considering an autoregressive (AR) model of the deformation in conjunction with a dynamic programming (DP) algorithm. The AR model is used at the pixel level and provides a suitable cost function to DP through the innovation process. In addition, a moving average (MA) model for the motion of the entire skeleton is used in combination with the local AR model for improved registration results. The performance of this motion estimation method is demonstrated on simulated and real digital angiographic image sequences. It is shown that motion estimation using elastic registration of skeletons is very successful especially with low contrast and noisy angiographic images.
Energy efficient wireless transmission of MPEG-4 fine granular scalable video

Fine granular scalability is a coding tool, recently introduced in the emerging MPEG-4 standard, which enables the creation of very flexible scalable video bitstreams. This paper investigates the transmission of fine granular scalable (FGS) video over wireless links, using power management for unequal error protection of the bitstream. In wireless systems, energy may be a limited resource, and a wise use of it is important for system efficiency. An algorithm is proposed which is able to optimally distribute the total available power for the transmission of the enhancement layer, given a distortion or energy constraint. Experimental results demonstrate the performance advantage of the proposed algorithm over fixed power schemes and heuristic approaches.
A class of robust entropic functionals for image restoration

This paper considers the concept of robust estimation in regularized image restoration. Robust functionals are employed for the representation of both the noise and the signal statistics. Such functionals allow the efficient suppression of a wide variety of noise processes and permit the reconstruction of sharper edges than their quadratic counterparts. A new class of robust entropic functionals is introduced, which operates only on the high-frequency content of the signal and reflects sharp deviations in the signal distribution. This class of functionals can also incorporate prior structural information regarding the original image, in a way similar to the maximum information principle. The convergence properties of robust iterative algorithms are studied for continuously and noncontinuously differentiable functionals. The definition of the robust approach is completed by introducing a method for the optimal selection of the regularization parameter. This method utilizes the structure of robust estimators that lack analytic specification. The properties of robust algorithms are demonstrated through restoration examples in different noise environments.
Vector quantization with memory and multi-labeling for isolated video-only automatic speech recognition

We describe a vector quantizer (VQ) with memory for automatic speech recognition (ASR) and compare the recognition performance results to those obtained with traditional memoryless VQ for ASR. Standard VQ for ASR quantizes the speech data independently of any past information. We introduce memory in a probabilistic framework for quantization state modeling. This is accomplished in the form of an ergodic hidden Markov model (HMM) in which the state occupied by the HMM represents the quantization label. We evaluate this approach in the context of video-only isolated digit ASR and implement both single stream (single labeling) and multi-stream (multi-labeling) systems. For single stream recognition, our approach increases the recognition rate from 62.67% to 66.95%. When using multi-labeling, our proposed vector quantizer with memory consistently outperforms the memoryless vector quantizer.
Bayesian Super-Resolution image reconstruction using an ℓ1 prior

This paper deals with the problem of high-resolution (HR) image reconstruction, from a set of degraded, under-sampled, shifted and rotated images, under the Bayesian paradigm, utilizing a variational approximation. Bayesian methods rely on image models that encapsulate prior image knowledge and avoid the ill-posedness of the image restoration problems. In this paper a new prior based on the lscr1 norm of vertical and horizontal first order differences of image pixel values is introduced and its parameters are estimated. The estimated HR images are compared with images provided by other HR reconstruction methods.
Design of benchmark imagery for validating facility annotation algorithms

The design of benchmark imagery for validation of image annotation algorithms is considered. Emphasis is placed on imagery that contains industrial facilities, such as chemical refineries. An application-level facility ontology is used as a means to define salient objects in the benchmark imagery. In-strinsic and extrinsic scene factors important for comprehensive validation are listed, and variability in the benchmarks discussed. Finally, the pros and cons of three forms of benchmark imagery: real, composite and synthetic, are delineated.
An adaptive regularized recursive displacement estimation algorithm

An adaptive regularized recursive displacement estimation algorithm is presented. An estimate of the displacement vector field (DVF) is obtained by minimizing the linearized displaced frame difference (DFD) using nu subsets (submasks) of a set of points that belong to a causal neighborhood (mask) around the working point. Assuming that the displacement vector is constant at all points inside the mask, nu systems of equations are formed based on the corresponding submasks. A set theoretic regularization approach is followed for solving this system of equations by using information about the noise and the solution. An expression for the variance of the linearization error is derived in quantifying the information about the noise. Prior information about the solution is incorporated into the algorithm using a causal oriented smoothness constraint (OSC) which also provides a spatially adaptive prediction model for the estimation DVF. It is shown that certain existing regularized recursive algorithms are special cases of the proposed algorithm, if a single mask is considered. Based on experiments with typical videoconferencing scenes, the improved performance of the proposed algorithm with respect to accuracy, robustness to occlusion and smoothness of the estimated DVF is demonstrated.
Product HMMs for audio-visual continuous speech recognition using facial animation parameters

The use of visual information in addition to acoustic can improve automatic speech recognition. In this paper we compare different approaches for audio-visual information integration and show how they affect automatic speech recognition performance. We utilize facial animation parameters (FAPs), supported by the MPEG-4 standard for the visual representation as visual features. We use both single-stream and multi-stream hidden Markov models (HMM) to integrate audio and visual information. We performed both state and phone synchronous multi-stream integration. Product HMM topology is used to model the phone-synchronous integration. ASR experiments were performed under noisy audio conditions using a relatively large vocabulary (approximately 1000 words) audio-visual database. The proposed phone-synchronous system, which performed the best, reduces the word error rate (WER) by approximately 20% relatively to audio-only ASR (A-ASR) WERs, at various SNRs with additive white Gaussian noise.
Resource Allocation for Downlink Multiuser Video Transmission Over Wireless Lossy Networks

Demand for multimedia services, such as video streaming over wireless networks, has grown dramatically in recent years. The downlink transmission of multiple video sequences to multiple users over a shared resource-limited wireless channel, however, is a daunting task. Among the many challenges in this area are the time-varying channel conditions, limited available resources, such as bandwidth and power, and the different transmission requirements of different video content. This work takes into account the time-varying nature of the wireless channels, as well as the importance of individual video packets, to develop a cross-layer resource allocation and packet scheduling scheme for multiuser video streaming over lossy wireless packet access networks. Assuming that accurate channel feedback is not available at the scheduler, random channel losses combined with complex error concealment at the receiver make it impossible for the scheduler to determine the actual distortion of the sequence at the receiver. Therefore, the objective of the optimization is to minimize the expected distortion of the received sequence, where the expectation is calculated at the scheduler with respect to the packet loss probability in the channel. The expected distortion is used to order the packets in the transmission queue of each user, and then gradients of the expected distortion are used to efficiently allocate resources across users. Simulations show that the proposed scheme performs significantly better than a conventional content-independent scheme for video transmission.
Two methods for least squares multi-channel image restoration

The problem of multi-channel restoration using both within and between-channel deterministic information is considered. A multi-channel image is a set of image planes that exhibit cross-plane similarity. Existing optimal restoration filters for single-plane images will yield suboptimal results when applied to multi-channel images, since between-channel information is not utilized. Multi-channel least squares restoration filters are developed using two approaches, the set theoretic and the constrained optimization. A geometric interpretation of the estimates of both filters is given. Color images, that is three-channel imagery with red, green, and blue components, are considered. Constraints that capture the within and between-channel properties of color images are developed. Issues associated with the computation of the two estimates are addressed. Finally, experiments using color images are shown.<<ETX>>
Compressed sensing super resolution of color images

In this work we estimate Super Resolution (SR) images from a sequence of true color Compressed Sensing (CS) observations. The red, green, blue (RGB) channels are sensed separately using a measurement matrix that can be synthesized practically. The joint optimization problem to estimate the registration parameters, and the High Resolution (HR) image is transformed into a sequence of unconstrained optimization sub-problems using the Alternate Direction Method of Multipliers (ADMM). A new, simple, and accurate, image registration procedure is proposed. The performed experiments show that the proposed method compares favorably to existing color CS reconstruction methods at unity zooming factor (P), obtaining very good performance varying P and the compression factor simultaneously. The algorithm is tested on real and synthetic images.